2024-03-18T14:29:43.997 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T14:29:43.997 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --verbose --archive /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837 -- /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/orig.config.yaml 2024-03-18T14:29:44.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T14:29:44.143 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837 branch: wip-batrick-testing-20240314.183243 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7608837' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - mons down, quorum - out of quorum - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8b+VFjfnKLwEQ6uY3JcaOTZH5yxx6WL+C+zBe1SIW8UO/HnHcyx2nfIhl4/QqVIeuGlyTWAdcypvujXZVRiKs= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTRjhLugTLlle1QVsYXcxdanH8C8KI6mwzrbyxYF4QNxlQSXCZlguETbBttISi1gL0qJ06WbHJ7J/XmH25BsSo= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_14:19:46 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-18T14:29:44.143 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa; will attempt to use it 2024-03-18T14:29:44.144 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks 2024-03-18T14:29:44.144 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T14:29:44.146 INFO:teuthology.task.internal:Checking packages... 2024-03-18T14:29:44.169 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-18T14:29:44.169 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:29:44.169 INFO:teuthology.packaging:ref: None 2024-03-18T14:29:44.169 INFO:teuthology.packaging:tag: None 2024-03-18T14:29:44.169 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-18T14:29:44.169 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:29:44.169 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-18T14:29:44.400 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-18T14:29:44.402 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T14:29:44.408 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T14:29:44.408 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T14:29:44.415 INFO:teuthology.task.internal:Saving configuration 2024-03-18T14:29:44.435 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T14:29:44.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T14:29:44.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 14:21:21.435928', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8b+VFjfnKLwEQ6uY3JcaOTZH5yxx6WL+C+zBe1SIW8UO/HnHcyx2nfIhl4/QqVIeuGlyTWAdcypvujXZVRiKs='} 2024-03-18T14:29:44.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 14:21:21.433729', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTRjhLugTLlle1QVsYXcxdanH8C8KI6mwzrbyxYF4QNxlQSXCZlguETbBttISi1gL0qJ06WbHJ7J/XmH25BsSo='} 2024-03-18T14:29:44.488 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T14:29:44.495 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-18T14:29:44.495 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-18T14:29:44.495 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T14:29:44.560 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd05f9c8160>, signals=[15]) 2024-03-18T14:29:44.561 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T14:29:44.566 INFO:teuthology.task.internal:Opening connections... 2024-03-18T14:29:44.566 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:29:44.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:29:44.670 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-03-18T14:29:44.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:29:44.756 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T14:29:44.763 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-03-18T14:29:44.781 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-03-18T14:29:44.781 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-03-18T14:29:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T14:29:44.839 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T14:29:44.839 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T14:29:44.839 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-03-18T14:29:44.857 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-18T14:29:44.876 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-18T14:29:44.876 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-18T14:29:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-03-18T14:29:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-03-18T14:29:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T14:29:44.933 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T14:29:44.933 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-18T14:29:44.949 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T14:29:44.958 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T14:29:45.012 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T14:29:45.012 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-03-18T14:29:45.014 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-03-18T14:29:45.030 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T14:29:45.037 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T14:29:45.037 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-03-18T14:29:45.071 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-03-18T14:29:45.105 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T14:29:45.161 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T14:29:45.176 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-18T14:29:45.176 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-18T14:29:45.176 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T14:29:45.176 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T14:29:45.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:29:45.193 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-18T14:29:45.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:29:45.194 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-03-18T14:29:45.248 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2024-03-18T14:29:45.249 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-361.el9.x86_64 2024-03-18T14:29:45.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-03-18T14:29:45.252 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-361.el9.x86_64 2024-03-18T14:29:45.252 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-361.el9.x86_64 2024-03-18T14:29:45.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-03-18T14:29:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-18T14:29:47.476 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-18T14:29:48.893 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-18T14:29:49.110 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-18T14:29:49.909 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-18T14:29:49.994 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-18T14:29:52.202 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-18T14:29:56.397 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.1 MB 00:00 2024-03-18T14:29:57.862 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-03-18T14:30:00.738 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-03-18T14:30:05.612 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-18T14:30:07.802 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T14:30:07.803 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T14:30:07.890 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-18T14:30:07.895 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 100 M 2024-03-18T14:30:07.896 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 123 M 2024-03-18T14:30:07.897 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:30:09.381 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 4.4 MB/s | 6.0 MB 00:01 2024-03-18T14:30:11.164 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 6.6 MB/s | 21 MB 00:03 2024-03-18T14:30:13.272 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 7.5 MB/s | 39 MB 00:05 2024-03-18T14:30:13.338 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 867 kB/s | 19 MB 00:22 2024-03-18T14:30:13.664 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 7.8 MB/s | 33 MB 00:04 2024-03-18T14:30:13.664 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:30:13.664 INFO:teuthology.orchestra.run.smithi100.stdout:Total 17 MB/s | 100 MB 00:05 2024-03-18T14:30:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:30:14.930 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:30:14.930 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:30:16.309 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:30:16.309 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:30:17.505 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-18T14:30:19.061 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:30:19.106 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-03-18T14:30:19.754 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-18T14:30:19.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:30:20.851 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-18T14:30:22.631 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:30:22.836 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:30:25.865 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-18T14:30:27.600 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:30:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:30:28.130 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T14:30:28.130 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T14:30:28.227 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:30:28.236 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:30:28.236 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-03-18T14:30:28.236 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:30:28.236 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:30:28.236 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout:Install 4 Packages 2024-03-18T14:30:28.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:30:28.238 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 100 M 2024-03-18T14:30:28.238 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 123 M 2024-03-18T14:30:28.238 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:30:28.667 INFO:teuthology.orchestra.run.smithi012.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-03-18T14:30:29.142 INFO:teuthology.orchestra.run.smithi012.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 26 MB/s | 21 MB 00:00 2024-03-18T14:30:29.733 INFO:teuthology.orchestra.run.smithi012.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 28 MB/s | 39 MB 00:01 2024-03-18T14:30:30.334 INFO:teuthology.orchestra.run.smithi012.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 20 MB/s | 33 MB 00:01 2024-03-18T14:30:30.334 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:30:30.334 INFO:teuthology.orchestra.run.smithi012.stdout:Total 47 MB/s | 100 MB 00:02 2024-03-18T14:30:31.259 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:30:31.707 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:30:31.708 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:30:32.544 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:30:33.096 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:30:33.096 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:30:35.950 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:30:36.631 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-18T14:30:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:30:39.726 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:30:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:30:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:30:45.141 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:01.666 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:01.667 INFO:teuthology.orchestra.run.smithi100.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T14:31:01.667 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:31:01.751 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:02.153 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:02.153 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-18T14:31:02.153 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:31:02.153 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-18T14:31:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:31:02.580 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:31:03.117 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T14:31:03.296 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:03.296 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T14:31:03.296 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-18T14:31:03.297 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T14:31:03.297 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T14:31:03.297 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-18T14:31:03.297 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T14:31:03.297 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T14:31:03.418 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:03.418 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T14:31:03.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-03-18T14:31:03.456 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-03-18T14:31:03.457 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:31:03.458 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T14:31:03.458 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T14:31:03.458 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-427.el9.x86_64 2024-03-18T14:31:03.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T14:31:04.073 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-03-18T14:31:05.507 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T14:31:05.548 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-03-18T14:31:05.551 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-03-18T14:31:05.566 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.ikVKPB18xA 2024-03-18T14:31:05.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ikVKPB18xA 2024-03-18T14:31:05.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.ikVKPB18xA 2024-03-18T14:31:05.765 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.ikVKPB18xA is 6KB 2024-03-18T14:31:05.813 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.ikVKPB18xA 2024-03-18T14:31:05.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T14:31:05.891 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T14:31:05.891 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T14:31:05.891 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T14:31:05.891 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-18T14:31:05.892 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-18T14:31:06.034 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-03-18T14:31:20.901 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:20.901 INFO:teuthology.orchestra.run.smithi012.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T14:31:20.901 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:31:20.990 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:21.416 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:21.416 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-18T14:31:21.417 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T14:31:21.417 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T14:31:21.820 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:31:21.821 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:31:22.360 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T14:31:22.515 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:22.515 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T14:31:22.515 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-18T14:31:22.515 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T14:31:22.515 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T14:31:22.515 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-18T14:31:22.515 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T14:31:22.516 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T14:31:22.642 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T14:31:22.642 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T14:31:22.643 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-03-18T14:31:22.690 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-03-18T14:31:22.690 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 68.el9 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8264254 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-03-18T14:31:22.691 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:31:22.692 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T14:31:22.693 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T14:31:22.693 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 5.14.0-427.el9.x86_64 2024-03-18T14:31:22.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T14:31:23.259 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-03-18T14:31:24.604 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T14:31:24.624 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-03-18T14:31:24.626 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-03-18T14:31:24.641 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.SXGnYK7qfL 2024-03-18T14:31:24.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SXGnYK7qfL 2024-03-18T14:31:24.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.SXGnYK7qfL 2024-03-18T14:31:24.840 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.SXGnYK7qfL is 6KB 2024-03-18T14:31:24.888 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.SXGnYK7qfL 2024-03-18T14:31:24.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T14:31:24.972 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T14:31:24.972 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T14:31:24.972 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T14:31:24.972 INFO:teuthology.orchestra.run.smithi012.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-18T14:31:24.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-18T14:31:25.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-03-18T14:31:36.046 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-18T14:31:36.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:31:36.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:31:54.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-18T14:31:55.112 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-18T14:31:55.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-03-18T14:31:55.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:32:03.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:32:03.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:32:06.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-18T14:32:13.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-03-18T14:32:18.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:32:18.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:32:22.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-03-18T14:32:22.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:32:36.898 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-18T14:32:51.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:32:51.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:32:52.310 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-18T14:32:52.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:32:52.813 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-427.el9.x86_64"... 2024-03-18T14:32:52.813 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-18T14:32:52.827 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-427.el9.x86_64 2024-03-18T14:32:52.828 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-18T14:32:52.828 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T14:32:52.828 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T14:32:53.829 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-18T14:32:53.829 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T14:32:53.954 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-03-18T14:32:54.028 DEBUG:teuthology.parallel:result is None 2024-03-18T14:32:54.534 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-03-18T14:32:55.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-03-18T14:32:55.046 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-427.el9.x86_64"... 2024-03-18T14:32:55.047 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-03-18T14:32:55.060 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-427.el9.x86_64 2024-03-18T14:32:55.060 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-18T14:32:55.060 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T14:32:55.060 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T14:32:56.062 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-18T14:32:56.062 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T14:32:56.118 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-03-18T14:32:56.156 DEBUG:teuthology.parallel:result is None 2024-03-18T14:32:56.156 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T14:32:56.170 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T14:32:56.170 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T14:32:56.172 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T14:32:56.186 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T14:32:56.193 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T14:32:56.204 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T14:32:56.204 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T14:32:56.227 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T14:32:56.262 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T14:32:56.272 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T14:32:56.272 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T14:32:56.286 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T14:32:56.315 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T14:32:56.324 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T14:32:56.328 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T14:32:56.337 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T14:32:56.338 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T14:32:56.346 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T14:32:56.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T14:32:56.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T14:32:56.409 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T14:32:56.419 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T14:32:56.419 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T14:32:56.446 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T14:32:56.468 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T14:32:56.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T14:32:56.609 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T14:32:56.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T14:32:56.740 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:32:56.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T14:32:56.812 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T14:32:56.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T14:32:56.911 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T14:32:56.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T14:32:57.040 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:32:57.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T14:32:57.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-03-18T14:32:57.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-03-18T14:32:57.155 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T14:32:57.198 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T14:32:57.661 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T14:32:57.669 INFO:teuthology.task.internal:Starting timer... 2024-03-18T14:32:57.669 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T14:32:57.681 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T14:32:57.690 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-18T14:32:57.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-03-18T14:32:57.753 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-03-18T14:32:57.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-03-18T14:32:57.821 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-03-18T14:32:57.823 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T14:32:57.823 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-03-18T14:32:57.842 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-03-18T14:32:57.842 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-03-18T14:32:57.867 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-03-18T14:32:57.867 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2024-03-18T14:32:57.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T14:32:57.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:32:57.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T14:32:57.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:32:57.941 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T14:32:57.942 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-03-18T14:32:57.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-03-18T14:32:58.027 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T14:32:58.039 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T14:32:58.057 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T14:32:58.058 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T14:37:41.428 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-03-18T14:37:41.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-03-18T14:37:41.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:37:41.547 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-03-18T14:37:41.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-03-18T14:37:41.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:37:41.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T14:37:41.731 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-18T14:37:41.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-18T14:37:41.816 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T14:37:41.827 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T14:37:41.827 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T14:37:41.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T14:37:41.843 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T14:37:41.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T14:37:41.869 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T14:37:41.885 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T14:37:41.889 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T14:37:41.907 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T14:37:41.914 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-03-18T14:37:41.930 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-03-18T14:37:41.938 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-03-18T14:37:41.944 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T14:37:41.951 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-03-18T14:37:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T14:37:41.968 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T14:37:41.985 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T14:37:42.015 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.033 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-18T14:37:42.036 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T14:37:42.037 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-18T14:37:42.037 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.037 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.037 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.037 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T14:37:42.037 INFO:teuthology.run_tasks:Running task install... 2024-03-18T14:37:42.047 DEBUG:teuthology.task.install:project ceph 2024-03-18T14:37:42.048 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-18T14:37:42.048 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-18T14:37:42.048 INFO:teuthology.task.install:Using flavor: default 2024-03-18T14:37:42.055 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T14:37:42.055 INFO:teuthology.task.install:extra packages: [] 2024-03-18T14:37:42.056 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-18T14:37:42.056 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:37:42.056 INFO:teuthology.packaging:ref: None 2024-03-18T14:37:42.056 INFO:teuthology.packaging:tag: None 2024-03-18T14:37:42.056 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:37:42.056 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:37:42.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-18T14:37:42.065 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-18T14:37:42.065 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:37:42.065 INFO:teuthology.packaging:ref: None 2024-03-18T14:37:42.065 INFO:teuthology.packaging:tag: None 2024-03-18T14:37:42.066 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:37:42.066 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:37:42.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-18T14:37:42.341 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/ 2024-03-18T14:37:42.356 INFO:teuthology.task.install.rpm:Package version is 17.2.7-684.g6716c9f3 2024-03-18T14:37:42.459 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/ 2024-03-18T14:37:42.459 INFO:teuthology.task.install.rpm:Package version is 17.2.7-684.g6716c9f3 2024-03-18T14:37:42.532 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T14:37:42.532 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:37:42.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T14:37:42.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-18T14:37:42.562 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:37:42.562 INFO:teuthology.packaging:ref: None 2024-03-18T14:37:42.562 INFO:teuthology.packaging:tag: None 2024-03-18T14:37:42.562 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:37:42.562 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:37:42.562 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T14:37:42.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T14:37:42.711 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T14:37:42.735 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/6716c9f3976cf9fc2fd546e9f1182c2d4421b451/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T14:37:42.736 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:37:42.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T14:37:42.766 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-18T14:37:42.767 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:37:42.767 INFO:teuthology.packaging:ref: None 2024-03-18T14:37:42.767 INFO:teuthology.packaging:tag: None 2024-03-18T14:37:42.767 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:37:42.767 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:37:42.767 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T14:37:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-03-18T14:37:42.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-03-18T14:37:42.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T14:37:42.917 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T14:37:42.986 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-03-18T14:37:42.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-03-18T14:37:43.022 INFO:teuthology.orchestra.run.smithi012.stdout:74 files removed 2024-03-18T14:37:43.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw 2024-03-18T14:37:43.224 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-03-18T14:37:43.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw 2024-03-18T14:37:43.700 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 235 kB/s | 77 kB 00:00 2024-03-18T14:37:44.203 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 122 kB/s | 77 kB 00:00 2024-03-18T14:37:44.310 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 16 kB/s | 9.2 kB 00:00 2024-03-18T14:37:44.771 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 18 kB/s | 9.2 kB 00:00 2024-03-18T14:37:44.877 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 3.4 kB/s | 1.8 kB 00:00 2024-03-18T14:37:45.354 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 3.3 kB/s | 1.8 kB 00:00 2024-03-18T14:37:45.537 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-18T14:37:45.882 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-18T14:37:47.038 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-18T14:37:47.558 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-18T14:37:48.900 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-03-18T14:37:49.861 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.0 MB 00:02 2024-03-18T14:37:53.966 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-03-18T14:37:55.170 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 19 MB 00:04 2024-03-18T14:37:57.966 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-18T14:37:59.900 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-03-18T14:38:01.010 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-18T14:38:01.579 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-18T14:38:03.212 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-18T14:38:04.700 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-18T14:38:05.918 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-18T14:38:08.471 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:38:08.484 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:08.484 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:38:08.484 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:08.484 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 14 M 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 3.1 M 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.9 M 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 5.0 M 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 23 M 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 25 k 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 638 k 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T14:38:08.485 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 457 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 4.9 M 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 45 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 117 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 156 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 320 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 293 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 99 k 2024-03-18T14:38:08.486 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:Install 22 Packages 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 2 Packages 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 58 M 2024-03-18T14:38:08.487 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:38:09.789 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-18T14:38:10.352 INFO:teuthology.orchestra.run.smithi100.stdout:(1/24): ceph-radosgw-17.2.7-684.g6716c9f3.el9.x 8.4 MB/s | 14 MB 00:01 2024-03-18T14:38:10.436 INFO:teuthology.orchestra.run.smithi100.stdout:(2/24): ceph-base-17.2.7-684.g6716c9f3.el9.x86_ 2.9 MB/s | 5.0 MB 00:01 2024-03-18T14:38:10.461 INFO:teuthology.orchestra.run.smithi100.stdout:(3/24): ceph-selinux-17.2.7-684.g6716c9f3.el9.x 231 kB/s | 25 kB 00:00 2024-03-18T14:38:10.670 INFO:teuthology.orchestra.run.smithi100.stdout:(4/24): libradosstriper1-17.2.7-684.g6716c9f3.e 2.1 MB/s | 457 kB 00:00 2024-03-18T14:38:10.695 INFO:teuthology.orchestra.run.smithi100.stdout:(5/24): libcephfs2-17.2.7-684.g6716c9f3.el9.x86 2.4 MB/s | 638 kB 00:00 2024-03-18T14:38:10.804 INFO:teuthology.orchestra.run.smithi100.stdout:(6/24): python3-ceph-argparse-17.2.7-684.g6716c 416 kB/s | 45 kB 00:00 2024-03-18T14:38:10.921 INFO:teuthology.orchestra.run.smithi100.stdout:(7/24): python3-ceph-common-17.2.7-684.g6716c9f 1.0 MB/s | 117 kB 00:00 2024-03-18T14:38:11.063 INFO:teuthology.orchestra.run.smithi100.stdout:(8/24): python3-cephfs-17.2.7-684.g6716c9f3.el9 1.1 MB/s | 156 kB 00:00 2024-03-18T14:38:11.230 INFO:teuthology.orchestra.run.smithi100.stdout:(9/24): python3-rados-17.2.7-684.g6716c9f3.el9. 1.9 MB/s | 320 kB 00:00 2024-03-18T14:38:11.430 INFO:teuthology.orchestra.run.smithi100.stdout:(10/24): librgw2-17.2.7-684.g6716c9f3.el9.x86_6 6.5 MB/s | 4.9 MB 00:00 2024-03-18T14:38:11.464 INFO:teuthology.orchestra.run.smithi100.stdout:(11/24): python3-rbd-17.2.7-684.g6716c9f3.el9.x 1.2 MB/s | 293 kB 00:00 2024-03-18T14:38:11.572 INFO:teuthology.orchestra.run.smithi100.stdout:(12/24): python3-rgw-17.2.7-684.g6716c9f3.el9.x 700 kB/s | 99 kB 00:00 2024-03-18T14:38:11.822 INFO:teuthology.orchestra.run.smithi100.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 444 kB/s | 159 kB 00:00 2024-03-18T14:38:12.081 INFO:teuthology.orchestra.run.smithi100.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 87 kB/s | 44 kB 00:00 2024-03-18T14:38:12.114 INFO:teuthology.orchestra.run.smithi100.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 661 kB 00:00 2024-03-18T14:38:12.223 INFO:teuthology.orchestra.run.smithi100.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 282 kB 00:00 2024-03-18T14:38:12.240 INFO:teuthology.orchestra.run.smithi100.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 329 kB/s | 41 kB 00:00 2024-03-18T14:38:12.527 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:38:12.532 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:12.532 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:38:12.532 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 14 M 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 3.1 M 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.9 M 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 5.0 M 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 23 M 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 25 k 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 638 k 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T14:38:12.533 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 457 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 4.9 M 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 45 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 117 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 156 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 320 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 293 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 99 k 2024-03-18T14:38:12.534 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout:Install 22 Packages 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 2 Packages 2024-03-18T14:38:12.535 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:12.538 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 58 M 2024-03-18T14:38:12.538 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:38:12.565 INFO:teuthology.orchestra.run.smithi100.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 149 kB/s | 49 kB 00:00 2024-03-18T14:38:12.732 INFO:teuthology.orchestra.run.smithi100.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 628 kB/s | 309 kB 00:00 2024-03-18T14:38:12.766 INFO:teuthology.orchestra.run.smithi100.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 336 kB/s | 67 kB 00:00 2024-03-18T14:38:13.298 INFO:teuthology.orchestra.run.smithi012.stdout:(1/24): ceph-base-17.2.7-684.g6716c9f3.el9.x86_ 7.1 MB/s | 5.0 MB 00:00 2024-03-18T14:38:13.356 INFO:teuthology.orchestra.run.smithi012.stdout:(2/24): ceph-selinux-17.2.7-684.g6716c9f3.el9.x 434 kB/s | 25 kB 00:00 2024-03-18T14:38:13.457 INFO:teuthology.orchestra.run.smithi012.stdout:(3/24): libcephfs2-17.2.7-684.g6716c9f3.el9.x86 6.2 MB/s | 638 kB 00:00 2024-03-18T14:38:13.549 INFO:teuthology.orchestra.run.smithi012.stdout:(4/24): libradosstriper1-17.2.7-684.g6716c9f3.e 4.9 MB/s | 457 kB 00:00 2024-03-18T14:38:13.766 INFO:teuthology.orchestra.run.smithi100.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-03-18T14:38:14.066 INFO:teuthology.orchestra.run.smithi100.stdout:(22/24): librados2-17.2.7-684.g6716c9f3.el9.x86 2.4 MB/s | 3.1 MB 00:01 2024-03-18T14:38:14.091 INFO:teuthology.orchestra.run.smithi012.stdout:(5/24): librgw2-17.2.7-684.g6716c9f3.el9.x86_64 9.1 MB/s | 4.9 MB 00:00 2024-03-18T14:38:14.158 INFO:teuthology.orchestra.run.smithi012.stdout:(6/24): python3-ceph-argparse-17.2.7-684.g6716c 680 kB/s | 45 kB 00:00 2024-03-18T14:38:14.233 INFO:teuthology.orchestra.run.smithi012.stdout:(7/24): python3-ceph-common-17.2.7-684.g6716c9f 1.5 MB/s | 117 kB 00:00 2024-03-18T14:38:14.300 INFO:teuthology.orchestra.run.smithi012.stdout:(8/24): python3-cephfs-17.2.7-684.g6716c9f3.el9 2.3 MB/s | 156 kB 00:00 2024-03-18T14:38:14.375 INFO:teuthology.orchestra.run.smithi012.stdout:(9/24): python3-rados-17.2.7-684.g6716c9f3.el9. 4.2 MB/s | 320 kB 00:00 2024-03-18T14:38:14.408 INFO:teuthology.orchestra.run.smithi100.stdout:(23/24): librbd1-17.2.7-684.g6716c9f3.el9.x86_6 4.6 MB/s | 2.9 MB 00:00 2024-03-18T14:38:14.601 INFO:teuthology.orchestra.run.smithi012.stdout:(10/24): ceph-common-17.2.7-684.g6716c9f3.el9.x 11 MB/s | 23 MB 00:02 2024-03-18T14:38:14.626 INFO:teuthology.orchestra.run.smithi012.stdout:(11/24): python3-rbd-17.2.7-684.g6716c9f3.el9.x 1.1 MB/s | 293 kB 00:00 2024-03-18T14:38:14.651 INFO:teuthology.orchestra.run.smithi012.stdout:(12/24): python3-rgw-17.2.7-684.g6716c9f3.el9.x 1.9 MB/s | 99 kB 00:00 2024-03-18T14:38:14.935 INFO:teuthology.orchestra.run.smithi012.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 156 kB/s | 44 kB 00:00 2024-03-18T14:38:14.985 INFO:teuthology.orchestra.run.smithi012.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 444 kB/s | 159 kB 00:00 2024-03-18T14:38:15.244 INFO:teuthology.orchestra.run.smithi012.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 661 kB 00:00 2024-03-18T14:38:15.269 INFO:teuthology.orchestra.run.smithi012.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 997 kB/s | 282 kB 00:00 2024-03-18T14:38:15.294 INFO:teuthology.orchestra.run.smithi012.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 826 kB/s | 41 kB 00:00 2024-03-18T14:38:15.886 INFO:teuthology.orchestra.run.smithi012.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 82 kB/s | 49 kB 00:00 2024-03-18T14:38:16.103 INFO:teuthology.orchestra.run.smithi012.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 311 kB/s | 67 kB 00:00 2024-03-18T14:38:16.212 INFO:teuthology.orchestra.run.smithi012.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 328 kB/s | 309 kB 00:00 2024-03-18T14:38:16.479 INFO:teuthology.orchestra.run.smithi012.stdout:(21/24): librados2-17.2.7-684.g6716c9f3.el9.x86 12 MB/s | 3.1 MB 00:00 2024-03-18T14:38:16.646 INFO:teuthology.orchestra.run.smithi012.stdout:(22/24): librbd1-17.2.7-684.g6716c9f3.el9.x86_6 18 MB/s | 2.9 MB 00:00 2024-03-18T14:38:17.283 INFO:teuthology.orchestra.run.smithi100.stdout:(24/24): ceph-common-17.2.7-684.g6716c9f3.el9.x 2.6 MB/s | 23 MB 00:08 2024-03-18T14:38:17.287 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:38:17.287 INFO:teuthology.orchestra.run.smithi100.stdout:Total 6.6 MB/s | 58 MB 00:08 2024-03-18T14:38:17.288 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:38:17.321 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:38:17.321 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:38:17.786 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:38:17.786 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:38:18.954 INFO:teuthology.orchestra.run.smithi012.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 568 kB/s | 1.6 MB 00:02 2024-03-18T14:38:18.988 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:38:19.331 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-18T14:38:19.393 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:19.484 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:19.568 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 3/26 2024-03-18T14:38:19.594 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:19.667 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-18T14:38:19.749 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-18T14:38:20.186 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-18T14:38:20.264 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 8/26 2024-03-18T14:38:20.427 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 8/26 2024-03-18T14:38:20.456 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-18T14:38:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x 10/26 2024-03-18T14:38:20.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 11/26 2024-03-18T14:38:20.545 INFO:teuthology.orchestra.run.smithi012.stdout:(24/24): ceph-radosgw-17.2.7-684.g6716c9f3.el9. 1.8 MB/s | 14 MB 00:07 2024-03-18T14:38:20.552 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:38:20.552 INFO:teuthology.orchestra.run.smithi012.stdout:Total 7.2 MB/s | 58 MB 00:08 2024-03-18T14:38:20.553 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:38:20.593 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:38:20.593 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:38:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 12/26 2024-03-18T14:38:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:20.699 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:20.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-18T14:38:20.775 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-18T14:38:20.808 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-18T14:38:21.078 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:38:21.078 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:38:21.101 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-18T14:38:21.138 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 18/26 2024-03-18T14:38:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 18/26 2024-03-18T14:38:21.291 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 19/26 2024-03-18T14:38:21.323 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86 20/26 2024-03-18T14:38:22.327 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:38:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-18T14:38:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:22.832 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:22.930 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 3/26 2024-03-18T14:38:22.966 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:23.055 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:23.113 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-18T14:38:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-18T14:38:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-18T14:38:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 8/26 2024-03-18T14:38:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 8/26 2024-03-18T14:38:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-18T14:38:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x 10/26 2024-03-18T14:38:23.794 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 11/26 2024-03-18T14:38:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 12/26 2024-03-18T14:38:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-18T14:38:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-18T14:38:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-18T14:38:24.349 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-18T14:38:24.389 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 22/26 2024-03-18T14:38:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 18/26 2024-03-18T14:38:24.478 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 22/26 2024-03-18T14:38:24.479 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T14:38:24.479 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T14:38:24.479 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:24.491 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 18/26 2024-03-18T14:38:24.515 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 19/26 2024-03-18T14:38:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86 20/26 2024-03-18T14:38:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:27.670 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 21/26 2024-03-18T14:38:27.711 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 22/26 2024-03-18T14:38:27.800 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 22/26 2024-03-18T14:38:27.801 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T14:38:27.801 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T14:38:27.801 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:27.846 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:33.947 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-03-18T14:38:33.948 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:35.232 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 24/26 2024-03-18T14:38:35.309 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 24/26 2024-03-18T14:38:35.309 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:38:35.309 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T14:38:35.310 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T14:38:35.310 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T14:38:35.310 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:35.311 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-18T14:38:35.405 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-18T14:38:35.405 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 1/26 2024-03-18T14:38:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:36.654 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 3/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 5/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 6/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 7/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x 8/26 2024-03-18T14:38:36.655 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86 9/26 2024-03-18T14:38:36.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 10/26 2024-03-18T14:38:36.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 11/26 2024-03-18T14:38:36.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 12/26 2024-03-18T14:38:36.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:36.656 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-18T14:38:36.657 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-18T14:38:36.658 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-18T14:38:36.658 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-18T14:38:36.659 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:36.659 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-18T14:38:36.659 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 25/26 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.546 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.547 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:37.548 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-03-18T14:38:37.737 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-03-18T14:38:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:37.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-test 2024-03-18T14:38:38.432 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:33 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:38:38.590 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:38:38.591 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:38:38.591 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:38.591 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 48 M 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 163 k 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout:Install 6 Packages 2024-03-18T14:38:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:38:38.593 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 49 M 2024-03-18T14:38:38.593 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 196 M 2024-03-18T14:38:38.593 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:38:39.074 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 24/26 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 24/26 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T14:38:39.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:39.137 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-18T14:38:39.210 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-18T14:38:39.210 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:39.283 INFO:teuthology.orchestra.run.smithi100.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 558 kB/s | 217 kB 00:00 2024-03-18T14:38:39.459 INFO:teuthology.orchestra.run.smithi100.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-03-18T14:38:39.550 INFO:teuthology.orchestra.run.smithi100.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 684 kB/s | 63 kB 00:00 2024-03-18T14:38:39.618 INFO:teuthology.orchestra.run.smithi100.stdout:(4/6): libcephsqlite-17.2.7-684.g6716c9f3.el9.x 225 kB/s | 163 kB 00:00 2024-03-18T14:38:39.988 INFO:teuthology.orchestra.run.smithi100.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 428 kB/s | 187 kB 00:00 2024-03-18T14:38:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 1/26 2024-03-18T14:38:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 3/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 4/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 5/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 6/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 7/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x 8/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86 9/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 10/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 11/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 12/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 13/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-18T14:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 23/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-18T14:38:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 25/26 2024-03-18T14:38:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-18T14:38:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:41.275 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-03-18T14:38:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T14:38:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:41.277 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:38:41.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-test 2024-03-18T14:38:42.182 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:33 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:38:42.336 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:38:42.336 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 48 M 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 163 k 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:38:42.337 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:38:42.338 INFO:teuthology.orchestra.run.smithi012.stdout:Install 6 Packages 2024-03-18T14:38:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:42.338 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 49 M 2024-03-18T14:38:42.338 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 196 M 2024-03-18T14:38:42.338 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:38:42.692 INFO:teuthology.orchestra.run.smithi012.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 701 kB/s | 217 kB 00:00 2024-03-18T14:38:42.792 INFO:teuthology.orchestra.run.smithi012.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.9 MB/s | 300 kB 00:00 2024-03-18T14:38:42.851 INFO:teuthology.orchestra.run.smithi012.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-18T14:38:42.918 INFO:teuthology.orchestra.run.smithi012.stdout:(4/6): libcephsqlite-17.2.7-684.g6716c9f3.el9.x 304 kB/s | 163 kB 00:00 2024-03-18T14:38:43.268 INFO:teuthology.orchestra.run.smithi012.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 449 kB/s | 187 kB 00:00 2024-03-18T14:38:45.410 INFO:teuthology.orchestra.run.smithi012.stdout:(6/6): ceph-test-17.2.7-684.g6716c9f3.el9.x86_6 16 MB/s | 48 MB 00:03 2024-03-18T14:38:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:38:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:Total 16 MB/s | 49 MB 00:03 2024-03-18T14:38:45.428 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:38:45.450 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:38:45.451 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:38:45.851 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:38:45.851 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:38:46.486 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:38:46.570 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-18T14:38:46.617 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-18T14:38:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-18T14:38:46.702 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-18T14:38:46.729 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 5/6 2024-03-18T14:38:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 5/6 2024-03-18T14:38:51.466 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 6/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 6/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 1/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 2/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-18T14:38:53.697 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-18T14:38:54.069 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-18T14:38:54.069 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: jq-1.6-16.el9.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:54.070 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:38:54.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph 2024-03-18T14:38:54.900 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:45 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:38:55.061 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================== 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================== 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 6.5 k 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.1 M 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 1.4 M 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 239 k 2024-03-18T14:38:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 3.7 M 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 15 M 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T14:38:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T14:38:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================== 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout:Install 33 Packages 2024-03-18T14:38:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:38:55.068 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 27 M 2024-03-18T14:38:55.068 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 100 M 2024-03-18T14:38:55.068 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:38:55.812 INFO:teuthology.orchestra.run.smithi012.stdout:(1/33): ceph-17.2.7-684.g6716c9f3.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2024-03-18T14:38:55.896 INFO:teuthology.orchestra.run.smithi012.stdout:(2/33): ceph-mgr-17.2.7-684.g6716c9f3.el9.x86_6 3.9 MB/s | 1.4 MB 00:00 2024-03-18T14:38:56.771 INFO:teuthology.orchestra.run.smithi012.stdout:(3/33): ceph-osd-17.2.7-684.g6716c9f3.el9.x86_6 18 MB/s | 15 MB 00:00 2024-03-18T14:38:56.846 INFO:teuthology.orchestra.run.smithi012.stdout:(4/33): ceph-mgr-modules-core-17.2.7-684.g6716c 3.1 MB/s | 239 kB 00:00 2024-03-18T14:38:57.047 INFO:teuthology.orchestra.run.smithi012.stdout:(5/33): ceph-mds-17.2.7-684.g6716c9f3.el9.x86_6 1.4 MB/s | 2.1 MB 00:01 2024-03-18T14:38:57.072 INFO:teuthology.orchestra.run.smithi012.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 641 kB/s | 144 kB 00:00 2024-03-18T14:38:57.097 INFO:teuthology.orchestra.run.smithi012.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 632 kB/s | 32 kB 00:00 2024-03-18T14:38:57.122 INFO:teuthology.orchestra.run.smithi012.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2024-03-18T14:38:57.164 INFO:teuthology.orchestra.run.smithi012.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 586 kB/s | 39 kB 00:00 2024-03-18T14:38:57.214 INFO:teuthology.orchestra.run.smithi012.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-03-18T14:38:57.257 INFO:teuthology.orchestra.run.smithi012.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-18T14:38:57.282 INFO:teuthology.orchestra.run.smithi012.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-18T14:38:57.307 INFO:teuthology.orchestra.run.smithi012.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-18T14:38:57.333 INFO:teuthology.orchestra.run.smithi012.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-18T14:38:57.524 INFO:teuthology.orchestra.run.smithi012.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 577 kB/s | 125 kB 00:00 2024-03-18T14:38:57.558 INFO:teuthology.orchestra.run.smithi012.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 193 kB/s | 43 kB 00:00 2024-03-18T14:38:57.792 INFO:teuthology.orchestra.run.smithi012.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-03-18T14:38:57.859 INFO:teuthology.orchestra.run.smithi012.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 514 kB/s | 172 kB 00:00 2024-03-18T14:38:57.875 INFO:teuthology.orchestra.run.smithi012.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-03-18T14:38:57.934 INFO:teuthology.orchestra.run.smithi012.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-03-18T14:38:57.976 INFO:teuthology.orchestra.run.smithi012.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-03-18T14:38:58.043 INFO:teuthology.orchestra.run.smithi012.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-03-18T14:38:58.060 INFO:teuthology.orchestra.run.smithi012.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-03-18T14:38:58.135 INFO:teuthology.orchestra.run.smithi012.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 504 kB/s | 46 kB 00:00 2024-03-18T14:38:58.160 INFO:teuthology.orchestra.run.smithi012.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 786 kB/s | 79 kB 00:00 2024-03-18T14:38:58.519 INFO:teuthology.orchestra.run.smithi012.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 46 kB/s | 16 kB 00:00 2024-03-18T14:38:58.561 INFO:teuthology.orchestra.run.smithi012.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 640 kB/s | 272 kB 00:00 2024-03-18T14:38:58.628 INFO:teuthology.orchestra.run.smithi012.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 827 kB/s | 90 kB 00:00 2024-03-18T14:38:58.670 INFO:teuthology.orchestra.run.smithi012.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 329 kB/s | 36 kB 00:00 2024-03-18T14:38:58.828 INFO:teuthology.orchestra.run.smithi012.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-18T14:38:58.928 INFO:teuthology.orchestra.run.smithi012.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.6 MB/s | 427 kB 00:00 2024-03-18T14:38:59.237 INFO:teuthology.orchestra.run.smithi012.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 49 kB/s | 20 kB 00:00 2024-03-18T14:39:04.078 INFO:teuthology.orchestra.run.smithi012.stdout:(33/33): ceph-mon-17.2.7-684.g6716c9f3.el9.x86_ 458 kB/s | 3.7 MB 00:08 2024-03-18T14:39:04.081 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:04.082 INFO:teuthology.orchestra.run.smithi012.stdout:Total 3.0 MB/s | 27 MB 00:09 2024-03-18T14:39:04.095 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:04.138 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:04.138 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:04.480 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:04.480 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:05.745 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:05.798 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-18T14:39:05.831 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-18T14:39:05.871 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-18T14:39:05.975 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-18T14:39:06.007 INFO:teuthology.orchestra.run.smithi100.stdout:(6/6): ceph-test-17.2.7-684.g6716c9f3.el9.x86_6 1.8 MB/s | 48 MB 00:27 2024-03-18T14:39:06.010 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:06.011 INFO:teuthology.orchestra.run.smithi100.stdout:Total 1.8 MB/s | 49 MB 00:27 2024-03-18T14:39:06.024 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:06.045 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:06.045 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:06.053 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-18T14:39:06.100 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-18T14:39:06.174 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-18T14:39:06.252 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-18T14:39:06.288 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-18T14:39:06.347 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-18T14:39:06.417 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-18T14:39:06.434 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:06.435 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-18T14:39:06.668 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-18T14:39:06.715 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-18T14:39:06.749 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-18T14:39:06.779 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-18T14:39:06.858 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-18T14:39:06.889 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-18T14:39:06.923 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-18T14:39:06.963 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-18T14:39:07.006 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-18T14:39:07.041 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-18T14:39:07.049 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:07.116 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-18T14:39:07.135 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-18T14:39:07.162 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-18T14:39:07.171 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-18T14:39:07.203 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-18T14:39:07.248 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-18T14:39:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-18T14:39:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 5/6 2024-03-18T14:39:07.388 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-18T14:39:07.512 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-18T14:39:07.668 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.n 28/33 2024-03-18T14:39:07.692 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 29/33 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 29/33 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T14:39:07.718 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 30/33 2024-03-18T14:39:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 30/33 2024-03-18T14:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T14:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T14:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T14:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:09.552 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 31/33 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 31/33 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T14:39:09.575 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:09.786 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 32/33 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 32/33 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T14:39:09.809 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:09.877 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 33/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 33/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 1/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 2/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 3/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 4/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 5/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.n 6/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-18T14:39:11.548 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-18T14:39:11.549 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-18T14:39:11.550 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-18T14:39:11.703 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 5/6 2024-03-18T14:39:11.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 6/6 2024-03-18T14:39:12.564 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-18T14:39:12.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:12.564 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:12.564 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:12.564 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T14:39:12.565 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T14:39:12.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T14:39:12.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T14:39:12.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T14:39:12.567 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:12.567 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:12.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-base 2024-03-18T14:39:13.449 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:04 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:13.549 INFO:teuthology.orchestra.run.smithi012.stdout:Package ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:13.604 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:13.605 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:13.605 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:13.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install cephadm 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 6/6 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 1/6 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 2/6 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-18T14:39:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-18T14:39:14.225 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:05 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:14.379 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 82 k 2024-03-18T14:39:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 82 k 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 372 k 2024-03-18T14:39:14.381 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-18T14:39:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:14.439 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:14.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph 2024-03-18T14:39:14.809 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm-17.2.7-684.g6716c9f3.el9.noarch.rpm 192 kB/s | 82 kB 00:00 2024-03-18T14:39:14.810 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:14.810 INFO:teuthology.orchestra.run.smithi012.stdout:Total 191 kB/s | 82 kB 00:00 2024-03-18T14:39:14.810 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:14.815 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:14.816 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:14.818 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:14.818 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:14.889 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:15.281 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:15.294 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:10 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:15.365 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:15.445 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:15.447 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-03-18T14:39:15.447 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:15.447 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-03-18T14:39:15.447 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:15.447 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 6.5 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.1 M 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 1.4 M 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 239 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 3.7 M 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 15 M 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T14:39:15.448 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T14:39:15.449 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout:Install 33 Packages 2024-03-18T14:39:15.450 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:15.451 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 27 M 2024-03-18T14:39:15.451 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 100 M 2024-03-18T14:39:15.451 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:15.746 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:15.875 INFO:teuthology.orchestra.run.smithi100.stdout:(1/33): ceph-17.2.7-684.g6716c9f3.el9.x86_64.rp 41 kB/s | 6.5 kB 00:00 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:16.057 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:16.267 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-immutable-object-cache 2024-03-18T14:39:16.342 INFO:teuthology.orchestra.run.smithi100.stdout:(2/33): ceph-mgr-17.2.7-684.g6716c9f3.el9.x86_6 2.3 MB/s | 1.4 MB 00:00 2024-03-18T14:39:16.467 INFO:teuthology.orchestra.run.smithi100.stdout:(3/33): ceph-mds-17.2.7-684.g6716c9f3.el9.x86_6 2.8 MB/s | 2.1 MB 00:00 2024-03-18T14:39:16.534 INFO:teuthology.orchestra.run.smithi100.stdout:(4/33): ceph-mgr-modules-core-17.2.7-684.g6716c 3.5 MB/s | 239 kB 00:00 2024-03-18T14:39:16.651 INFO:teuthology.orchestra.run.smithi100.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 144 kB 00:00 2024-03-18T14:39:16.684 INFO:teuthology.orchestra.run.smithi100.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 950 kB/s | 32 kB 00:00 2024-03-18T14:39:16.718 INFO:teuthology.orchestra.run.smithi100.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-18T14:39:16.751 INFO:teuthology.orchestra.run.smithi100.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-18T14:39:16.818 INFO:teuthology.orchestra.run.smithi100.stdout:(9/33): ceph-mon-17.2.7-684.g6716c9f3.el9.x86_6 3.9 MB/s | 3.7 MB 00:00 2024-03-18T14:39:16.835 INFO:teuthology.orchestra.run.smithi100.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-18T14:39:16.839 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:07 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:16.894 INFO:teuthology.orchestra.run.smithi100.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-18T14:39:16.911 INFO:teuthology.orchestra.run.smithi100.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-18T14:39:16.936 INFO:teuthology.orchestra.run.smithi100.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2024-03-18T14:39:16.962 INFO:teuthology.orchestra.run.smithi100.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-18T14:39:16.993 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repo Size 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 137 k 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:16.994 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 137 k 2024-03-18T14:39:16.995 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 379 k 2024-03-18T14:39:16.995 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:17.095 INFO:teuthology.orchestra.run.smithi100.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 789 kB/s | 125 kB 00:00 2024-03-18T14:39:17.223 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-immutable-object-cache-17.2.7-684.g6716c9f 602 kB/s | 137 kB 00:00 2024-03-18T14:39:17.224 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:17.224 INFO:teuthology.orchestra.run.smithi012.stdout:Total 599 kB/s | 137 kB 00:00 2024-03-18T14:39:17.224 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:17.231 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:17.232 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:17.270 INFO:teuthology.orchestra.run.smithi100.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 141 kB/s | 43 kB 00:00 2024-03-18T14:39:17.286 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:17.286 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:17.424 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:17.474 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:17.498 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:17.498 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:17.498 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T14:39:17.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:17.629 INFO:teuthology.orchestra.run.smithi100.stdout:(17/33): ceph-osd-17.2.7-684.g6716c9f3.el9.x86_ 12 MB/s | 15 MB 00:01 2024-03-18T14:39:17.688 INFO:teuthology.orchestra.run.smithi100.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 290 kB/s | 172 kB 00:00 2024-03-18T14:39:17.746 INFO:teuthology.orchestra.run.smithi100.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 754 kB/s | 358 kB 00:00 2024-03-18T14:39:17.780 INFO:teuthology.orchestra.run.smithi100.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 261 kB/s | 18 kB 00:00 2024-03-18T14:39:17.814 INFO:teuthology.orchestra.run.smithi100.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 345 kB/s | 23 kB 00:00 2024-03-18T14:39:17.830 INFO:teuthology.orchestra.run.smithi100.stdout:(22/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 53 kB/s | 11 kB 00:00 2024-03-18T14:39:17.847 INFO:teuthology.orchestra.run.smithi100.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 289 kB/s | 19 kB 00:00 2024-03-18T14:39:17.873 INFO:teuthology.orchestra.run.smithi100.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-03-18T14:39:17.931 INFO:teuthology.orchestra.run.smithi100.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 943 kB/s | 79 kB 00:00 2024-03-18T14:39:17.965 INFO:teuthology.orchestra.run.smithi100.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 345 kB/s | 46 kB 00:00 2024-03-18T14:39:17.990 INFO:teuthology.orchestra.run.smithi100.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-18T14:39:18.015 INFO:teuthology.orchestra.run.smithi100.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 196 kB/s | 16 kB 00:00 2024-03-18T14:39:18.074 INFO:teuthology.orchestra.run.smithi100.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 428 kB/s | 36 kB 00:00 2024-03-18T14:39:18.110 INFO:teuthology.orchestra.run.smithi100.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 621 kB/s | 90 kB 00:00 2024-03-18T14:39:18.151 INFO:teuthology.orchestra.run.smithi100.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-18T14:39:18.185 INFO:teuthology.orchestra.run.smithi100.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-03-18T14:39:18.210 INFO:teuthology.orchestra.run.smithi100.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-03-18T14:39:18.216 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:18.216 INFO:teuthology.orchestra.run.smithi100.stdout:Total 9.7 MB/s | 27 MB 00:02 2024-03-18T14:39:18.229 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:18.266 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:18.266 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:18.514 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:18.515 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:18.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:18.515 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:18.602 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:18.602 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:18.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-mgr 2024-03-18T14:39:19.332 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:10 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:19.432 INFO:teuthology.orchestra.run.smithi012.stdout:Package ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:19.487 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:19.488 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:19.488 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:19.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-mgr-dashboard 2024-03-18T14:39:19.865 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:19.938 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-18T14:39:19.970 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-18T14:39:20.012 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-18T14:39:20.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-18T14:39:20.112 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:11 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:20.190 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-18T14:39:20.231 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-18T14:39:20.268 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:20.268 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:20.268 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 1.7 M 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 23 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 15 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-18T14:39:20.269 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:Install 7 Packages 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 2.0 M 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 8.1 M 2024-03-18T14:39:20.270 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:20.311 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-18T14:39:20.365 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-18T14:39:20.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-18T14:39:20.461 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-18T14:39:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-18T14:39:20.610 INFO:teuthology.orchestra.run.smithi012.stdout:(1/7): ceph-grafana-dashboards-17.2.7-684.g6716 123 kB/s | 23 kB 00:00 2024-03-18T14:39:20.668 INFO:teuthology.orchestra.run.smithi012.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 155 kB/s | 9.0 kB 00:00 2024-03-18T14:39:20.709 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-18T14:39:20.727 INFO:teuthology.orchestra.run.smithi012.stdout:(3/7): python3-jwt-2.4.0-1.el9.noarch.rpm 700 kB/s | 41 kB 00:00 2024-03-18T14:39:20.758 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-18T14:39:20.761 INFO:teuthology.orchestra.run.smithi012.stdout:(4/7): ceph-prometheus-alerts-17.2.7-684.g6716c 43 kB/s | 15 kB 00:00 2024-03-18T14:39:20.794 INFO:teuthology.orchestra.run.smithi012.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-03-18T14:39:20.797 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-18T14:39:20.831 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-18T14:39:20.861 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-18T14:39:20.886 INFO:teuthology.orchestra.run.smithi012.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-03-18T14:39:20.928 INFO:teuthology.orchestra.run.smithi012.stdout:(7/7): ceph-mgr-dashboard-17.2.7-684.g6716c9f3. 3.3 MB/s | 1.7 MB 00:00 2024-03-18T14:39:20.929 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:20.930 INFO:teuthology.orchestra.run.smithi012.stdout:Total 3.0 MB/s | 2.0 MB 00:00 2024-03-18T14:39:20.930 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:20.937 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-18T14:39:20.938 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:20.938 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:20.971 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-18T14:39:21.006 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:21.006 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:21.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-18T14:39:21.053 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-18T14:39:21.088 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-18T14:39:21.123 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-18T14:39:21.211 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-18T14:39:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-18T14:39:21.334 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-18T14:39:21.431 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:21.471 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-18T14:39:21.504 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-18T14:39:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-18T14:39:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-18T14:39:21.586 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-18T14:39:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-18T14:39:21.646 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.no 5/7 2024-03-18T14:39:21.744 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.n 28/33 2024-03-18T14:39:21.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 29/33 2024-03-18T14:39:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 29/33 2024-03-18T14:39:21.800 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:21.800 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T14:39:21.800 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T14:39:21.800 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T14:39:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:21.884 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.n 6/7 2024-03-18T14:39:21.923 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 7/7 2024-03-18T14:39:22.544 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 7/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.n 1/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 2/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.no 3/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-18T14:39:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:22.874 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:23.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-18T14:39:23.135 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 30/33 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 30/33 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T14:39:23.162 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:23.562 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 31/33 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 31/33 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T14:39:23.585 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:23.658 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:14 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:23.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 32/33 2024-03-18T14:39:23.809 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 32/33 2024-03-18T14:39:23.809 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:23.809 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T14:39:23.810 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T14:39:23.810 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T14:39:23.810 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:23.815 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================== 2024-03-18T14:39:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================== 2024-03-18T14:39:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 7.4 M 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:23.817 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:=========================================================================================== 2024-03-18T14:39:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:Install 12 Packages 2024-03-18T14:39:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 37 M 2024-03-18T14:39:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 209 M 2024-03-18T14:39:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:23.875 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 33/33 2024-03-18T14:39:24.376 INFO:teuthology.orchestra.run.smithi012.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 192 kB/s | 32 kB 00:00 2024-03-18T14:39:24.410 INFO:teuthology.orchestra.run.smithi012.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 536 kB/s | 17 kB 00:00 2024-03-18T14:39:24.451 INFO:teuthology.orchestra.run.smithi012.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 835 kB/s | 34 kB 00:00 2024-03-18T14:39:24.502 INFO:teuthology.orchestra.run.smithi012.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-03-18T14:39:24.694 INFO:teuthology.orchestra.run.smithi012.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-03-18T14:39:24.761 INFO:teuthology.orchestra.run.smithi012.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-18T14:39:24.803 INFO:teuthology.orchestra.run.smithi012.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-03-18T14:39:25.119 INFO:teuthology.orchestra.run.smithi012.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-18T14:39:25.237 INFO:teuthology.orchestra.run.smithi012.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-03-18T14:39:25.345 INFO:teuthology.orchestra.run.smithi012.stdout:(10/12): ceph-mgr-diskprediction-local-17.2.7-6 6.5 MB/s | 7.4 MB 00:01 2024-03-18T14:39:25.370 INFO:teuthology.orchestra.run.smithi012.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 781 kB/s | 195 kB 00:00 2024-03-18T14:39:25.529 INFO:teuthology.orchestra.run.smithi012.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 841 kB/s | 245 kB 00:00 2024-03-18T14:39:25.530 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:25.531 INFO:teuthology.orchestra.run.smithi012.stdout:Total 22 MB/s | 37 MB 00:01 2024-03-18T14:39:25.571 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:25.595 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:25.595 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 33/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 1/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 2/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 3/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 4/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 5/33 2024-03-18T14:39:25.627 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.n 6/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-18T14:39:25.628 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-18T14:39:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-18T14:39:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-18T14:39:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-18T14:39:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-18T14:39:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-18T14:39:25.630 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-18T14:39:26.046 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:26.046 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:26.637 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T14:39:26.638 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T14:39:26.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T14:39:26.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T14:39:26.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T14:39:26.640 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:26.640 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:26.972 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-18T14:39:26.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-base 2024-03-18T14:39:26.999 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-18T14:39:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-18T14:39:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-18T14:39:27.334 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-18T14:39:27.369 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-18T14:39:27.447 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-18T14:39:27.539 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:22 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:27.636 INFO:teuthology.orchestra.run.smithi100.stdout:Package ceph-base-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:27.688 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:27.689 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:39:27.689 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:27.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install cephadm 2024-03-18T14:39:28.140 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-18T14:39:28.260 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-18T14:39:28.305 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:23 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:28.453 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:28.454 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:28.454 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:28.454 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 82 k 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 82 k 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 372 k 2024-03-18T14:39:28.455 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:28.643 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm-17.2.7-684.g6716c9f3.el9.noarch.rpm 437 kB/s | 82 kB 00:00 2024-03-18T14:39:28.644 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:28.644 INFO:teuthology.orchestra.run.smithi100.stdout:Total 433 kB/s | 82 kB 00:00 2024-03-18T14:39:28.644 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:28.649 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:28.649 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:28.651 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:28.651 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:28.730 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:29.123 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:29.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:29.560 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-18T14:39:29.635 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:29.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/1 2024-03-18T14:39:29.925 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:29.925 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:29.926 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:29.926 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:29.926 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:30.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-immutable-object-cache 2024-03-18T14:39:30.626 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-18T14:39:30.659 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 12/12 2024-03-18T14:39:30.698 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:25 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:30.848 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 137 k 2024-03-18T14:39:30.849 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 137 k 2024-03-18T14:39:30.850 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 379 k 2024-03-18T14:39:30.851 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:31.085 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-immutable-object-cache-17.2.7-684.g6716c9f 585 kB/s | 137 kB 00:00 2024-03-18T14:39:31.085 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:31.085 INFO:teuthology.orchestra.run.smithi100.stdout:Total 582 kB/s | 137 kB 00:00 2024-03-18T14:39:31.085 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:31.094 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:31.094 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:31.144 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:31.144 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:31.279 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:31.336 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:31.360 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:31.360 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:39:31.361 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T14:39:31.361 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.e 1/1 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:32.350 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:32.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr 2024-03-18T14:39:33.188 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:28 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:33.287 INFO:teuthology.orchestra.run.smithi100.stdout:Package ceph-mgr-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 12/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 1/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-18T14:39:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-18T14:39:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-18T14:39:33.340 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:33.341 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:39:33.341 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:33.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-dashboard 2024-03-18T14:39:33.798 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T14:39:33.799 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T14:39:33.800 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T14:39:33.800 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:33.800 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:33.965 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:28 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:34.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-mgr-rook 2024-03-18T14:39:34.114 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 1.7 M 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:39:34.115 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 23 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 15 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout:Install 7 Packages 2024-03-18T14:39:34.116 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:34.117 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 2.0 M 2024-03-18T14:39:34.117 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 8.1 M 2024-03-18T14:39:34.117 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:34.497 INFO:teuthology.orchestra.run.smithi100.stdout:(1/7): ceph-grafana-dashboards-17.2.7-684.g6716 131 kB/s | 23 kB 00:00 2024-03-18T14:39:34.523 INFO:teuthology.orchestra.run.smithi100.stdout:(2/7): ceph-prometheus-alerts-17.2.7-684.g6716c 72 kB/s | 15 kB 00:00 2024-03-18T14:39:34.614 INFO:teuthology.orchestra.run.smithi100.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-03-18T14:39:34.657 INFO:teuthology.orchestra.run.smithi100.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 306 kB/s | 41 kB 00:00 2024-03-18T14:39:34.678 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:25 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:34.682 INFO:teuthology.orchestra.run.smithi100.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 458 kB/s | 31 kB 00:00 2024-03-18T14:39:34.724 INFO:teuthology.orchestra.run.smithi100.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-03-18T14:39:34.834 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 47 k 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T14:39:34.835 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout:Install 13 Packages 2024-03-18T14:39:34.836 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:34.837 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 2.0 M 2024-03-18T14:39:34.837 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 26 M 2024-03-18T14:39:34.837 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:35.018 INFO:teuthology.orchestra.run.smithi012.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2024-03-18T14:39:35.043 INFO:teuthology.orchestra.run.smithi012.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 116 kB/s | 18 kB 00:00 2024-03-18T14:39:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-18T14:39:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T14:39:35.132 INFO:teuthology.orchestra.run.smithi100.stdout:(7/7): ceph-mgr-dashboard-17.2.7-684.g6716c9f3. 2.1 MB/s | 1.7 MB 00:00 2024-03-18T14:39:35.134 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:35.134 INFO:teuthology.orchestra.run.smithi100.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-03-18T14:39:35.135 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:35.143 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:35.143 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:35.144 INFO:teuthology.orchestra.run.smithi012.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-18T14:39:35.169 INFO:teuthology.orchestra.run.smithi012.stdout:(6/13): ceph-mgr-rook-17.2.7-684.g6716c9f3.el9. 168 kB/s | 47 kB 00:00 2024-03-18T14:39:35.194 INFO:teuthology.orchestra.run.smithi012.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-03-18T14:39:35.212 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:35.212 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:35.253 INFO:teuthology.orchestra.run.smithi012.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 257 kB/s | 15 kB 00:00 2024-03-18T14:39:35.278 INFO:teuthology.orchestra.run.smithi012.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-03-18T14:39:35.345 INFO:teuthology.orchestra.run.smithi012.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-03-18T14:39:35.370 INFO:teuthology.orchestra.run.smithi012.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 1.1 MB/s | 203 kB 00:00 2024-03-18T14:39:35.396 INFO:teuthology.orchestra.run.smithi012.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-03-18T14:39:35.463 INFO:teuthology.orchestra.run.smithi012.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.9 MB/s | 1.0 MB 00:00 2024-03-18T14:39:35.465 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:35.465 INFO:teuthology.orchestra.run.smithi012.stdout:Total 3.3 MB/s | 2.0 MB 00:00 2024-03-18T14:39:35.465 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:35.479 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:35.479 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:35.583 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:35.584 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-18T14:39:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-18T14:39:35.672 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-18T14:39:35.704 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-18T14:39:35.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.no 5/7 2024-03-18T14:39:35.963 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.n 6/7 2024-03-18T14:39:36.003 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 7/7 2024-03-18T14:39:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-18T14:39:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-18T14:39:36.380 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-18T14:39:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-18T14:39:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-18T14:39:36.554 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 7/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.n 1/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 2/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.no 3/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-18T14:39:36.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-18T14:39:36.696 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-18T14:39:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-18T14:39:36.901 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-18T14:39:36.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:36.901 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:36.901 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:36.902 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:37.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-18T14:39:37.517 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-18T14:39:37.563 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-18T14:39:37.605 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-18T14:39:37.651 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-18T14:39:37.675 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 13/13 2024-03-18T14:39:37.707 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:32 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:37.859 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout:=========================================================================================== 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout:=========================================================================================== 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 7.4 M 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T14:39:37.861 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout:=========================================================================================== 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout:Install 12 Packages 2024-03-18T14:39:37.862 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:37.863 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 37 M 2024-03-18T14:39:37.863 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 209 M 2024-03-18T14:39:37.863 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 13/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 1/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-18T14:39:38.476 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-18T14:39:38.656 INFO:teuthology.orchestra.run.smithi100.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 198 kB/s | 32 kB 00:00 2024-03-18T14:39:38.689 INFO:teuthology.orchestra.run.smithi100.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 519 kB/s | 17 kB 00:00 2024-03-18T14:39:38.765 INFO:teuthology.orchestra.run.smithi100.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-18T14:39:38.790 INFO:teuthology.orchestra.run.smithi100.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 346 kB/s | 34 kB 00:00 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T14:39:38.852 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:38.853 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:38.999 INFO:teuthology.orchestra.run.smithi100.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-03-18T14:39:39.065 INFO:teuthology.orchestra.run.smithi100.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-18T14:39:39.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-mgr-cephadm 2024-03-18T14:39:39.091 INFO:teuthology.orchestra.run.smithi100.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-18T14:39:39.566 INFO:teuthology.orchestra.run.smithi100.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-18T14:39:39.638 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:30 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:39.699 INFO:teuthology.orchestra.run.smithi100.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-03-18T14:39:39.794 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 123 k 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T14:39:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:Install 6 Packages 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 6.8 M 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 33 M 2024-03-18T14:39:39.796 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:39.825 INFO:teuthology.orchestra.run.smithi100.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-03-18T14:39:39.867 INFO:teuthology.orchestra.run.smithi100.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-18T14:39:40.533 INFO:teuthology.orchestra.run.smithi100.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-6 3.6 MB/s | 7.4 MB 00:02 2024-03-18T14:39:40.535 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:40.536 INFO:teuthology.orchestra.run.smithi100.stdout:Total 14 MB/s | 37 MB 00:02 2024-03-18T14:39:40.555 INFO:teuthology.orchestra.run.smithi012.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-18T14:39:40.575 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:40.598 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:40.598 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:40.797 INFO:teuthology.orchestra.run.smithi012.stdout:(2/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 2.5 MB/s | 548 kB 00:00 2024-03-18T14:39:40.855 INFO:teuthology.orchestra.run.smithi012.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 991 kB/s | 58 kB 00:00 2024-03-18T14:39:40.914 INFO:teuthology.orchestra.run.smithi012.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 874 kB/s | 51 kB 00:00 2024-03-18T14:39:41.044 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:41.044 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:41.289 INFO:teuthology.orchestra.run.smithi012.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 235 kB/s | 248 kB 00:01 2024-03-18T14:39:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-18T14:39:41.879 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-18T14:39:41.973 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-18T14:39:42.016 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-18T14:39:42.202 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-18T14:39:42.232 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-18T14:39:42.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-18T14:39:42.953 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-18T14:39:43.052 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-18T14:39:43.881 INFO:teuthology.orchestra.run.smithi012.stdout:(6/6): ceph-mgr-cephadm-17.2.7-684.g6716c9f3.el 34 kB/s | 123 kB 00:03 2024-03-18T14:39:43.883 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:43.883 INFO:teuthology.orchestra.run.smithi012.stdout:Total 1.7 MB/s | 6.8 MB 00:04 2024-03-18T14:39:43.898 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:43.910 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:43.910 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-18T14:39:44.327 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:44.498 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-18T14:39:44.550 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-18T14:39:45.111 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-18T14:39:45.196 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-18T14:39:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-18T14:39:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 6/6 2024-03-18T14:39:45.302 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-18T14:39:45.323 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 12/12 2024-03-18T14:39:46.177 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 6/6 2024-03-18T14:39:46.177 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/6 2024-03-18T14:39:46.177 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-18T14:39:46.177 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-18T14:39:46.177 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-18T14:39:46.178 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:46.483 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:46.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-fuse 2024-03-18T14:39:47.314 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:38 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:47.470 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 813 k 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:47.471 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout:Install 2 Packages 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 892 k 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 2.6 M 2024-03-18T14:39:47.472 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:47.673 INFO:teuthology.orchestra.run.smithi012.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 392 kB/s | 79 kB 00:00 2024-03-18T14:39:47.815 INFO:teuthology.orchestra.run.smithi012.stdout:(2/2): ceph-fuse-17.2.7-684.g6716c9f3.el9.x86_6 2.3 MB/s | 813 kB 00:00 2024-03-18T14:39:47.816 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:Total 2.5 MB/s | 892 kB 00:00 2024-03-18T14:39:47.817 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:47.838 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:47.896 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 12/12 2024-03-18T14:39:47.896 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9 1/12 2024-03-18T14:39:47.896 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-18T14:39:47.897 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-18T14:39:47.915 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:47.915 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:48.103 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:48.248 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-18T14:39:48.317 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2/2 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T14:39:48.336 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:48.337 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:48.689 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-rook 2024-03-18T14:39:48.996 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2/2 2024-03-18T14:39:48.996 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/2 2024-03-18T14:39:49.263 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:44 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:49.332 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-18T14:39:49.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:49.333 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:49.333 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-18T14:39:49.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:49.333 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:49.414 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:49.415 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:49.415 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:49.415 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 47 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T14:39:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout:Install 13 Packages 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 2.0 M 2024-03-18T14:39:49.417 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 26 M 2024-03-18T14:39:49.418 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:49.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install librados-devel 2024-03-18T14:39:49.699 INFO:teuthology.orchestra.run.smithi100.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 201 kB/s | 24 kB 00:00 2024-03-18T14:39:49.725 INFO:teuthology.orchestra.run.smithi100.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 123 kB/s | 18 kB 00:00 2024-03-18T14:39:49.775 INFO:teuthology.orchestra.run.smithi100.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-18T14:39:49.800 INFO:teuthology.orchestra.run.smithi100.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T14:39:49.825 INFO:teuthology.orchestra.run.smithi100.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-18T14:39:49.851 INFO:teuthology.orchestra.run.smithi100.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 853 kB/s | 43 kB 00:00 2024-03-18T14:39:49.876 INFO:teuthology.orchestra.run.smithi100.stdout:(7/13): ceph-mgr-rook-17.2.7-684.g6716c9f3.el9. 159 kB/s | 47 kB 00:00 2024-03-18T14:39:49.934 INFO:teuthology.orchestra.run.smithi100.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 180 kB/s | 15 kB 00:00 2024-03-18T14:39:49.960 INFO:teuthology.orchestra.run.smithi100.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-03-18T14:39:50.052 INFO:teuthology.orchestra.run.smithi100.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 1.1 MB/s | 203 kB 00:00 2024-03-18T14:39:50.077 INFO:teuthology.orchestra.run.smithi100.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-03-18T14:39:50.136 INFO:teuthology.orchestra.run.smithi100.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-03-18T14:39:50.222 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:41 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:50.361 INFO:teuthology.orchestra.run.smithi100.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.4 MB/s | 1.0 MB 00:00 2024-03-18T14:39:50.366 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:50.366 INFO:teuthology.orchestra.run.smithi100.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-03-18T14:39:50.366 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:50.376 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:50.376 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repo Size 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 126 k 2024-03-18T14:39:50.377 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 126 k 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 448 k 2024-03-18T14:39:50.378 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:50.475 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:50.475 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:50.828 INFO:teuthology.orchestra.run.smithi012.stdout:librados-devel-17.2.7-684.g6716c9f3.el9.x86_64. 281 kB/s | 126 kB 00:00 2024-03-18T14:39:50.828 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:50.829 INFO:teuthology.orchestra.run.smithi012.stdout:Total 280 kB/s | 126 kB 00:00 2024-03-18T14:39:50.829 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:50.834 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:50.834 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:50.893 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:50.894 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:50.991 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:51.041 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:51.156 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-18T14:39:51.162 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:51.212 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-18T14:39:51.267 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-18T14:39:51.305 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-18T14:39:51.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-18T14:39:51.439 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-18T14:39:51.556 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-18T14:39:51.601 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:51.605 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:51.901 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:52.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install libcephfs2 2024-03-18T14:39:52.324 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-18T14:39:52.370 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-18T14:39:52.403 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-18T14:39:52.449 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-18T14:39:52.473 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 13/13 2024-03-18T14:39:52.741 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:43 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:52.841 INFO:teuthology.orchestra.run.smithi012.stdout:Package libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:52.897 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:52.898 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:52.898 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:52.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install libcephfs-devel 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 13/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 1/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-18T14:39:53.319 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-18T14:39:53.320 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-18T14:39:53.524 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:44 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:53.678 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:53.679 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:53.679 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repo Size 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 25 k 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 25 k 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 110 k 2024-03-18T14:39:53.680 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:39:53.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T14:39:53.682 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T14:39:53.683 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T14:39:53.683 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T14:39:53.683 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T14:39:53.683 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:53.683 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:39:53.925 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-cephadm 2024-03-18T14:39:53.972 INFO:teuthology.orchestra.run.smithi012.stdout:libcephfs-devel-17.2.7-684.g6716c9f3.el9.x86_64 85 kB/s | 25 kB 00:00 2024-03-18T14:39:53.973 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:53.973 INFO:teuthology.orchestra.run.smithi012.stdout:Total 85 kB/s | 25 kB 00:00 2024-03-18T14:39:53.973 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:39:53.977 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:39:53.977 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:39:53.978 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:39:53.979 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:39:54.058 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:39:54.156 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:54.488 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:49 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:39:54.501 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:54.638 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:17.2.7-684.g6716c9f3.el9 ceph-noarch 123 k 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:39:54.639 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout:Install 6 Packages 2024-03-18T14:39:54.640 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:39:54.641 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 6.8 M 2024-03-18T14:39:54.641 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 33 M 2024-03-18T14:39:54.641 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:39:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:39:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:39:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:39:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:39:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:54.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install librados2 2024-03-18T14:39:55.171 INFO:teuthology.orchestra.run.smithi100.stdout:(1/6): ceph-mgr-cephadm-17.2.7-684.g6716c9f3.el 596 kB/s | 123 kB 00:00 2024-03-18T14:39:55.496 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:46 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:55.597 INFO:teuthology.orchestra.run.smithi100.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 392 kB/s | 248 kB 00:00 2024-03-18T14:39:55.597 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:55.653 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:55.654 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:55.654 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:55.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install librbd1 2024-03-18T14:39:55.997 INFO:teuthology.orchestra.run.smithi100.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 144 kB/s | 58 kB 00:00 2024-03-18T14:39:56.030 INFO:teuthology.orchestra.run.smithi100.stdout:(4/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 656 kB/s | 548 kB 00:00 2024-03-18T14:39:56.164 INFO:teuthology.orchestra.run.smithi100.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 306 kB/s | 51 kB 00:00 2024-03-18T14:39:56.279 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:47 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:56.379 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:56.435 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:56.436 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:56.436 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:56.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install python3-rados 2024-03-18T14:39:57.066 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:48 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:57.167 INFO:teuthology.orchestra.run.smithi012.stdout:Package python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:57.222 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:57.223 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:57.224 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:57.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install python3-rgw 2024-03-18T14:39:57.865 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:48 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:57.966 INFO:teuthology.orchestra.run.smithi012.stdout:Package python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:58.022 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:58.023 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:58.023 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:58.078 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install python3-cephfs 2024-03-18T14:39:58.653 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:49 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:58.753 INFO:teuthology.orchestra.run.smithi012.stdout:Package python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:58.758 INFO:teuthology.orchestra.run.smithi100.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 5.8 MB 00:03 2024-03-18T14:39:58.759 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:39:58.759 INFO:teuthology.orchestra.run.smithi100.stdout:Total 1.6 MB/s | 6.8 MB 00:04 2024-03-18T14:39:58.774 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:39:58.786 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:39:58.786 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:39:58.808 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:58.809 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:58.810 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:58.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install python3-rbd 2024-03-18T14:39:58.882 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:39:58.882 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:39:59.172 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:39:59.338 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-18T14:39:59.390 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-18T14:39:59.443 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:50 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:39:59.543 INFO:teuthology.orchestra.run.smithi012.stdout:Package python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:39:59.599 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:39:59.600 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-03-18T14:39:59.600 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:39:59.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install rbd-fuse 2024-03-18T14:39:59.923 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-18T14:40:00.010 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-18T14:40:00.074 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-18T14:40:00.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 6/6 2024-03-18T14:40:00.227 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:51 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:40:00.381 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:40:00.382 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-03-18T14:40:00.382 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:00.382 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:40:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 82 k 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 82 k 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 222 k 2024-03-18T14:40:00.383 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 6/6 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 1/6 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-18T14:40:00.888 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-18T14:40:01.200 INFO:teuthology.orchestra.run.smithi012.stdout:rbd-fuse-17.2.7-684.g6716c9f3.el9.x86_64.rpm 100 kB/s | 82 kB 00:00 2024-03-18T14:40:01.200 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:01.200 INFO:teuthology.orchestra.run.smithi012.stdout:Total 100 kB/s | 82 kB 00:00 2024-03-18T14:40:01.201 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:17.2.7-684.g6716c9f3.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:01.206 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:01.208 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:40:01.209 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:40:01.268 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:40:01.269 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:40:01.409 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:40:01.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-fuse 2024-03-18T14:40:01.512 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:01.894 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:02.036 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:57 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:02.185 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:02.185 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 813 k 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout:Install 2 Packages 2024-03-18T14:40:02.186 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:02.187 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 892 k 2024-03-18T14:40:02.187 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 2.6 M 2024-03-18T14:40:02.187 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:02.205 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:02.205 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:02.206 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:40:02.206 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:02.206 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:02.206 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:40:02.374 INFO:teuthology.orchestra.run.smithi100.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 420 kB/s | 79 kB 00:00 2024-03-18T14:40:02.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install rbd-mirror 2024-03-18T14:40:03.022 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:54 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:40:03.179 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repo Size 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.9 M 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 2.9 M 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 12 M 2024-03-18T14:40:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:40:03.966 INFO:teuthology.orchestra.run.smithi100.stdout:(2/2): ceph-fuse-17.2.7-684.g6716c9f3.el9.x86_6 457 kB/s | 813 kB 00:01 2024-03-18T14:40:03.967 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:03.968 INFO:teuthology.orchestra.run.smithi100.stdout:Total 501 kB/s | 892 kB 00:01 2024-03-18T14:40:03.968 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:03.992 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:03.992 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:04.070 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:04.071 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:04.386 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-18T14:40:04.449 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2/2 2024-03-18T14:40:05.157 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2/2 2024-03-18T14:40:05.158 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/2 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:05.476 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:05.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librados-devel 2024-03-18T14:40:06.246 INFO:teuthology.orchestra.run.smithi012.stdout:rbd-mirror-17.2.7-684.g6716c9f3.el9.x86_64.rpm 981 kB/s | 2.9 MB 00:03 2024-03-18T14:40:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:Total 980 kB/s | 2.9 MB 00:03 2024-03-18T14:40:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:40:06.259 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:40:06.259 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:40:06.338 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:40:06.338 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:40:06.345 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:01 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:06.495 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 126 k 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:40:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:06.497 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 126 k 2024-03-18T14:40:06.497 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 448 k 2024-03-18T14:40:06.497 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:06.767 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:40:06.838 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T14:40:06.863 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:06.896 INFO:teuthology.orchestra.run.smithi100.stdout:librados-devel-17.2.7-684.g6716c9f3.el9.x86_64. 316 kB/s | 126 kB 00:00 2024-03-18T14:40:06.896 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:06.896 INFO:teuthology.orchestra.run.smithi100.stdout:Total 315 kB/s | 126 kB 00:00 2024-03-18T14:40:06.896 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:06.902 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:06.902 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:06.956 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:06.957 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:07.105 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:07.202 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:07.659 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:08.018 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:40:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:08.019 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:40:08.038 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:08.038 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:08.038 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:08.038 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:08.039 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:08.039 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:08.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install rbd-nbd 2024-03-18T14:40:08.365 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install libcephfs2 2024-03-18T14:40:08.782 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:01:59 ago on Mon 18 Mar 2024 02:38:09 PM UTC. 2024-03-18T14:40:08.932 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:03 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 165 k 2024-03-18T14:40:08.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:08.942 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-03-18T14:40:08.942 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-03-18T14:40:08.942 INFO:teuthology.orchestra.run.smithi012.stdout:Install 1 Package 2024-03-18T14:40:08.942 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:08.943 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 165 k 2024-03-18T14:40:08.943 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 477 k 2024-03-18T14:40:08.943 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-03-18T14:40:09.030 INFO:teuthology.orchestra.run.smithi100.stdout:Package libcephfs2-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:09.084 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:09.085 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:09.085 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:09.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install libcephfs-devel 2024-03-18T14:40:09.166 INFO:teuthology.orchestra.run.smithi012.stdout:rbd-nbd-17.2.7-684.g6716c9f3.el9.x86_64.rpm 742 kB/s | 165 kB 00:00 2024-03-18T14:40:09.166 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:09.166 INFO:teuthology.orchestra.run.smithi012.stdout:Total 737 kB/s | 165 kB 00:00 2024-03-18T14:40:09.166 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-03-18T14:40:09.174 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-03-18T14:40:09.174 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-03-18T14:40:09.235 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-03-18T14:40:09.235 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-03-18T14:40:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-03-18T14:40:09.499 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:09.730 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:04 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:09.879 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 25 k 2024-03-18T14:40:09.880 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 25 k 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 110 k 2024-03-18T14:40:09.881 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:09.906 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:10.194 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:10.194 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:10.195 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-03-18T14:40:10.195 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:10.195 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:40:10.195 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-03-18T14:40:10.257 INFO:teuthology.orchestra.run.smithi100.stdout:libcephfs-devel-17.2.7-684.g6716c9f3.el9.x86_64 66 kB/s | 25 kB 00:00 2024-03-18T14:40:10.258 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:10.258 INFO:teuthology.orchestra.run.smithi100.stdout:Total 66 kB/s | 25 kB 00:00 2024-03-18T14:40:10.258 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:10.262 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:10.262 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:10.263 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:10.263 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:10.428 DEBUG:teuthology.parallel:result is None 2024-03-18T14:40:10.446 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:10.820 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:11.111 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:11.111 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:11.112 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:11.112 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:11.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librados2 2024-03-18T14:40:11.857 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:06 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:11.955 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:12.007 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:12.008 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:12.008 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:12.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librbd1 2024-03-18T14:40:12.654 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:07 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:12.751 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:12.803 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:12.805 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:12.805 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:12.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rados 2024-03-18T14:40:13.444 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:08 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:13.541 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rados-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:13.594 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:13.595 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:13.595 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:13.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rgw 2024-03-18T14:40:14.232 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:09 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:14.329 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rgw-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:14.381 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:14.382 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:14.382 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:14.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-cephfs 2024-03-18T14:40:15.016 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:10 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:15.111 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-cephfs-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:15.163 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:15.164 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:15.164 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:15.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rbd 2024-03-18T14:40:15.814 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:10 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:15.910 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rbd-2:17.2.7-684.g6716c9f3.el9.x86_64 is already installed. 2024-03-18T14:40:15.964 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:15.965 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-03-18T14:40:15.965 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:16.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-fuse 2024-03-18T14:40:16.598 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:11 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:16.746 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 82 k 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:40:16.747 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:16.748 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 82 k 2024-03-18T14:40:16.748 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 222 k 2024-03-18T14:40:16.748 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:17.052 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-fuse-17.2.7-684.g6716c9f3.el9.x86_64.rpm 269 kB/s | 82 kB 00:00 2024-03-18T14:40:17.053 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:17.053 INFO:teuthology.orchestra.run.smithi100.stdout:Total 267 kB/s | 82 kB 00:00 2024-03-18T14:40:17.053 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:17.060 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:17.060 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:17.115 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:17.116 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:17.248 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:17.349 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:17.747 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:18.044 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:18.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-mirror 2024-03-18T14:40:18.873 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:13 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:19.029 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-03-18T14:40:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 2.9 M 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 2.9 M 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 12 M 2024-03-18T14:40:19.030 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:19.710 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-mirror-17.2.7-684.g6716c9f3.el9.x86_64.rpm 4.3 MB/s | 2.9 MB 00:00 2024-03-18T14:40:19.710 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:19.711 INFO:teuthology.orchestra.run.smithi100.stdout:Total 4.3 MB/s | 2.9 MB 00:00 2024-03-18T14:40:19.711 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:19.720 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:19.720 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:19.791 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:19.791 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:20.196 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:20.256 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:20.280 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:20.281 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T14:40:20.281 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T14:40:20.281 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T14:40:20.281 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T14:40:20.281 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:21.361 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:21.361 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:21.362 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:21.362 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:21.362 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:21.362 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:21.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-nbd 2024-03-18T14:40:22.168 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:17 ago on Mon 18 Mar 2024 02:38:05 PM UTC. 2024-03-18T14:40:22.318 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:17.2.7-684.g6716c9f3.el9 ceph 165 k 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-03-18T14:40:22.319 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:22.320 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 165 k 2024-03-18T14:40:22.320 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 477 k 2024-03-18T14:40:22.320 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-18T14:40:22.736 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-nbd-17.2.7-684.g6716c9f3.el9.x86_64.rpm 397 kB/s | 165 kB 00:00 2024-03-18T14:40:22.736 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:40:22.736 INFO:teuthology.orchestra.run.smithi100.stdout:Total 395 kB/s | 165 kB 00:00 2024-03-18T14:40:22.737 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-18T14:40:22.744 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-18T14:40:22.744 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-18T14:40:22.800 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-18T14:40:22.800 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-18T14:40:22.949 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-18T14:40:23.067 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 1/1 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:17.2.7-684.g6716c9f3.el9.x86_64 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:40:23.811 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-18T14:40:24.063 DEBUG:teuthology.parallel:result is None 2024-03-18T14:40:24.064 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:40:24.064 INFO:teuthology.packaging:ref: None 2024-03-18T14:40:24.064 INFO:teuthology.packaging:tag: None 2024-03-18T14:40:24.064 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:40:24.064 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:40:24.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-18T14:40:24.275 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T14:40:24.299 INFO:teuthology.orchestra.run.smithi012.stdout:17.2.7-684.g6716c9f3.el9 2024-03-18T14:40:24.300 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-684.g6716c9f3.el9 2024-03-18T14:40:24.300 INFO:teuthology.task.install:The correct ceph version 17.2.7-684.g6716c9f3 is installed. 2024-03-18T14:40:24.301 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T14:40:24.302 INFO:teuthology.packaging:ref: None 2024-03-18T14:40:24.302 INFO:teuthology.packaging:tag: None 2024-03-18T14:40:24.302 INFO:teuthology.packaging:branch: quincy 2024-03-18T14:40:24.302 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:40:24.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-18T14:40:24.520 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T14:40:24.542 INFO:teuthology.orchestra.run.smithi100.stdout:17.2.7-684.g6716c9f3.el9 2024-03-18T14:40:24.542 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-684.g6716c9f3.el9 2024-03-18T14:40:24.542 INFO:teuthology.task.install:The correct ceph version 17.2.7-684.g6716c9f3 is installed. 2024-03-18T14:40:24.544 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T14:40:24.545 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:40:24.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T14:40:24.576 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:40:24.576 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T14:40:24.610 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T14:40:24.610 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:40:24.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T14:40:24.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T14:40:24.717 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:40:24.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T14:40:24.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T14:40:24.810 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T14:40:24.811 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:40:24.811 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T14:40:24.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T14:40:24.910 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:40:24.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T14:40:24.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T14:40:25.002 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T14:40:25.003 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:40:25.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T14:40:25.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T14:40:25.100 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:40:25.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T14:40:25.127 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T14:40:25.193 INFO:teuthology.run_tasks:Running task print... 2024-03-18T14:40:25.202 INFO:teuthology.task.print:**** done install task... 2024-03-18T14:40:25.202 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-18T14:40:25.301 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'mons down, quorum', 'out of quorum', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-18T14:40:25.301 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-18T14:40:25.301 INFO:tasks.cephadm:Cluster fsid is 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:40:25.302 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-18T14:40:25.302 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-18T14:40:25.302 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi012': '172.21.15.12', 'mon.smithi100': '172.21.15.100'} 2024-03-18T14:40:25.302 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-18T14:40:25.302 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-03-18T14:40:25.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-03-18T14:40:25.358 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-18T14:40:25.359 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:25.771 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 18 14:40 /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:25.772 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:26.131 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 18 14:40 /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:26.131 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:26.149 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T14:40:26.176 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-18T14:40:26.177 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-18T14:40:26.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-18T14:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-18T14:40:27.088 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-18T14:41:16.605 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)", 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "19857db3290c58169e44a8aec8ef9646981a77ffd5b146ecef1b6faf59d6c3ab", 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9" 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-03-18T14:41:16.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:41:16.733 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-18T14:41:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)", 2024-03-18T14:41:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "19857db3290c58169e44a8aec8ef9646981a77ffd5b146ecef1b6faf59d6c3ab", 2024-03-18T14:41:16.733 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-03-18T14:41:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9" 2024-03-18T14:41:16.734 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-18T14:41:16.734 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-18T14:41:16.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-03-18T14:41:16.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-03-18T14:41:16.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-03-18T14:41:16.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-03-18T14:41:16.882 INFO:tasks.cephadm:Writing seed config... 2024-03-18T14:41:16.883 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-18T14:41:16.883 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-18T14:41:16.884 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-18T14:41:16.885 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-18T14:41:16.886 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-18T14:41:16.887 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:41:16.887 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-18T14:41:16.911 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 74f0a062-e535-11ee-95c9-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-18T14:41:16.911 DEBUG:teuthology.orchestra.run.smithi012:mon.smithi012> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service 2024-03-18T14:41:16.953 INFO:tasks.cephadm:Bootstrapping... 2024-03-18T14:41:16.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 74f0a062-e535-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-18T14:41:17.164 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-03-18T14:41:17.165 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '74f0a062-e535-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-03-18T14:41:17.191 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T14:41:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-18T14:41:17.191 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-03-18T14:41:17.216 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T14:41:17.216 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-03-18T14:41:17.216 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-03-18T14:41:17.224 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T14:41:17.224 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T14:41:17.230 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T14:41:17.230 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:41:17.236 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-03-18T14:41:17.243 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-03-18T14:41:17.243 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-03-18T14:41:17.243 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-03-18T14:41:17.265 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T14:41:17.265 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-18T14:41:17.265 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-03-18T14:41:17.265 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-03-18T14:41:17.272 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T14:41:17.272 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T14:41:17.278 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T14:41:17.278 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:41:17.284 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139811284427872 on /run/cephadm/74f0a062-e535-11ee-95c9-87774f69a715.lock 2024-03-18T14:41:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139811284427872 acquired on /run/cephadm/74f0a062-e535-11ee-95c9-87774f69a715.lock 2024-03-18T14:41:17.291 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-03-18T14:41:17.291 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-03-18T14:41:17.291 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-03-18T14:41:17.295 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-03-18T14:41:17.295 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-03-18T14:41:17.297 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-18T14:41:17.297 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-18T14:41:17.298 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-03-18T14:41:17.300 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T14:41:17.301 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-03-18T14:41:17.301 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-03-18T14:41:17.301 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-18T14:41:17.301 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-18T14:41:17.302 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-18T14:41:17.781 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 19857db3290c58169e44a8aec8ef9646981a77ffd5b146ecef1b6faf59d6c3ab 2024-03-18T14:41:17.781 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-18T14:41:17.781 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-03-18T14:41:17.781 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-18T14:41:17.781 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:dde26ec1da815cb53894018b523d9a5c6c7454b327da4e857899d556db304927 2024-03-18T14:41:17.782 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:19857db3290c58169e44a8aec8ef9646981a77ffd5b146ecef1b6faf59d6c3ab 2024-03-18T14:41:17.782 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-18T14:41:18.693 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable) 2024-03-18T14:41:18.693 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable) 2024-03-18T14:41:18.693 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-03-18T14:41:19.574 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-03-18T14:41:19.575 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-03-18T14:41:20.446 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCPUvhlctzmNBAAKvGJAgeED8G0R4pkBXYrEA== 2024-03-18T14:41:21.327 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCQUvhlI7DPLhAARNpnYS9vVKP30eyig0MWCw== 2024-03-18T14:41:22.191 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCRUvhl2uOuJBAAjhH32+Ak80CqyXOH98qcEQ== 2024-03-18T14:41:22.192 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-03-18T14:41:23.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for smithi012 [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = octopus 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:41:23.056 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-03-18T14:41:24.020 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.smithi012 on 2024-03-18T14:41:24.212 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-18T14:41:24.350 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-18T14:41:24.499 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target. 2024-03-18T14:41:24.499 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target. 2024-03-18T14:41:24.667 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012 2024-03-18T14:41:24.667 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service: Unit ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service not loaded. 2024-03-18T14:41:24.831 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target.wants/ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service → /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715@.service. 2024-03-18T14:41:25.367 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T14:41:25.367 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-03-18T14:41:25.374 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T14:41:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:41:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-03-18T14:41:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T14:41:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-03-18T14:41:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-03-18T14:41:26.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi012 (age 0.574807s) 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-18T14:41:26.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.811+0000 7f554e29e700 1 Processor -- start 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.811+0000 7f554e29e700 1 -- start start 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.811+0000 7f554e29e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.811+0000 7f554e29e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548105820 con 0x7f5548071360 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5547fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5547fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38046/0 (socket says 172.21.15.12:38046) 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5547fff700 1 -- 172.21.15.12:0/435928887 learned_addr learned my addr 172.21.15.12:0/435928887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5547fff700 1 -- 172.21.15.12:0/435928887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5548105960 con 0x7f5548071360 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5547fff700 1 --2- 172.21.15.12:0/435928887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55380098e0 tx=0x7f5538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4872e12bbbfce4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.812+0000 7f5546ffd700 1 -- 172.21.15.12:0/435928887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553800b860 con 0x7f5548071360 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f5546ffd700 1 -- 172.21.15.12:0/435928887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800b9c0 con 0x7f5548071360 2024-03-18T14:41:26.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f5546ffd700 1 -- 172.21.15.12:0/435928887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553800bc90 con 0x7f5548071360 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 -- 172.21.15.12:0/435928887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 msgr2=0x7f55481052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 --2- 172.21.15.12:0/435928887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55380098e0 tx=0x7f5538009250 comp rx=0 tx=0).stop 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 -- 172.21.15.12:0/435928887 shutdown_connections 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 --2- 172.21.15.12:0/435928887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f55481052e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 -- 172.21.15.12:0/435928887 >> 172.21.15.12:0/435928887 conn(0x7f55480fcd50 msgr2=0x7f55480ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 -- 172.21.15.12:0/435928887 shutdown_connections 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.813+0000 7f554e29e700 1 -- 172.21.15.12:0/435928887 wait complete. 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f554e29e700 1 Processor -- start 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f554e29e700 1 -- start start 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f554e29e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f554e29e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548111390 con 0x7f5548071360 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f5547fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f5547fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38052/0 (socket says 172.21.15.12:38052) 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f5547fff700 1 -- 172.21.15.12:0/3970571134 learned_addr learned my addr 172.21.15.12:0/3970571134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.814+0000 7f5547fff700 1 -- 172.21.15.12:0/3970571134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380094f0 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5547fff700 1 --2- 172.21.15.12:0/3970571134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5538009290 tx=0x7f5538016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553801c5c0 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553801c740 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548111590 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f553801ca50 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5548111f70 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f553801a450 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.815+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5538025790 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.816+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554804bed0 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.819+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f553801a930 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.895+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5548112a50 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.896+0000 7f5544ff9700 1 -- 172.21.15.12:0/3970571134 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f553801aad0 con 0x7f5548071360 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 msgr2=0x7f5548110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 --2- 172.21.15.12:0/3970571134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5538009290 tx=0x7f5538016000 comp rx=0 tx=0).stop 2024-03-18T14:41:26.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 shutdown_connections 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 --2- 172.21.15.12:0/3970571134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5548071360 0x7f5548110e50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 >> 172.21.15.12:0/3970571134 conn(0x7f55480fcd50 msgr2=0x7f55480fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 shutdown_connections 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:25.897+0000 7f554e29e700 1 -- 172.21.15.12:0/3970571134 wait complete. 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-03-18T14:41:26.434 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-03-18T14:41:29.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:29.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T14:41:29.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:29.969 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T14:41:29.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.808+0000 7f74bbd10700 1 Processor -- start 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74bbd10700 1 -- start start 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74bbd10700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74bbd10700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4103070 con 0x7f74b4102720 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74b9aac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74b9aac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38058/0 (socket says 172.21.15.12:38058) 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.809+0000 7f74b9aac700 1 -- 172.21.15.12:0/2119665452 learned_addr learned my addr 172.21.15.12:0/2119665452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74b9aac700 1 -- 172.21.15.12:0/2119665452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b41031b0 con 0x7f74b4102720 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74b9aac700 1 --2- 172.21.15.12:0/2119665452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74b00098e0 tx=0x7f74b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeabbc50df1bf8d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74b8aaa700 1 -- 172.21.15.12:0/2119665452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74b000b860 con 0x7f74b4102720 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74b8aaa700 1 -- 172.21.15.12:0/2119665452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000b9c0 con 0x7f74b4102720 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74b8aaa700 1 -- 172.21.15.12:0/2119665452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74b000bc90 con 0x7f74b4102720 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74bbd10700 1 -- 172.21.15.12:0/2119665452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 msgr2=0x7f74b4102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:29.971 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74bbd10700 1 --2- 172.21.15.12:0/2119665452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74b00098e0 tx=0x7f74b0009250 comp rx=0 tx=0).stop 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74bbd10700 1 -- 172.21.15.12:0/2119665452 shutdown_connections 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74bbd10700 1 --2- 172.21.15.12:0/2119665452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4102b30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.810+0000 7f74bbd10700 1 -- 172.21.15.12:0/2119665452 >> 172.21.15.12:0/2119665452 conn(0x7f74b40fdee0 msgr2=0x7f74b41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 -- 172.21.15.12:0/2119665452 shutdown_connections 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 -- 172.21.15.12:0/2119665452 wait complete. 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 Processor -- start 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 -- start start 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74bbd10700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4111340 con 0x7f74b4102720 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74b9aac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74b9aac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38064/0 (socket says 172.21.15.12:38064) 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.811+0000 7f74b9aac700 1 -- 172.21.15.12:0/2632098186 learned_addr learned my addr 172.21.15.12:0/2632098186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74b9aac700 1 -- 172.21.15.12:0/2632098186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00094f0 con 0x7f74b4102720 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74b9aac700 1 --2- 172.21.15.12:0/2632098186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003080 tx=0x7f74b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:29.972 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74b001c5c0 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b001c720 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4111540 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74b001ca30 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b4111f70 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f74b001a450 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.812+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f74b002e790 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.813+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b405f5f0 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.815+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f74b001a890 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.884+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f74b410aab0 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.887+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f74b001aa30 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.887+0000 7f74aa7fc700 1 -- 172.21.15.12:0/2632098186 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f74b001bd60 con 0x7f74b4102720 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 msgr2=0x7f74b4110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 --2- 172.21.15.12:0/2632098186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003080 tx=0x7f74b0016000 comp rx=0 tx=0).stop 2024-03-18T14:41:29.973 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 shutdown_connections 2024-03-18T14:41:29.974 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 --2- 172.21.15.12:0/2632098186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74b4102720 0x7f74b4110e00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:29.974 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 >> 172.21.15.12:0/2632098186 conn(0x7f74b40fdee0 msgr2=0x7f74b4107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:29.974 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 shutdown_connections 2024-03-18T14:41:29.974 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:26.888+0000 7f74bbd10700 1 -- 172.21.15.12:0/2632098186 wait complete. 2024-03-18T14:41:29.974 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-03-18T14:41:33.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:33 smithi012 ceph-mon[28798]: from='client.? 172.21.15.12:0/3832188133' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cc596a700 1 Processor -- start 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cc596a700 1 -- start start 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cc596a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cc596a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0103080 con 0x7f1cc0102730 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47616/0 (socket says 172.21.15.12:47616) 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cbeffd700 1 -- 172.21.15.12:0/2283908508 learned_addr learned my addr 172.21.15.12:0/2283908508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.040+0000 7f1cbeffd700 1 -- 172.21.15.12:0/2283908508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc01031c0 con 0x7f1cc0102730 2024-03-18T14:41:36.165 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cbeffd700 1 --2- 172.21.15.12:0/2283908508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00098e0 tx=0x7f1cb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd727aaed81c095 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cbdffb700 1 -- 172.21.15.12:0/2283908508 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cb000b860 con 0x7f1cc0102730 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cbdffb700 1 -- 172.21.15.12:0/2283908508 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1cb000be50 con 0x7f1cc0102730 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cbdffb700 1 -- 172.21.15.12:0/2283908508 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cb0010380 con 0x7f1cc0102730 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cc596a700 1 -- 172.21.15.12:0/2283908508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 msgr2=0x7f1cc0102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cc596a700 1 --2- 172.21.15.12:0/2283908508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00098e0 tx=0x7f1cb0009250 comp rx=0 tx=0).stop 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cc596a700 1 -- 172.21.15.12:0/2283908508 shutdown_connections 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cc596a700 1 --2- 172.21.15.12:0/2283908508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc0102b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.041+0000 7f1cc596a700 1 -- 172.21.15.12:0/2283908508 >> 172.21.15.12:0/2283908508 conn(0x7f1cc00fdf10 msgr2=0x7f1cc0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 -- 172.21.15.12:0/2283908508 shutdown_connections 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 -- 172.21.15.12:0/2283908508 wait complete. 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 Processor -- start 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 -- start start 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:36.166 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cc596a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0111b30 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.042+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47618/0 (socket says 172.21.15.12:47618) 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cbeffd700 1 -- 172.21.15.12:0/3832188133 learned_addr learned my addr 172.21.15.12:0/3832188133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cbeffd700 1 -- 172.21.15.12:0/3832188133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00094f0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cbeffd700 1 --2- 172.21.15.12:0/3832188133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cb00107c0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc0111d30 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1cb0010db0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cb001ada0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc01128a0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1cb00196d0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.043+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1cb0024920 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.044+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc01121c0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.046+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1cb0029060 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.118+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1cc004bed0 con 0x7f1cc0102730 2024-03-18T14:41:36.167 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.118+0000 7f1ca77fe700 1 -- 172.21.15.12:0/3832188133 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1cb0020070 con 0x7f1cc0102730 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 msgr2=0x7f1cc01115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 --2- 172.21.15.12:0/3832188133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0009250 comp rx=0 tx=0).stop 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 shutdown_connections 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 --2- 172.21.15.12:0/3832188133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cc0102730 0x7f1cc01115f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 >> 172.21.15.12:0/3832188133 conn(0x7f1cc00fdf10 msgr2=0x7f1cc00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 shutdown_connections 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:33.119+0000 7f1cc596a700 1 -- 172.21.15.12:0/3832188133 wait complete. 2024-03-18T14:41:36.168 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-03-18T14:41:36.445 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:36 smithi012 systemd[1]: Stopping Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:41:36.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:36 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[28793]: 2024-03-18T14:41:36.442+0000 7f688962b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi012 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:41:36.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:36 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[28793]: 2024-03-18T14:41:36.442+0000 7f688962b700 -1 mon.smithi012@0(leader) e1 *** Got Signal Terminated *** 2024-03-18T14:41:36.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:36 smithi012 podman[29056]: 2024-03-18 14:41:36.657386322 +0000 UTC m=+0.326563945 container died e21057bba00f8e41916ba6e7a1e56fbbc9c649054e16c6b769249b2ae2b2093d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-6716c9f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-18T14:41:37.189 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29056]: 2024-03-18 14:41:37.00929027 +0000 UTC m=+0.678467907 container cleanup e21057bba00f8e41916ba6e7a1e56fbbc9c649054e16c6b769249b2ae2b2093d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=quincy-6716c9f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-18T14:41:37.189 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 bash[29056]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012 2024-03-18T14:41:37.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29070]: 2024-03-18 14:41:37.2853202 +0000 UTC m=+0.627027056 container remove e21057bba00f8e41916ba6e7a1e56fbbc9c649054e16c6b769249b2ae2b2093d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-6716c9f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-18T14:41:37.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service: Deactivated successfully. 2024-03-18T14:41:37.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 systemd[1]: Stopped Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:41:37.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 systemd[1]: Starting Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:41:37.827 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-18T14:41:37.851 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29163]: 2024-03-18 14:41:37.578025637 +0000 UTC m=+0.091362660 container create e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=quincy-6716c9f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T14:41:37.851 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29163]: 2024-03-18 14:41:37.504441495 +0000 UTC m=+0.017778491 image pull 19857db3290c58169e44a8aec8ef9646981a77ffd5b146ecef1b6faf59d6c3ab quay.ceph.io/ceph-ci/ceph:quincy 2024-03-18T14:41:37.851 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29163]: 2024-03-18 14:41:37.74683704 +0000 UTC m=+0.260174034 container init e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=quincy-6716c9f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True) 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 podman[29163]: 2024-03-18 14:41:37.749571716 +0000 UTC m=+0.262908700 container start e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-6716c9f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable), process ceph-mon, pid 2 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: pidfile_write: ignore empty --pid-file 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: load: jerasure load: lrc 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: RocksDB version: 6.15.5 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Compile date Mar 16 2024 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: DB SUMMARY 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: DB Session ID: X6ECT87PFDILF8O1I2YR 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: CURRENT file: CURRENT 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T14:41:37.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi012/store.db dir, Total Num: 1, files: 000008.sst 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi012/store.db: 000010.log size: 98284 ; 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.error_if_exists: 0 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.create_if_missing: 0 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.paranoid_checks: 1 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.env: 0x55fd0c395a80 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.fs: Posix File System 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.info_log: 0x55fd0cf39c00 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.statistics: (nil) 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.use_fsync: 0 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_log_file_size: 0 2024-03-18T14:41:37.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_fallocate: 1 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.use_direct_reads: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.db_log_dir: 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi012/store.db 2024-03-18T14:41:37.854 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.write_buffer_manager: 0x55fd0dcaeab0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T14:41:37.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T14:41:37.856 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.unordered_write: 0 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.row_cache: None 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.wal_filter: None 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T14:41:37.857 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.preserve_deletes: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.two_write_queues: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.atomic_flush: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.log_readahead_size: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T14:41:37.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_background_jobs: 2 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_background_compactions: -1 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_subcompactions: 1 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_open_files: -1 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T14:41:37.859 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_background_flushes: -1 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Compression algorithms supported: 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kZSTD supported: 0 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kXpressCompression supported: 0 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kZlibCompression supported: 1 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kSnappyCompression supported: 1 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kLZ4Compression supported: 1 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: kBZip2Compression supported: 0 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi012/store.db/MANIFEST-000009 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.merge_operator: 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_filter: None 2024-03-18T14:41:37.860 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd0cf8ef58) 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_type: 0 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: data_block_index_type: 0 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_shortening: 1 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: hash_index_allow_collision: 1 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: checksum: 1 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: no_block_cache: 0 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache: 0x55fd0cfc1090 2024-03-18T14:41:37.861 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_options: 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: capacity : 536870912 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: num_shard_bits : 4 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: strict_capacity_limit : 0 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_compressed: (nil) 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: persistent_cache: (nil) 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_size: 4096 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_size_deviation: 10 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_restart_interval: 16 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_block_restart_interval: 1 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: metadata_block_size: 4096 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: partition_filters: 0 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: use_delta_encoding: 1 2024-03-18T14:41:37.862 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: whole_key_filtering: 1 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: verify_compression: 0 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: format_version: 4 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: enable_index_compression: 1 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_align: 0 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression: NoCompression 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T14:41:37.863 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.num_levels: 7 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:41:37.864 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T14:41:37.865 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.arena_block_size: 4194304 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T14:41:37.866 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.table_properties_collectors: 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.inplace_update_support: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.bloom_locality: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.max_successive_merges: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.ttl: 2592000 2024-03-18T14:41:37.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.enable_blob_files: false 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.min_blob_size: 0 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi012/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710772897775356, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710772897776474, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93069, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91164, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13388, "raw_average_key_size": 49, "raw_value_size": 83852, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1710772897, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "2aa264f1-b5b4-4df3-8f2c-581da17794f1", "db_session_id": "X6ECT87PFDILF8O1I2YR"}} 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710772897776912, "job": 1, "event": "recovery_finished"} 2024-03-18T14:41:37.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi012/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55fd0cf55180 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: DB pointer 0x55fd0d02a000 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** DB Stats ** 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:41:37.869 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: L0 2/0 92.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Sum 2/0 92.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative compaction: 0.00 GB write, 21.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:41:37.870 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval compaction: 0.00 GB write, 21.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: L0 2/0 92.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Sum 2/0 92.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:41:37.871 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 119.0 0.00 0.00 1 0.001 0 0 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative compaction: 0.00 GB write, 21.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: starting mon.smithi012 rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi012 fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???) e1 preinit fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).mds e1 new map 2024-03-18T14:41:37.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).mds e1 print_map 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: e1 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: legacy client fscid: -1 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: No filesystems configured 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 bash[29163]: e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 systemd[1]: Started Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mon.smithi012 is new leader, mons smithi012 in quorum (ranks 0) 2024-03-18T14:41:37.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: monmap e1: 1 mons at {smithi012=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T14:41:37.874 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: fsmap 2024-03-18T14:41:37.874 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T14:41:37.874 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:37 smithi012 ceph-mon[29182]: mgrmap e1: no daemons active 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.266+0000 7f4dd7f44700 1 Processor -- start 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.266+0000 7f4dd7f44700 1 -- start start 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.266+0000 7f4dd7f44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.266+0000 7f4dd7f44700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0103060 con 0x7f4dd0102710 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd5ce0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd5ce0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47624/0 (socket says 172.21.15.12:47624) 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd5ce0700 1 -- 172.21.15.12:0/3427087384 learned_addr learned my addr 172.21.15.12:0/3427087384 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd5ce0700 1 -- 172.21.15.12:0/3427087384 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd01031a0 con 0x7f4dd0102710 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd5ce0700 1 --2- 172.21.15.12:0/3427087384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009e50 tx=0x7f4dc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ed914cb01ead4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd4cde700 1 -- 172.21.15.12:0/3427087384 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dc00092e0 con 0x7f4dd0102710 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd4cde700 1 -- 172.21.15.12:0/3427087384 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4dc0012430 con 0x7f4dd0102710 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.267+0000 7f4dd4cde700 1 -- 172.21.15.12:0/3427087384 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dc00075c0 con 0x7f4dd0102710 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 -- 172.21.15.12:0/3427087384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 msgr2=0x7f4dd0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 --2- 172.21.15.12:0/3427087384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009e50 tx=0x7f4dc0009840 comp rx=0 tx=0).stop 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 -- 172.21.15.12:0/3427087384 shutdown_connections 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 --2- 172.21.15.12:0/3427087384 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0102b20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 -- 172.21.15.12:0/3427087384 >> 172.21.15.12:0/3427087384 conn(0x7f4dd00fde50 msgr2=0x7f4dd01002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 -- 172.21.15.12:0/3427087384 shutdown_connections 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.268+0000 7f4dd7f44700 1 -- 172.21.15.12:0/3427087384 wait complete. 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd7f44700 1 Processor -- start 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd7f44700 1 -- start start 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd7f44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:38.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd7f44700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0111ad0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd5ce0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd5ce0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47634/0 (socket says 172.21.15.12:47634) 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd5ce0700 1 -- 172.21.15.12:0/2607919782 learned_addr learned my addr 172.21.15.12:0/2607919782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd5ce0700 1 -- 172.21.15.12:0/2607919782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0009ae0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.269+0000 7f4dd5ce0700 1 --2- 172.21.15.12:0/2607919782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009fd0 tx=0x7f4dc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dc00092e0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4dc0007cc0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd0111cd0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dc001c9a0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd0112840 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4dc0019070 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.270+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4dc0025b10 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.271+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd005f5f0 con 0x7f4dd0102710 2024-03-18T14:41:38.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.273+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4dc002a060 con 0x7f4dd0102710 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.342+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4dd0104530 con 0x7f4dd0102710 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dce7fc700 1 -- 172.21.15.12:0/2607919782 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4dc0020070 con 0x7f4dd0102710 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 msgr2=0x7f4dd0111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 --2- 172.21.15.12:0/2607919782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009fd0 tx=0x7f4dc0009840 comp rx=0 tx=0).stop 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 shutdown_connections 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 --2- 172.21.15.12:0/2607919782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd0102710 0x7f4dd0111590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 >> 172.21.15.12:0/2607919782 conn(0x7f4dd00fde50 msgr2=0x7f4dd00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 shutdown_connections 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:38.347+0000 7f4dd7f44700 1 -- 172.21.15.12:0/2607919782 wait complete. 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-03-18T14:41:38.815 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 9283 ... 2024-03-18T14:41:38.980 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-74f0a062-e535-11ee-95c9-87774f69a715@mgr.smithi012.tmscgt 2024-03-18T14:41:38.980 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-74f0a062-e535-11ee-95c9-87774f69a715@mgr.smithi012.tmscgt.service: Unit ceph-74f0a062-e535-11ee-95c9-87774f69a715@mgr.smithi012.tmscgt.service not loaded. 2024-03-18T14:41:39.125 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715.target.wants/ceph-74f0a062-e535-11ee-95c9-87774f69a715@mgr.smithi012.tmscgt.service → /etc/systemd/system/ceph-74f0a062-e535-11ee-95c9-87774f69a715@.service. 2024-03-18T14:41:39.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:39 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2607919782' entity='client.admin' 2024-03-18T14:41:39.681 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T14:41:39.681 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-03-18T14:41:39.688 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T14:41:39.688 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:41:39.688 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-03-18T14:41:39.688 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T14:41:39.696 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T14:41:39.696 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-03-18T14:41:39.704 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-03-18T14:41:40.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:40 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2322508993' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T14:41:40.835 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "74f0a062-e535-11ee-95c9-87774f69a715", 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T14:41:40.846 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "smithi012" 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-18T14:41:40.847 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T14:41:40.848 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T14:41:40.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T14:41:25.323607+0000", 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86ff856700 1 Processor -- start 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86ff856700 1 -- start start 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86ff856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86ff856700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8105810 con 0x7f86f8076c00 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86fd5f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86fd5f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43240/0 (socket says 172.21.15.12:43240) 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.134+0000 7f86fd5f2700 1 -- 172.21.15.12:0/3338907245 learned_addr learned my addr 172.21.15.12:0/3338907245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.135+0000 7f86fd5f2700 1 -- 172.21.15.12:0/3338907245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8105950 con 0x7f86f8076c00 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.135+0000 7f86fd5f2700 1 --2- 172.21.15.12:0/3338907245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0098e0 tx=0x7f86ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c537e6d69fd35b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.135+0000 7f86ebfff700 1 -- 172.21.15.12:0/3338907245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86ec00b860 con 0x7f86f8076c00 2024-03-18T14:41:40.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.135+0000 7f86ebfff700 1 -- 172.21.15.12:0/3338907245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86ec00be50 con 0x7f86f8076c00 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.135+0000 7f86ebfff700 1 -- 172.21.15.12:0/3338907245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86ec010380 con 0x7f86f8076c00 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 -- 172.21.15.12:0/3338907245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 msgr2=0x7f86f81052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 --2- 172.21.15.12:0/3338907245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0098e0 tx=0x7f86ec009250 comp rx=0 tx=0).stop 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 -- 172.21.15.12:0/3338907245 shutdown_connections 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 --2- 172.21.15.12:0/3338907245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f81052d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 -- 172.21.15.12:0/3338907245 >> 172.21.15.12:0/3338907245 conn(0x7f86f80fcd80 msgr2=0x7f86f80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 -- 172.21.15.12:0/3338907245 shutdown_connections 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.136+0000 7f86ff856700 1 -- 172.21.15.12:0/3338907245 wait complete. 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86ff856700 1 Processor -- start 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86ff856700 1 -- start start 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86ff856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86ff856700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8111ac0 con 0x7f86f8076c00 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86fd5f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86fd5f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43242/0 (socket says 172.21.15.12:43242) 2024-03-18T14:41:40.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86fd5f2700 1 -- 172.21.15.12:0/2322508993 learned_addr learned my addr 172.21.15.12:0/2322508993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86fd5f2700 1 -- 172.21.15.12:0/2322508993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0094f0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86fd5f2700 1 --2- 172.21.15.12:0/2322508993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006b00 tx=0x7f86ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86ec0107c0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.137+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86ec010db0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.138+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f8111cc0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.138+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86ec01acf0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.138+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f8112830 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.138+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f86ec024440 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.138+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f86ec019710 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.139+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f805f5f0 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.141+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f86ec01a530 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.214+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f86f8106730 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.214+0000 7f86e9ffb700 1 -- 172.21.15.12:0/2322508993 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f86ec010920 con 0x7f86f8076c00 2024-03-18T14:41:40.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 msgr2=0x7f86f8111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 --2- 172.21.15.12:0/2322508993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006b00 tx=0x7f86ec009250 comp rx=0 tx=0).stop 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 shutdown_connections 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 --2- 172.21.15.12:0/2322508993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f8076c00 0x7f86f8111580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 >> 172.21.15.12:0/2322508993 conn(0x7f86f80fcd80 msgr2=0x7f86f80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 shutdown_connections 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:40.216+0000 7f86ff856700 1 -- 172.21.15.12:0/2322508993 wait complete. 2024-03-18T14:41:40.854 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-03-18T14:41:43.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:43 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1452905733' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "74f0a062-e535-11ee-95c9-87774f69a715", 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-03-18T14:41:44.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "smithi012" 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T14:41:44.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T14:41:44.009 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T14:41:44.009 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T14:41:44.009 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T14:41:44.009 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:44.010 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:44.011 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T14:41:25.323607+0000", 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f779da93700 1 Processor -- start 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f779da93700 1 -- start start 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f779da93700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f779da93700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798101410 con 0x7f7798100ac0 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f77977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f77977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43256/0 (socket says 172.21.15.12:43256) 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f77977fe700 1 -- 172.21.15.12:0/885602828 learned_addr learned my addr 172.21.15.12:0/885602828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f77977fe700 1 -- 172.21.15.12:0/885602828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798101550 con 0x7f7798100ac0 2024-03-18T14:41:44.012 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.292+0000 7f77977fe700 1 --2- 172.21.15.12:0/885602828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77880098e0 tx=0x7f7788009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1440e60712298a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f77967fc700 1 -- 172.21.15.12:0/885602828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f778800b860 con 0x7f7798100ac0 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f77967fc700 1 -- 172.21.15.12:0/885602828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f778800be50 con 0x7f7798100ac0 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f77967fc700 1 -- 172.21.15.12:0/885602828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7788010380 con 0x7f7798100ac0 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 -- 172.21.15.12:0/885602828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 msgr2=0x7f7798100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 --2- 172.21.15.12:0/885602828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77880098e0 tx=0x7f7788009250 comp rx=0 tx=0).stop 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 -- 172.21.15.12:0/885602828 shutdown_connections 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 --2- 172.21.15.12:0/885602828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 -- 172.21.15.12:0/885602828 >> 172.21.15.12:0/885602828 conn(0x7f77980fc500 msgr2=0x7f77980fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 -- 172.21.15.12:0/885602828 shutdown_connections 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.293+0000 7f779da93700 1 -- 172.21.15.12:0/885602828 wait complete. 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f779da93700 1 Processor -- start 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f779da93700 1 -- start start 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f779da93700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:44.013 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f779da93700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798111c90 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f77977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f77977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43258/0 (socket says 172.21.15.12:43258) 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.294+0000 7f77977fe700 1 -- 172.21.15.12:0/1452905733 learned_addr learned my addr 172.21.15.12:0/1452905733 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f77977fe700 1 -- 172.21.15.12:0/1452905733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880094f0 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f77977fe700 1 --2- 172.21.15.12:0/1452905733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f7788009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77880107c0 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7798111e90 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7788010db0 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f778801ad00 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7798112a00 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7788024440 con 0x7f7798100ac0 2024-03-18T14:41:44.014 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.295+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7788019710 con 0x7f7798100ac0 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.296+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779805f5f0 con 0x7f7798100ac0 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.298+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f77880426d0 con 0x7f7798100ac0 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.370+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7798063c70 con 0x7f7798100ac0 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.371+0000 7f777ffff700 1 -- 172.21.15.12:0/1452905733 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f7788031020 con 0x7f7798100ac0 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 msgr2=0x7f7798111750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 --2- 172.21.15.12:0/1452905733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f7788009250 comp rx=0 tx=0).stop 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 shutdown_connections 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 --2- 172.21.15.12:0/1452905733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7798100ac0 0x7f7798111750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 >> 172.21.15.12:0/1452905733 conn(0x7f77980fc500 msgr2=0x7f77980fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 shutdown_connections 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:43.372+0000 7f779da93700 1 -- 172.21.15.12:0/1452905733 wait complete. 2024-03-18T14:41:44.015 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: mgrmap e2: smithi012.tmscgt(active, starting, since 0.0028974s) 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' 2024-03-18T14:41:46.567 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:46 smithi012 ceph-mon[29182]: from='mgr.14100 172.21.15.12:0/50947968' entity='mgr.smithi012.tmscgt' 2024-03-18T14:41:47.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "74f0a062-e535-11ee-95c9-87774f69a715", 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "smithi012" 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:47.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T14:41:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T14:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T14:41:47.051 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T14:41:25.323607+0000", 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.484+0000 7fd43d03a700 1 Processor -- start 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.484+0000 7fd43d03a700 1 -- start start 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.484+0000 7fd43d03a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.484+0000 7fd43d03a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380790d0 con 0x7fd438078cc0 2024-03-18T14:41:47.052 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.485+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.485+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43362/0 (socket says 172.21.15.12:43362) 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.485+0000 7fd436d9d700 1 -- 172.21.15.12:0/495463613 learned_addr learned my addr 172.21.15.12:0/495463613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.485+0000 7fd436d9d700 1 -- 172.21.15.12:0/495463613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd438077760 con 0x7fd438078cc0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.485+0000 7fd436d9d700 1 --2- 172.21.15.12:0/495463613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd428009e50 tx=0x7fd428009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d27e5a1d80bacfb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd435d9b700 1 -- 172.21.15.12:0/495463613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4280092e0 con 0x7fd438078cc0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd435d9b700 1 -- 172.21.15.12:0/495463613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd428012430 con 0x7fd438078cc0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd435d9b700 1 -- 172.21.15.12:0/495463613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4280075c0 con 0x7fd438078cc0 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 -- 172.21.15.12:0/495463613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 msgr2=0x7fd438077220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 --2- 172.21.15.12:0/495463613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd428009e50 tx=0x7fd428009840 comp rx=0 tx=0).stop 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 -- 172.21.15.12:0/495463613 shutdown_connections 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 --2- 172.21.15.12:0/495463613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438077220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 -- 172.21.15.12:0/495463613 >> 172.21.15.12:0/495463613 conn(0x7fd4380fde40 msgr2=0x7fd4381002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 -- 172.21.15.12:0/495463613 shutdown_connections 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.486+0000 7fd43d03a700 1 -- 172.21.15.12:0/495463613 wait complete. 2024-03-18T14:41:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd43d03a700 1 Processor -- start 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd43d03a700 1 -- start start 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd43d03a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd43d03a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438113d50 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43374/0 (socket says 172.21.15.12:43374) 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd436d9d700 1 -- 172.21.15.12:0/2536553855 learned_addr learned my addr 172.21.15.12:0/2536553855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.487+0000 7fd436d9d700 1 -- 172.21.15.12:0/2536553855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009ae0 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd436d9d700 1 --2- 172.21.15.12:0/2536553855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd428009fd0 tx=0x7fd428009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4280092e0 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd428007d10 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd438113f50 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd42801c8c0 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.488+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd438114ac0 con 0x7fd438078cc0 2024-03-18T14:41:47.054 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.489+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7fd428019070 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.489+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43805f5f0 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.489+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd428049990 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.491+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd428019910 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.563+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd438114e60 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.564+0000 7fd42f7fe700 1 -- 172.21.15.12:0/2536553855 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fd4280174f0 con 0x7fd438078cc0 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 msgr2=0x7fd438113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 --2- 172.21.15.12:0/2536553855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd428009fd0 tx=0x7fd428009840 comp rx=0 tx=0).stop 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 shutdown_connections 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 --2- 172.21.15.12:0/2536553855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd438078cc0 0x7fd438113810 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 >> 172.21.15.12:0/2536553855 conn(0x7fd4380fde40 msgr2=0x7fd4380feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 shutdown_connections 2024-03-18T14:41:47.055 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:46.565+0000 7fd43d03a700 1 -- 172.21.15.12:0/2536553855 wait complete. 2024-03-18T14:41:47.056 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-03-18T14:41:47.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:47 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2536553855' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T14:41:47.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:47 smithi012 ceph-mon[29182]: mgrmap e3: smithi012.tmscgt(active, since 1.006s) 2024-03-18T14:41:50.172 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:49 smithi012 ceph-mon[29182]: mgrmap e4: smithi012.tmscgt(active, since 2s) 2024-03-18T14:41:50.172 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:49 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1682276792' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T14:41:50.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:50.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:41:50.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "74f0a062-e535-11ee-95c9-87774f69a715", 2024-03-18T14:41:50.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T14:41:50.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "smithi012" 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:50.315 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T14:41:50.316 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T14:41:50.317 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T14:41:25.323607+0000", 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T14:41:50.318 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.466+0000 7f3a24b98700 1 Processor -- start 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a24b98700 1 -- start start 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a24b98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a24b98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200790f0 con 0x7f3a20078ce0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59804/0 (socket says 172.21.15.12:59804) 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a1e59c700 1 -- 172.21.15.12:0/1370823310 learned_addr learned my addr 172.21.15.12:0/1370823310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.467+0000 7f3a1e59c700 1 -- 172.21.15.12:0/1370823310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20077720 con 0x7f3a20078ce0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a1e59c700 1 --2- 172.21.15.12:0/1370823310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098e0 tx=0x7f3a14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293885d6d6d42115 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a1d59a700 1 -- 172.21.15.12:0/1370823310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a1400b860 con 0x7f3a20078ce0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a1d59a700 1 -- 172.21.15.12:0/1370823310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3a1400be50 con 0x7f3a20078ce0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a1d59a700 1 -- 172.21.15.12:0/1370823310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a14010380 con 0x7f3a20078ce0 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a24b98700 1 -- 172.21.15.12:0/1370823310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 msgr2=0x7f3a200771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:50.319 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.468+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1370823310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098e0 tx=0x7f3a14009250 comp rx=0 tx=0).stop 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- 172.21.15.12:0/1370823310 shutdown_connections 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1370823310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a200771e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- 172.21.15.12:0/1370823310 >> 172.21.15.12:0/1370823310 conn(0x7f3a200fdf10 msgr2=0x7f3a20100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- 172.21.15.12:0/1370823310 shutdown_connections 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- 172.21.15.12:0/1370823310 wait complete. 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 Processor -- start 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- start start 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a24b98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20113d30 con 0x7f3a20078ce0 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59816/0 (socket says 172.21.15.12:59816) 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.469+0000 7f3a1e59c700 1 -- 172.21.15.12:0/1682276792 learned_addr learned my addr 172.21.15.12:0/1682276792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a1e59c700 1 -- 172.21.15.12:0/1682276792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140094f0 con 0x7f3a20078ce0 2024-03-18T14:41:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a1e59c700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400b440 tx=0x7f3a14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a14010760 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a20113f30 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3a14010d50 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a1401adc0 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.470+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a20114aa0 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.471+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f3a14019410 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.471+0000 7f3a0effd700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 0x7f3a08038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.471+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3a14049f90 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.471+0000 7f3a1dd9b700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 0x7f3a08038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.471+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a201143c0 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.472+0000 7f3a1dd9b700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 0x7f3a08038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.476+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3a14023940 con 0x7f3a20078ce0 2024-03-18T14:41:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.748+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3a2005f5f0 con 0x7f3a20078ce0 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.749+0000 7f3a0effd700 1 -- 172.21.15.12:0/1682276792 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f3a14023350 con 0x7f3a20078ce0 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 msgr2=0x7f3a08038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 0x7f3a08038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10006e40 comp rx=0 tx=0).stop 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 msgr2=0x7f3a201137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400b440 tx=0x7f3a14009250 comp rx=0 tx=0).stop 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 shutdown_connections 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f3a08036720 0x7f3a08038bd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 --2- 172.21.15.12:0/1682276792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a20078ce0 0x7f3a201137f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 >> 172.21.15.12:0/1682276792 conn(0x7f3a200fdf10 msgr2=0x7f3a200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 shutdown_connections 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:49.751+0000 7f3a24b98700 1 -- 172.21.15.12:0/1682276792 wait complete. 2024-03-18T14:41:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-03-18T14:41:51.205 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:50 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3317721693' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T14:41:51.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:51.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T14:41:51.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:41:51.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T14:41:51.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd7f6b700 1 Processor -- start 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd7f6b700 1 -- start start 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd7f6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd7f6b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0103080 con 0x7f0cd0102730 2024-03-18T14:41:51.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd5d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd5d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59820/0 (socket says 172.21.15.12:59820) 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd5d07700 1 -- 172.21.15.12:0/2870039762 learned_addr learned my addr 172.21.15.12:0/2870039762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.699+0000 7f0cd5d07700 1 -- 172.21.15.12:0/2870039762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd01031c0 con 0x7f0cd0102730 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd5d07700 1 --2- 172.21.15.12:0/2870039762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0098e0 tx=0x7f0ccc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c073c8ace12ff95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd4d05700 1 -- 172.21.15.12:0/2870039762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ccc00b860 con 0x7f0cd0102730 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd4d05700 1 -- 172.21.15.12:0/2870039762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0ccc00be50 con 0x7f0cd0102730 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd4d05700 1 -- 172.21.15.12:0/2870039762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ccc010380 con 0x7f0cd0102730 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/2870039762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 msgr2=0x7f0cd0102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/2870039762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0098e0 tx=0x7f0ccc009250 comp rx=0 tx=0).stop 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/2870039762 shutdown_connections 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/2870039762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd0102b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/2870039762 >> 172.21.15.12:0/2870039762 conn(0x7f0cd00fdf10 msgr2=0x7f0cd0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.700+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/2870039762 shutdown_connections 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/2870039762 wait complete. 2024-03-18T14:41:51.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd7f6b700 1 Processor -- start 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd7f6b700 1 -- start start 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd7f6b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd7f6b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0111b30 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd5d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd5d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59836/0 (socket says 172.21.15.12:59836) 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.701+0000 7f0cd5d07700 1 -- 172.21.15.12:0/3317721693 learned_addr learned my addr 172.21.15.12:0/3317721693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cd5d07700 1 -- 172.21.15.12:0/3317721693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0094f0 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cd5d07700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b440 tx=0x7f0ccc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ccc010760 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0ccc010d50 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0111d30 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ccc01adc0 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.702+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd01128a0 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f0ccc019410 con 0x7f0cd0102730 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cc67fc700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 0x7f0cbc038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:51.477 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0ccc04a0a0 con 0x7f0cd0102730 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd01121c0 con 0x7f0cd0102730 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cd5506700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 0x7f0cbc038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.703+0000 7f0cd5506700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 0x7f0cbc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009990 tx=0x7f0cc0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.708+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0ccc023940 con 0x7f0cd0102730 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.900+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0cd0104670 con 0x7f0cd0102730 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.901+0000 7f0cc67fc700 1 -- 172.21.15.12:0/3317721693 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0ccc023350 con 0x7f0cd0102730 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 msgr2=0x7f0cbc038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 0x7f0cbc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009990 tx=0x7f0cc0006e40 comp rx=0 tx=0).stop 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 msgr2=0x7f0cd01115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b440 tx=0x7f0ccc009250 comp rx=0 tx=0).stop 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 shutdown_connections 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f0cbc036720 0x7f0cbc038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:51.478 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 --2- 172.21.15.12:0/3317721693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0cd0102730 0x7f0cd01115f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:51.479 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 >> 172.21.15.12:0/3317721693 conn(0x7f0cd00fdf10 msgr2=0x7f0cd00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:51.479 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 shutdown_connections 2024-03-18T14:41:51.479 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:50.903+0000 7f0cd7f6b700 1 -- 172.21.15.12:0/3317721693 wait complete. 2024-03-18T14:41:51.479 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-03-18T14:41:52.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:52 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/714288612' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-18T14:41:52.795 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.907+0000 7f95c7663700 1 Processor -- start 2024-03-18T14:41:52.795 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c7663700 1 -- start start 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c7663700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c7663700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00790b0 con 0x7f95c0078ca0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c53ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c53ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59842/0 (socket says 172.21.15.12:59842) 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c53ff700 1 -- 172.21.15.12:0/126635916 learned_addr learned my addr 172.21.15.12:0/126635916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c53ff700 1 -- 172.21.15.12:0/126635916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0077740 con 0x7f95c0078ca0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95c53ff700 1 --2- 172.21.15.12:0/126635916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0098e0 tx=0x7f95bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6c1b66d548bf40 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.908+0000 7f95b7fff700 1 -- 172.21.15.12:0/126635916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95bc00b860 con 0x7f95c0078ca0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95b7fff700 1 -- 172.21.15.12:0/126635916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95bc00be50 con 0x7f95c0078ca0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95b7fff700 1 -- 172.21.15.12:0/126635916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95bc010380 con 0x7f95c0078ca0 2024-03-18T14:41:52.796 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 -- 172.21.15.12:0/126635916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 msgr2=0x7f95c0077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 --2- 172.21.15.12:0/126635916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0098e0 tx=0x7f95bc009250 comp rx=0 tx=0).stop 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 -- 172.21.15.12:0/126635916 shutdown_connections 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 --2- 172.21.15.12:0/126635916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0077200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 -- 172.21.15.12:0/126635916 >> 172.21.15.12:0/126635916 conn(0x7f95c00fdea0 msgr2=0x7f95c01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 -- 172.21.15.12:0/126635916 shutdown_connections 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.909+0000 7f95c7663700 1 -- 172.21.15.12:0/126635916 wait complete. 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c7663700 1 Processor -- start 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c7663700 1 -- start start 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c7663700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c7663700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0113d70 con 0x7f95c0078ca0 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c53ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c53ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59852/0 (socket says 172.21.15.12:59852) 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c53ff700 1 -- 172.21.15.12:0/714288612 learned_addr learned my addr 172.21.15.12:0/714288612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.910+0000 7f95c53ff700 1 -- 172.21.15.12:0/714288612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0094f0 con 0x7f95c0078ca0 2024-03-18T14:41:52.797 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95c53ff700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95bc011bd0 tx=0x7f95bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95bc0107c0 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95bc010db0 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95bc01ad00 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c0113f70 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.911+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c0114ae0 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.912+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f95bc01a560 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.912+0000 7f95b5ffb700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 0x7f95ac038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.912+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f95bc049ff0 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.912+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c005f5f0 con 0x7f95c0078ca0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.912+0000 7f95c4bfe700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 0x7f95ac038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.913+0000 7f95c4bfe700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 0x7f95ac038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009990 tx=0x7f95b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:52.798 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:51.917+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f95bc020020 con 0x7f95c0078ca0 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.163+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f95c0114de0 con 0x7f95c0078ca0 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.215+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f95bc075db0 con 0x7f95c0078ca0 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.217+0000 7f95b5ffb700 1 -- 172.21.15.12:0/714288612 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f95bc024770 con 0x7f95c0078ca0 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 msgr2=0x7f95ac038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 0x7f95ac038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009990 tx=0x7f95b0006e40 comp rx=0 tx=0).stop 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 msgr2=0x7f95c0113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95bc011bd0 tx=0x7f95bc009250 comp rx=0 tx=0).stop 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 shutdown_connections 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f95ac036790 0x7f95ac038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 --2- 172.21.15.12:0/714288612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c0078ca0 0x7f95c0113830 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 >> 172.21.15.12:0/714288612 conn(0x7f95c00fdea0 msgr2=0x7f95c00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 shutdown_connections 2024-03-18T14:41:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:52.220+0000 7f95c7663700 1 -- 172.21.15.12:0/714288612 wait complete. 2024-03-18T14:41:53.435 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:53 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/714288612' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-18T14:41:53.435 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:53 smithi012 ceph-mon[29182]: mgrmap e5: smithi012.tmscgt(active, since 5s) 2024-03-18T14:41:54.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:41:54.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-18T14:41:54.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T14:41:54.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "smithi012.tmscgt", 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd793a0700 1 Processor -- start 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd793a0700 1 -- start start 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd793a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd793a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd741046f0 con 0x7efd74103da0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59880/0 (socket says 172.21.15.12:59880) 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd72ffd700 1 -- 172.21.15.12:0/3356635348 learned_addr learned my addr 172.21.15.12:0/3356635348 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.228+0000 7efd72ffd700 1 -- 172.21.15.12:0/3356635348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74104830 con 0x7efd74103da0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd72ffd700 1 --2- 172.21.15.12:0/3356635348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd640098e0 tx=0x7efd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d4515f330680f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd71ffb700 1 -- 172.21.15.12:0/3356635348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd6400b860 con 0x7efd74103da0 2024-03-18T14:41:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd71ffb700 1 -- 172.21.15.12:0/3356635348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efd6400be50 con 0x7efd74103da0 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd71ffb700 1 -- 172.21.15.12:0/3356635348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd64010380 con 0x7efd74103da0 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 -- 172.21.15.12:0/3356635348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 msgr2=0x7efd741041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 --2- 172.21.15.12:0/3356635348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd640098e0 tx=0x7efd64009250 comp rx=0 tx=0).stop 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 -- 172.21.15.12:0/3356635348 shutdown_connections 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 --2- 172.21.15.12:0/3356635348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741041b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 -- 172.21.15.12:0/3356635348 >> 172.21.15.12:0/3356635348 conn(0x7efd740fc560 msgr2=0x7efd740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.229+0000 7efd793a0700 1 -- 172.21.15.12:0/3356635348 shutdown_connections 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd793a0700 1 -- 172.21.15.12:0/3356635348 wait complete. 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd793a0700 1 Processor -- start 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd793a0700 1 -- start start 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd793a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd793a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd74111b20 con 0x7efd74103da0 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.230+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:41:54.147 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59896/0 (socket says 172.21.15.12:59896) 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd72ffd700 1 -- 172.21.15.12:0/2850870927 learned_addr learned my addr 172.21.15.12:0/2850870927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd72ffd700 1 -- 172.21.15.12:0/2850870927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640094f0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd72ffd700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd6400ba90 tx=0x7efd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd640107c0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd74111d20 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efd64010db0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd6401add0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.231+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd741120f0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efd6402e440 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd5b7fe700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 0x7efd5c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efd6404b540 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd727fc700 1 -- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 msgr2=0x7efd5c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd727fc700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 0x7efd5c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.232+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7405f5f0 con 0x7efd74103da0 2024-03-18T14:41:54.148 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.237+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7efd64018940 con 0x7efd74103da0 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.433+0000 7efd727fc700 1 -- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 msgr2=0x7efd5c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.433+0000 7efd727fc700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 0x7efd5c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.486+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7efd74112ad0 con 0x7efd74103da0 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.486+0000 7efd5b7fe700 1 -- 172.21.15.12:0/2850870927 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7efd64018350 con 0x7efd74103da0 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 msgr2=0x7efd5c038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 0x7efd5c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 msgr2=0x7efd741115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd6400ba90 tx=0x7efd64009250 comp rx=0 tx=0).stop 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 shutdown_connections 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7efd5c036790 0x7efd5c038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 --2- 172.21.15.12:0/2850870927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd74103da0 0x7efd741115e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.488+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 >> 172.21.15.12:0/2850870927 conn(0x7efd740fc560 msgr2=0x7efd740fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.489+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 shutdown_connections 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:53.489+0000 7efd793a0700 1 -- 172.21.15.12:0/2850870927 wait complete. 2024-03-18T14:41:54.149 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-03-18T14:41:54.150 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-03-18T14:41:54.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:54 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2850870927' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: Active manager daemon smithi012.tmscgt restarted 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: osdmap e2: 0 total, 0 up, 0 in 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: mgrmap e6: smithi012.tmscgt(active, starting, since 0.00304379s) 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:41:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:41:59.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:58 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:00.208 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:59 smithi012 ceph-mon[29182]: Found migration_current of "None". Setting to last migration. 2024-03-18T14:42:00.209 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:59 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:42:00.209 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:59 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:42:00.209 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:41:59 smithi012 ceph-mon[29182]: mgrmap e7: smithi012.tmscgt(active, since 1.00509s) 2024-03-18T14:42:00.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:42:00.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-18T14:42:00.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T14:42:00.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:42:00.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f2665d2f700 1 Processor -- start 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f2665d2f700 1 -- start start 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f2665d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f2665d2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660103080 con 0x7f2660102730 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59898/0 (socket says 172.21.15.12:59898) 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f265f7fe700 1 -- 172.21.15.12:0/1906593414 learned_addr learned my addr 172.21.15.12:0/1906593414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f265f7fe700 1 -- 172.21.15.12:0/1906593414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26601031c0 con 0x7f2660102730 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.585+0000 7f265f7fe700 1 --2- 172.21.15.12:0/1906593414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26540098e0 tx=0x7f2654009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef27005a16af953 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f265e7fc700 1 -- 172.21.15.12:0/1906593414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f265400b860 con 0x7f2660102730 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f265e7fc700 1 -- 172.21.15.12:0/1906593414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f265400be50 con 0x7f2660102730 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f265e7fc700 1 -- 172.21.15.12:0/1906593414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2654010380 con 0x7f2660102730 2024-03-18T14:42:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 -- 172.21.15.12:0/1906593414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 msgr2=0x7f2660102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 --2- 172.21.15.12:0/1906593414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26540098e0 tx=0x7f2654009250 comp rx=0 tx=0).stop 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 -- 172.21.15.12:0/1906593414 shutdown_connections 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 --2- 172.21.15.12:0/1906593414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660102b40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 -- 172.21.15.12:0/1906593414 >> 172.21.15.12:0/1906593414 conn(0x7f26600fdf10 msgr2=0x7f2660100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 -- 172.21.15.12:0/1906593414 shutdown_connections 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.586+0000 7f2665d2f700 1 -- 172.21.15.12:0/1906593414 wait complete. 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f2665d2f700 1 Processor -- start 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f2665d2f700 1 -- start start 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f2665d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f2665d2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660079110 con 0x7f2660102730 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f265f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59908/0 (socket says 172.21.15.12:59908) 2024-03-18T14:42:00.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f265f7fe700 1 -- 172.21.15.12:0/792824622 learned_addr learned my addr 172.21.15.12:0/792824622 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.587+0000 7f265f7fe700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540094f0 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f265f7fe700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26540107c0 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2660077180 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2654010db0 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f265401ada0 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.588+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660077670 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2654019410 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f2647fff700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26480393a0 con 0x7f26480367e0 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2654049d20 con 0x7f2660102730 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f265effd700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:42:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.589+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.790+0000 7f265effd700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:54.790+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:55.191+0000 7f265effd700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:55.191+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:55.992+0000 7f265effd700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:55.992+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:57.593+0000 7f265effd700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/4273277683 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:57.593+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:58.907+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f265400ed40 con 0x7f2660102730 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:58.907+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 msgr2=0x7f2648038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:58.907+0000 7f2647fff700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.908+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f26540196c0 con 0x7f2660102730 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.908+0000 7f2647fff700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 0x7f264803c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:00.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.908+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26480393a0 con 0x7f2648039fc0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.908+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 0x7f264803c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.909+0000 7f265effd700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 0x7f264803c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f26500039d0 tx=0x7f2650009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.910+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f26480393a0 con 0x7f2648039fc0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.915+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f266004bed0 con 0x7f2648039fc0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2647fff700 1 -- 172.21.15.12:0/792824622 <== mgr.14120 v2:172.21.15.12:6800/3609733177 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f266004bed0 con 0x7f2648039fc0 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 msgr2=0x7f264803c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 0x7f264803c3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f26500039d0 tx=0x7f2650009350 comp rx=0 tx=0).stop 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 msgr2=0x7f2660078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2654006fd0 tx=0x7f2654009250 comp rx=0 tx=0).stop 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 shutdown_connections 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2648039fc0 0x7f264803c3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2660102730 0x7f2660078bd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 --2- 172.21.15.12:0/792824622 >> [v2:172.21.15.12:6800/4273277683,v1:172.21.15.12:6801/4273277683] conn(0x7f26480367e0 0x7f2648038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:00.433 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 >> 172.21.15.12:0/792824622 conn(0x7f26600fdf10 msgr2=0x7f26600feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:00.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 shutdown_connections 2024-03-18T14:42:00.434 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:41:59.916+0000 7f2665d2f700 1 -- 172.21.15.12:0/792824622 wait complete. 2024-03-18T14:42:00.434 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-03-18T14:42:00.434 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-03-18T14:42:01.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:00 smithi012 ceph-mon[29182]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T14:42:01.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:00 smithi012 ceph-mon[29182]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af6c2a700 1 Processor -- start 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af6c2a700 1 -- start start 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af6c2a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af6c2a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0103090 con 0x7f5af0102740 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53006/0 (socket says 172.21.15.12:53006) 2024-03-18T14:42:01.665 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af49c6700 1 -- 172.21.15.12:0/2799640404 learned_addr learned my addr 172.21.15.12:0/2799640404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.832+0000 7f5af49c6700 1 -- 172.21.15.12:0/2799640404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af01031d0 con 0x7f5af0102740 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af49c6700 1 --2- 172.21.15.12:0/2799640404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907cc1f99b551e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5aef7fe700 1 -- 172.21.15.12:0/2799640404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ae400b860 con 0x7f5af0102740 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5aef7fe700 1 -- 172.21.15.12:0/2799640404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5ae400be50 con 0x7f5af0102740 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5aef7fe700 1 -- 172.21.15.12:0/2799640404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ae4010380 con 0x7f5af0102740 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 -- 172.21.15.12:0/2799640404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 msgr2=0x7f5af0102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/2799640404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 -- 172.21.15.12:0/2799640404 shutdown_connections 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/2799640404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af0102b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 -- 172.21.15.12:0/2799640404 >> 172.21.15.12:0/2799640404 conn(0x7f5af00fdea0 msgr2=0x7f5af01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 -- 172.21.15.12:0/2799640404 shutdown_connections 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.833+0000 7f5af6c2a700 1 -- 172.21.15.12:0/2799640404 wait complete. 2024-03-18T14:42:01.666 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af6c2a700 1 Processor -- start 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af6c2a700 1 -- start start 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af6c2a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af6c2a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0111af0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53014/0 (socket says 172.21.15.12:53014) 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.834+0000 7f5af49c6700 1 -- 172.21.15.12:0/1895615562 learned_addr learned my addr 172.21.15.12:0/1895615562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5af49c6700 1 -- 172.21.15.12:0/1895615562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40094f0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5af49c6700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400ba90 tx=0x7f5ae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ae40107c0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5ae4010db0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af0111cf0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ae401add0 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.835+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af0112860 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.836+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f5ae402e440 con 0x7f5af0102740 2024-03-18T14:42:01.667 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.836+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af005f5f0 con 0x7f5af0102740 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.836+0000 7f5aed7fa700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 0x7f5ae0038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.836+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5ae404a560 con 0x7f5af0102740 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.837+0000 7f5aeffff700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 0x7f5ae0038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.837+0000 7f5aeffff700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 0x7f5ae0038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009990 tx=0x7f5adc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.841+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5ae4018940 con 0x7f5af0102740 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:00.959+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5ae40243f0 con 0x7f5af0102740 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.055+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af00ff120 con 0x7f5ae0036660 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.066+0000 7f5aed7fa700 1 -- 172.21.15.12:0/1895615562 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5af00ff120 con 0x7f5ae0036660 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 msgr2=0x7f5ae0038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 0x7f5ae0038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009990 tx=0x7f5adc006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 msgr2=0x7f5af01115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400ba90 tx=0x7f5ae4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 shutdown_connections 2024-03-18T14:42:01.668 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5ae0036660 0x7f5ae0038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:01.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 --2- 172.21.15.12:0/1895615562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5af0102740 0x7f5af01115b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:01.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 >> 172.21.15.12:0/1895615562 conn(0x7f5af00fdea0 msgr2=0x7f5af00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:01.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 shutdown_connections 2024-03-18T14:42:01.669 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:01.069+0000 7f5af6c2a700 1 -- 172.21.15.12:0/1895615562 wait complete. 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: mgrmap e8: smithi012.tmscgt(active, since 2s) 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:01] ENGINE Bus STARTING 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:02.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:01 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.052+0000 7f92f9715700 1 Processor -- start 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.052+0000 7f92f9715700 1 -- start start 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.052+0000 7f92f9715700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.052+0000 7f92f9715700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4079090 con 0x7f92f4078c80 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:02.872 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53024/0 (socket says 172.21.15.12:53024) 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f2ffd700 1 -- 172.21.15.12:0/601787039 learned_addr learned my addr 172.21.15.12:0/601787039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f2ffd700 1 -- 172.21.15.12:0/601787039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4077720 con 0x7f92f4078c80 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f2ffd700 1 --2- 172.21.15.12:0/601787039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e50 tx=0x7f92ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f84641cfb807b6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f1ffb700 1 -- 172.21.15.12:0/601787039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92ec0092e0 con 0x7f92f4078c80 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f1ffb700 1 -- 172.21.15.12:0/601787039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f92ec012430 con 0x7f92f4078c80 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.053+0000 7f92f1ffb700 1 -- 172.21.15.12:0/601787039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92ec0075c0 con 0x7f92f4078c80 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 -- 172.21.15.12:0/601787039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 msgr2=0x7f92f40771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 --2- 172.21.15.12:0/601787039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e50 tx=0x7f92ec009840 comp rx=0 tx=0).stop 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 -- 172.21.15.12:0/601787039 shutdown_connections 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 --2- 172.21.15.12:0/601787039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f40771e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 -- 172.21.15.12:0/601787039 >> 172.21.15.12:0/601787039 conn(0x7f92f40fdee0 msgr2=0x7f92f41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 -- 172.21.15.12:0/601787039 shutdown_connections 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.054+0000 7f92f9715700 1 -- 172.21.15.12:0/601787039 wait complete. 2024-03-18T14:42:02.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f9715700 1 Processor -- start 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f9715700 1 -- start start 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f9715700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f9715700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4113d90 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53032/0 (socket says 172.21.15.12:53032) 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f2ffd700 1 -- 172.21.15.12:0/3738306697 learned_addr learned my addr 172.21.15.12:0/3738306697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f2ffd700 1 -- 172.21.15.12:0/3738306697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec009ae0 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f2ffd700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009fd0 tx=0x7f92ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92ec0092e0 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f92ec007d10 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.055+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f4113f90 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.056+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92ec01c8c0 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.056+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f4114b00 con 0x7f92f4078c80 2024-03-18T14:42:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.056+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f92ec019070 con 0x7f92f4078c80 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.056+0000 7f92eb7fe700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 0x7f92e0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.057+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f92ec049f30 con 0x7f92f4078c80 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.057+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f405f5f0 con 0x7f92f4078c80 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.057+0000 7f92f27fc700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 0x7f92e0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.057+0000 7f92f27fc700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 0x7f92e0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009990 tx=0x7f92dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.062+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f92ec017020 con 0x7f92f4078c80 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.257+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f40ff290 con 0x7f92e0036790 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.257+0000 7f92eb7fe700 1 -- 172.21.15.12:0/3738306697 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f92f40ff290 con 0x7f92e0036790 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 msgr2=0x7f92e0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 0x7f92e0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009990 tx=0x7f92dc006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 msgr2=0x7f92f4113850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009fd0 tx=0x7f92ec009840 comp rx=0 tx=0).stop 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 shutdown_connections 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f92e0036790 0x7f92e0038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:02.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 --2- 172.21.15.12:0/3738306697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f4078c80 0x7f92f4113850 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:02.876 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.259+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 >> 172.21.15.12:0/3738306697 conn(0x7f92f40fdee0 msgr2=0x7f92f40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:02.876 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.260+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 shutdown_connections 2024-03-18T14:42:02.876 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:02.260+0000 7f92f9715700 1 -- 172.21.15.12:0/3738306697 wait complete. 2024-03-18T14:42:02.876 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-03-18T14:42:03.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:02 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:01] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:42:03.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:02 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:01] ENGINE Bus STARTED 2024-03-18T14:42:03.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:02 smithi012 ceph-mon[29182]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf2e47700 1 Processor -- start 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf2e47700 1 -- start start 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf2e47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf2e47700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec105930 con 0x7feaec076b70 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf0be3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf0be3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53036/0 (socket says 172.21.15.12:53036) 2024-03-18T14:42:04.637 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.344+0000 7feaf0be3700 1 -- 172.21.15.12:0/3798947155 learned_addr learned my addr 172.21.15.12:0/3798947155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.345+0000 7feaf0be3700 1 -- 172.21.15.12:0/3798947155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec105a70 con 0x7feaec076b70 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.345+0000 7feaf0be3700 1 --2- 172.21.15.12:0/3798947155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feae00098e0 tx=0x7feae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e851bb40fd5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.345+0000 7feaeb7fe700 1 -- 172.21.15.12:0/3798947155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feae000b860 con 0x7feaec076b70 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.345+0000 7feaeb7fe700 1 -- 172.21.15.12:0/3798947155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7feae000be50 con 0x7feaec076b70 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.345+0000 7feaeb7fe700 1 -- 172.21.15.12:0/3798947155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feae0010380 con 0x7feaec076b70 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- 172.21.15.12:0/3798947155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 msgr2=0x7feaec1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3798947155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feae00098e0 tx=0x7feae0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- 172.21.15.12:0/3798947155 shutdown_connections 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3798947155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1053f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- 172.21.15.12:0/3798947155 >> 172.21.15.12:0/3798947155 conn(0x7feaec0fce40 msgr2=0x7feaec0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- 172.21.15.12:0/3798947155 shutdown_connections 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- 172.21.15.12:0/3798947155 wait complete. 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 Processor -- start 2024-03-18T14:42:04.638 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.346+0000 7feaf2e47700 1 -- start start 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf2e47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf2e47700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec111b10 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf0be3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf0be3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53046/0 (socket says 172.21.15.12:53046) 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf0be3700 1 -- 172.21.15.12:0/3745257203 learned_addr learned my addr 172.21.15.12:0/3745257203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf0be3700 1 -- 172.21.15.12:0/3745257203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae00094f0 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feaf0be3700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feae0011bd0 tx=0x7feae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.347+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feae00107c0 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.348+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7feae0010db0 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.348+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec111d10 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.348+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feae001ad60 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.348+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec112880 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7feae002e440 con 0x7feaec076b70 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feae97fa700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 0x7feadc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:04.639 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec05f5f0 con 0x7feaec076b70 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7feae004adb0 con 0x7feaec076b70 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feaebfff700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 0x7feadc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.349+0000 7feaebfff700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 0x7feadc038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fead8009990 tx=0x7fead8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.354+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7feae0018350 con 0x7feaec076b70 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:03.548+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7feaec0fe210 con 0x7feadc036790 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.027+0000 7feae97fa700 1 -- 172.21.15.12:0/3745257203 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feaec0fe210 con 0x7feadc036790 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 msgr2=0x7feadc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 0x7feadc038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fead8009990 tx=0x7fead8006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 msgr2=0x7feaec1115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feae0011bd0 tx=0x7feae0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 shutdown_connections 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7feadc036790 0x7feadc038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 --2- 172.21.15.12:0/3745257203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feaec076b70 0x7feaec1115d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 >> 172.21.15.12:0/3745257203 conn(0x7feaec0fce40 msgr2=0x7feaec0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:04.640 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.029+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 shutdown_connections 2024-03-18T14:42:04.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:04.030+0000 7feaf2e47700 1 -- 172.21.15.12:0/3745257203 wait complete. 2024-03-18T14:42:05.236 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:05 smithi012 ceph-mon[29182]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:05.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:05 smithi012 ceph-mon[29182]: Generating ssh key... 2024-03-18T14:42:05.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:05 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:05.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:05 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4cw4yrZPdCMk2dyJahmh5HTnjntjZTrVE9j2kvxicSIGI85+NMPaOI9YCjYt5jInPhl9fPxHp7u30qPJqOgPhjOxEwdcaFEerTda1+X3j3DA+VNYCAshVwfvU56LRKJdY2LWAuK1tSJZkGxqQb+LAZZSmK865eA2CGYQ7MlzHltTgMoYL9t/zmv51/w/nw0Fc/E1dkUHoIMjsQdjGUEaKhCNhTKp/qoFhT7ogkr4saJEIj7ZwRKuiuIquqQOU3FYHszwu/x/I1AmLycH5IDUiIzBxZS9y03IbfuR36aHetQ/MVyjf8fEwqq8Kv3GoGMmN3NYX4m+s9TsQ9ss3qrYX7nE8z4E8K9OZrAVo1V75YCu72+qa+ypa06Sw3buwEUQo4we0T+p7UawEntIMKe9bZu8UkaLjlSrljBKscYl5Eqdy3W0TKa12j71h1oWb9VOomGJoCjuYsu24IHhaAFrCN1isWVpLZ5nMSNH8stNDiRLu6G0T8VaoeR1FFYErbL8= ceph-74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.030+0000 7f5916451700 1 Processor -- start 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.030+0000 7f5916451700 1 -- start start 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f5916451700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f5916451700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910101f20 con 0x7f59101015d0 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f590ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f590ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53056/0 (socket says 172.21.15.12:53056) 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f590ffff700 1 -- 172.21.15.12:0/4136669552 learned_addr learned my addr 172.21.15.12:0/4136669552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f590ffff700 1 -- 172.21.15.12:0/4136669552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910102060 con 0x7f59101015d0 2024-03-18T14:42:05.802 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.031+0000 7f590ffff700 1 --2- 172.21.15.12:0/4136669552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59000098e0 tx=0x7f5900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f53cfe991a8439 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f590effd700 1 -- 172.21.15.12:0/4136669552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f590000b860 con 0x7f59101015d0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f590effd700 1 -- 172.21.15.12:0/4136669552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f590000be50 con 0x7f59101015d0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f590effd700 1 -- 172.21.15.12:0/4136669552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5900010380 con 0x7f59101015d0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 -- 172.21.15.12:0/4136669552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 msgr2=0x7f59101019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 --2- 172.21.15.12:0/4136669552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59000098e0 tx=0x7f5900009250 comp rx=0 tx=0).stop 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 -- 172.21.15.12:0/4136669552 shutdown_connections 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 --2- 172.21.15.12:0/4136669552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f59101019e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 -- 172.21.15.12:0/4136669552 >> 172.21.15.12:0/4136669552 conn(0x7f59100fcd50 msgr2=0x7f59100ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 -- 172.21.15.12:0/4136669552 shutdown_connections 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.032+0000 7f5916451700 1 -- 172.21.15.12:0/4136669552 wait complete. 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f5916451700 1 Processor -- start 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f5916451700 1 -- start start 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f5916451700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f5916451700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591010f8c0 con 0x7f59101015d0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f590ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:05.803 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f590ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53072/0 (socket says 172.21.15.12:53072) 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.033+0000 7f590ffff700 1 -- 172.21.15.12:0/1210447643 learned_addr learned my addr 172.21.15.12:0/1210447643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f590ffff700 1 -- 172.21.15.12:0/1210447643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000094f0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f590ffff700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f590000ba90 tx=0x7f5900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59000107c0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5900010db0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591010fac0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f590001add0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.034+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910110630 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.035+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f590002e440 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.035+0000 7f590cff9700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 0x7f58f803aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.035+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f590004a6f0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.035+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591005f5f0 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.035+0000 7f590f7fe700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 0x7f58f803aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.036+0000 7f590f7fe700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 0x7f58f803aee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5904009990 tx=0x7f5904006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.040+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5900018940 con 0x7f59101015d0 2024-03-18T14:42:05.804 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.234+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5910110a40 con 0x7f58f8038a30 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.235+0000 7f590cff9700 1 -- 172.21.15.12:0/1210447643 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f5910110a40 con 0x7f58f8038a30 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 msgr2=0x7f58f803aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 0x7f58f803aee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5904009990 tx=0x7f5904006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 msgr2=0x7f591010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f590000ba90 tx=0x7f5900009250 comp rx=0 tx=0).stop 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 shutdown_connections 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f58f8038a30 0x7f58f803aee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 --2- 172.21.15.12:0/1210447643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59101015d0 0x7f591010f380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 >> 172.21.15.12:0/1210447643 conn(0x7f59100fcd50 msgr2=0x7f5910105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 shutdown_connections 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:05.237+0000 7f5916451700 1 -- 172.21.15.12:0/1210447643 wait complete. 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-03-18T14:42:05.805 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-03-18T14:42:06.113 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:06 smithi012 ceph-mon[29182]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:07.576 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:07 smithi012 ceph-mon[29182]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:07.577 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:07 smithi012 ceph-mon[29182]: Deploying cephadm binary to smithi012 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc86516c700 1 Processor -- start 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc86516c700 1 -- start start 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc86516c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc86516c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8601030c0 con 0x7fc860102770 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53086/0 (socket says 172.21.15.12:53086) 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc85ed9d700 1 -- 172.21.15.12:0/3080761662 learned_addr learned my addr 172.21.15.12:0/3080761662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:09.078 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc85ed9d700 1 -- 172.21.15.12:0/3080761662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860103200 con 0x7fc860102770 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.219+0000 7fc85ed9d700 1 --2- 172.21.15.12:0/3080761662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098e0 tx=0x7fc850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3697a7b562e6604 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc85dd9b700 1 -- 172.21.15.12:0/3080761662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc85000b860 con 0x7fc860102770 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc85dd9b700 1 -- 172.21.15.12:0/3080761662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc85000be50 con 0x7fc860102770 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc85dd9b700 1 -- 172.21.15.12:0/3080761662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc850010380 con 0x7fc860102770 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 -- 172.21.15.12:0/3080761662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 msgr2=0x7fc860102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 --2- 172.21.15.12:0/3080761662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098e0 tx=0x7fc850009250 comp rx=0 tx=0).stop 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 -- 172.21.15.12:0/3080761662 shutdown_connections 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 --2- 172.21.15.12:0/3080761662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860102b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 -- 172.21.15.12:0/3080761662 >> 172.21.15.12:0/3080761662 conn(0x7fc8600fde90 msgr2=0x7fc860100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:09.079 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 -- 172.21.15.12:0/3080761662 shutdown_connections 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.220+0000 7fc86516c700 1 -- 172.21.15.12:0/3080761662 wait complete. 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc86516c700 1 Processor -- start 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc86516c700 1 -- start start 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc86516c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc86516c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860079110 con 0x7fc860102770 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc85ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53098/0 (socket says 172.21.15.12:53098) 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.221+0000 7fc85ed9d700 1 -- 172.21.15.12:0/2002268232 learned_addr learned my addr 172.21.15.12:0/2002268232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc85ed9d700 1 -- 172.21.15.12:0/2002268232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500094f0 con 0x7fc860102770 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc85ed9d700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc85000b440 tx=0x7fc850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc850010760 con 0x7fc860102770 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc850010d50 con 0x7fc860102770 2024-03-18T14:42:09.080 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc860077180 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc85001ae20 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.222+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc860077670 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.223+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc85002e440 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.223+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86004bed0 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.223+0000 7fc8477fe700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 0x7fc848038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.223+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc85004a6b0 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.223+0000 7fc85e59c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 0x7fc848038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.224+0000 7fc85e59c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 0x7fc848038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc854009990 tx=0x7fc854006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.228+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc850018940 con 0x7fc860102770 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:06.423+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) v1 -- 0x7fc860077d80 con 0x7fc8480367e0 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.460+0000 7fc8477fe700 1 -- 172.21.15.12:0/2002268232 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fc860077d80 con 0x7fc8480367e0 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 msgr2=0x7fc848038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 0x7fc848038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc854009990 tx=0x7fc854006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:09.081 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 msgr2=0x7fc860078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc85000b440 tx=0x7fc850009250 comp rx=0 tx=0).stop 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 shutdown_connections 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc8480367e0 0x7fc848038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 --2- 172.21.15.12:0/2002268232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc860102770 0x7fc860078bd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 >> 172.21.15.12:0/2002268232 conn(0x7fc8600fde90 msgr2=0x7fc8600feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 shutdown_connections 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:08.462+0000 7fc86516c700 1 -- 172.21.15.12:0/2002268232 wait complete. 2024-03-18T14:42:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying mon service with default placement... 2024-03-18T14:42:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:09 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:09 smithi012 ceph-mon[29182]: Added host smithi012 2024-03-18T14:42:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:09 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:10.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:10 smithi012 ceph-mon[29182]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:10.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:10 smithi012 ceph-mon[29182]: Saving service mon spec with placement count:5 2024-03-18T14:42:10.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:10 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.598+0000 7fbe6286b700 1 Processor -- start 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.598+0000 7fbe6286b700 1 -- start start 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.598+0000 7fbe6286b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.598+0000 7fbe6286b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c063ce0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.598+0000 7fbe5bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39236/0 (socket says 172.21.15.12:39236) 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5bfff700 1 -- 172.21.15.12:0/1528291490 learned_addr learned my addr 172.21.15.12:0/1528291490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5bfff700 1 -- 172.21.15.12:0/1528291490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c104710 con 0x7fbe5c103dc0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5bfff700 1 --2- 172.21.15.12:0/1528291490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0098e0 tx=0x7fbe4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0315088ab2116 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5affd700 1 -- 172.21.15.12:0/1528291490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe4c00b860 con 0x7fbe5c103dc0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5affd700 1 -- 172.21.15.12:0/1528291490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbe4c00be50 con 0x7fbe5c103dc0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.599+0000 7fbe5affd700 1 -- 172.21.15.12:0/1528291490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe4c010380 con 0x7fbe5c103dc0 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- 172.21.15.12:0/1528291490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 msgr2=0x7fbe5c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 --2- 172.21.15.12:0/1528291490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0098e0 tx=0x7fbe4c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- 172.21.15.12:0/1528291490 shutdown_connections 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 --2- 172.21.15.12:0/1528291490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c1041d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- 172.21.15.12:0/1528291490 >> 172.21.15.12:0/1528291490 conn(0x7fbe5c0fc500 msgr2=0x7fbe5c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- 172.21.15.12:0/1528291490 shutdown_connections 2024-03-18T14:42:11.311 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- 172.21.15.12:0/1528291490 wait complete. 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 Processor -- start 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- start start 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.600+0000 7fbe6286b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c111ab0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe5bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe5bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39246/0 (socket says 172.21.15.12:39246) 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe5bfff700 1 -- 172.21.15.12:0/2048209953 learned_addr learned my addr 172.21.15.12:0/2048209953 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe5bfff700 1 -- 172.21.15.12:0/2048209953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0094f0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe5bfff700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c000c00 tx=0x7fbe4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe4c0107c0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbe4c010db0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c111cb0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe4c01ad00 con 0x7fbe5c103dc0 2024-03-18T14:42:11.312 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.601+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c112820 con 0x7fbe5c103dc0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.602+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbe4c0193c0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.602+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c05f5f0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.602+0000 7fbe58ff9700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 0x7fbe48038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.602+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbe4c04a800 con 0x7fbe5c103dc0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.602+0000 7fbe5b7fe700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 0x7fbe48038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.603+0000 7fbe5b7fe700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 0x7fbe48038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.607+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fbe4c018af0 con 0x7fbe5c103dc0 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.816+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe5c0fd7f0 con 0x7fbe48036790 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.823+0000 7fbe58ff9700 1 -- 172.21.15.12:0/2048209953 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbe5c0fd7f0 con 0x7fbe48036790 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 msgr2=0x7fbe48038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 0x7fbe48038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 msgr2=0x7fbe5c111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c000c00 tx=0x7fbe4c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 shutdown_connections 2024-03-18T14:42:11.313 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fbe48036790 0x7fbe48038c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 --2- 172.21.15.12:0/2048209953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe5c103dc0 0x7fbe5c111570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 >> 172.21.15.12:0/2048209953 conn(0x7fbe5c0fc500 msgr2=0x7fbe5c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 shutdown_connections 2024-03-18T14:42:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:09.825+0000 7fbe6286b700 1 -- 172.21.15.12:0/2048209953 wait complete. 2024-03-18T14:42:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying mgr service with default placement... 2024-03-18T14:42:12.220 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:11 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:12.220 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:11 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:13.055 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:12 smithi012 ceph-mon[29182]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:13.055 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:12 smithi012 ceph-mon[29182]: Saving service mgr spec with placement count:2 2024-03-18T14:42:13.055 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:12 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febdbbc2700 1 Processor -- start 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febdbbc2700 1 -- start start 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febdbbc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febdbbc2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4070df0 con 0x7febd4103d60 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febd995e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febd995e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39256/0 (socket says 172.21.15.12:39256) 2024-03-18T14:42:13.062 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febd995e700 1 -- 172.21.15.12:0/3521102886 learned_addr learned my addr 172.21.15.12:0/3521102886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.760+0000 7febd995e700 1 -- 172.21.15.12:0/3521102886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd41046b0 con 0x7febd4103d60 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febd995e700 1 --2- 172.21.15.12:0/3521102886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7febd00098e0 tx=0x7febd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b648a804defde89d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febd895c700 1 -- 172.21.15.12:0/3521102886 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd000b860 con 0x7febd4103d60 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febd895c700 1 -- 172.21.15.12:0/3521102886 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd000be90 con 0x7febd4103d60 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febd895c700 1 -- 172.21.15.12:0/3521102886 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd00103c0 con 0x7febd4103d60 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 -- 172.21.15.12:0/3521102886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 msgr2=0x7febd4104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 --2- 172.21.15.12:0/3521102886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7febd00098e0 tx=0x7febd0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 -- 172.21.15.12:0/3521102886 shutdown_connections 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 --2- 172.21.15.12:0/3521102886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd4104170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 -- 172.21.15.12:0/3521102886 >> 172.21.15.12:0/3521102886 conn(0x7febd40fc4a0 msgr2=0x7febd40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 -- 172.21.15.12:0/3521102886 shutdown_connections 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.761+0000 7febdbbc2700 1 -- 172.21.15.12:0/3521102886 wait complete. 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febdbbc2700 1 Processor -- start 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febdbbc2700 1 -- start start 2024-03-18T14:42:13.063 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febdbbc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febdbbc2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4111ae0 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febd995e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febd995e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39270/0 (socket says 172.21.15.12:39270) 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febd995e700 1 -- 172.21.15.12:0/839828006 learned_addr learned my addr 172.21.15.12:0/839828006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.762+0000 7febd995e700 1 -- 172.21.15.12:0/839828006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00094f0 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febd995e700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febd00090b0 tx=0x7febd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd0010840 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd0010e70 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4111ce0 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd0024e40 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.763+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4112850 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7febd001a4c0 con 0x7febd4103d60 2024-03-18T14:42:13.064 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febca7fc700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 0x7febc0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd405f5f0 con 0x7febd4103d60 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febd915d700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 0x7febc0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7febd004a760 con 0x7febd4103d60 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.764+0000 7febd915d700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 0x7febc0038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febc4009990 tx=0x7febc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.769+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7febd00109a0 con 0x7febd4103d60 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.982+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7febd40fd750 con 0x7febc0036790 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.987+0000 7febca7fc700 1 -- 172.21.15.12:0/839828006 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7febd40fd750 con 0x7febc0036790 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 msgr2=0x7febc0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 0x7febc0038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febc4009990 tx=0x7febc4006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 msgr2=0x7febd41115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febd00090b0 tx=0x7febd0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 shutdown_connections 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7febc0036790 0x7febc0038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 --2- 172.21.15.12:0/839828006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febd4103d60 0x7febd41115a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:13.065 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 >> 172.21.15.12:0/839828006 conn(0x7febd40fc4a0 msgr2=0x7febd40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:13.066 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 shutdown_connections 2024-03-18T14:42:13.066 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:11.989+0000 7febdbbc2700 1 -- 172.21.15.12:0/839828006 wait complete. 2024-03-18T14:42:13.066 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying crash service with default placement... 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f1699a7d700 1 Processor -- start 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f1699a7d700 1 -- start start 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f1699a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f1699a7d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694078010 con 0x7f1694077c00 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f16937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f16937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39286/0 (socket says 172.21.15.12:39286) 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.503+0000 7f16937fe700 1 -- 172.21.15.12:0/2209265589 learned_addr learned my addr 172.21.15.12:0/2209265589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:14.269 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.504+0000 7f16937fe700 1 -- 172.21.15.12:0/2209265589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1694076650 con 0x7f1694077c00 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.504+0000 7f16937fe700 1 --2- 172.21.15.12:0/2209265589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e50 tx=0x7f1684009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e8f116aadc06f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.504+0000 7f16927fc700 1 -- 172.21.15.12:0/2209265589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16840092e0 con 0x7f1694077c00 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.504+0000 7f16927fc700 1 -- 172.21.15.12:0/2209265589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1684012430 con 0x7f1694077c00 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.504+0000 7f16927fc700 1 -- 172.21.15.12:0/2209265589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16840075a0 con 0x7f1694077c00 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 -- 172.21.15.12:0/2209265589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 msgr2=0x7f1694076110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 --2- 172.21.15.12:0/2209265589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1684009e50 tx=0x7f1684009840 comp rx=0 tx=0).stop 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 -- 172.21.15.12:0/2209265589 shutdown_connections 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 --2- 172.21.15.12:0/2209265589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694076110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 -- 172.21.15.12:0/2209265589 >> 172.21.15.12:0/2209265589 conn(0x7f16940fde70 msgr2=0x7f1694100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 -- 172.21.15.12:0/2209265589 shutdown_connections 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.505+0000 7f1699a7d700 1 -- 172.21.15.12:0/2209265589 wait complete. 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f1699a7d700 1 Processor -- start 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f1699a7d700 1 -- start start 2024-03-18T14:42:14.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f1699a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f1699a7d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941181b0 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f16937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f16937fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39292/0 (socket says 172.21.15.12:39292) 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f16937fe700 1 -- 172.21.15.12:0/1459288609 learned_addr learned my addr 172.21.15.12:0/1459288609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f16937fe700 1 -- 172.21.15.12:0/1459288609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009ae0 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.506+0000 7f16937fe700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1684012f20 tx=0x7f1684009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16840092e0 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16941183b0 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1684007d30 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1684025930 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1694118f20 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.507+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1684019050 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.508+0000 7f168bfff700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 0x7f1680038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.508+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f168404a070 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.508+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169405f5f0 con 0x7f1694077c00 2024-03-18T14:42:14.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.508+0000 7f1692ffd700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 0x7f1680038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.508+0000 7f1692ffd700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 0x7f1680038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.513+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f1684020080 con 0x7f1694077c00 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.718+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f1694119220 con 0x7f1680036790 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.722+0000 7f168bfff700 1 -- 172.21.15.12:0/1459288609 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1694119220 con 0x7f1680036790 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 msgr2=0x7f1680038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 0x7f1680038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 msgr2=0x7f1694117c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1684012f20 tx=0x7f1684009840 comp rx=0 tx=0).stop 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 shutdown_connections 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f1680036790 0x7f1680038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 --2- 172.21.15.12:0/1459288609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1694077c00 0x7f1694117c70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.724+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 >> 172.21.15.12:0/1459288609 conn(0x7f16940fde70 msgr2=0x7f169410ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.725+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 shutdown_connections 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:13.725+0000 7f1699a7d700 1 -- 172.21.15.12:0/1459288609 wait complete. 2024-03-18T14:42:14.272 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying prometheus service with default placement... 2024-03-18T14:42:14.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:14 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:14.371 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:14 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:14.371 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:14 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:15.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:15 smithi012 ceph-mon[29182]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:15.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:15 smithi012 ceph-mon[29182]: Saving service crash spec with placement * 2024-03-18T14:42:15.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:15 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5d633b700 1 Processor -- start 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5d633b700 1 -- start start 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5d633b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5d633b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0103070 con 0x7fa5d0102720 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39296/0 (socket says 172.21.15.12:39296) 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.645+0000 7fa5cffff700 1 -- 172.21.15.12:0/2013331198 learned_addr learned my addr 172.21.15.12:0/2013331198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.646+0000 7fa5cffff700 1 -- 172.21.15.12:0/2013331198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d01031b0 con 0x7fa5d0102720 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.646+0000 7fa5cffff700 1 --2- 172.21.15.12:0/2013331198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000fa00 tx=0x7fa5c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66e44b14909ca31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.646+0000 7fa5ceffd700 1 -- 172.21.15.12:0/2013331198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5c0009940 con 0x7fa5d0102720 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.646+0000 7fa5ceffd700 1 -- 172.21.15.12:0/2013331198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5c000e4e0 con 0x7fa5d0102720 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.646+0000 7fa5ceffd700 1 -- 172.21.15.12:0/2013331198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5c00175c0 con 0x7fa5d0102720 2024-03-18T14:42:15.889 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 -- 172.21.15.12:0/2013331198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 msgr2=0x7fa5d0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2013331198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000fa00 tx=0x7fa5c0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 -- 172.21.15.12:0/2013331198 shutdown_connections 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2013331198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d0102b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 -- 172.21.15.12:0/2013331198 >> 172.21.15.12:0/2013331198 conn(0x7fa5d00fde40 msgr2=0x7fa5d01002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 -- 172.21.15.12:0/2013331198 shutdown_connections 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.647+0000 7fa5d633b700 1 -- 172.21.15.12:0/2013331198 wait complete. 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5d633b700 1 Processor -- start 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5d633b700 1 -- start start 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5d633b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5d633b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d010f8b0 con 0x7fa5d0102720 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5cffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39302/0 (socket says 172.21.15.12:39302) 2024-03-18T14:42:15.890 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5cffff700 1 -- 172.21.15.12:0/2134704014 learned_addr learned my addr 172.21.15.12:0/2134704014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.648+0000 7fa5cffff700 1 -- 172.21.15.12:0/2134704014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c000f610 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5cffff700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000edd0 tx=0x7fa5c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5c0017830 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d010fab0 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5c0017e60 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5c001f3e0 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.649+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d0110620 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.650+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc0052e0 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.650+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa5c001f660 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.650+0000 7fa5ccff9700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 0x7fa5b8038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.650+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa5c004e4a0 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.650+0000 7fa5cf7fe700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 0x7fa5b8038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.651+0000 7fa5cf7fe700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 0x7fa5b8038cc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009990 tx=0x7fa5c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.655+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fa5c001e930 con 0x7fa5d0102720 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.866+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5bc000bf0 con 0x7fa5b8036810 2024-03-18T14:42:15.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.870+0000 7fa5ccff9700 1 -- 172.21.15.12:0/2134704014 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fa5bc000bf0 con 0x7fa5b8036810 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 msgr2=0x7fa5b8038cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 0x7fa5b8038cc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009990 tx=0x7fa5c4006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 msgr2=0x7fa5d010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000edd0 tx=0x7fa5c0009250 comp rx=0 tx=0).stop 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 shutdown_connections 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fa5b8036810 0x7fa5b8038cc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 --2- 172.21.15.12:0/2134704014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5d0102720 0x7fa5d010f370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.872+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 >> 172.21.15.12:0/2134704014 conn(0x7fa5d00fde40 msgr2=0x7fa5d00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.873+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 shutdown_connections 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:14.873+0000 7fa5d633b700 1 -- 172.21.15.12:0/2134704014 wait complete. 2024-03-18T14:42:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying grafana service with default placement... 2024-03-18T14:42:16.405 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:16 smithi012 ceph-mon[29182]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:16.406 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:16 smithi012 ceph-mon[29182]: Saving service prometheus spec with placement count:1 2024-03-18T14:42:17.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2faffde700 1 Processor -- start 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2faffde700 1 -- start start 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2faffde700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2faffde700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81058f0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2fadd7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2fadd7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39306/0 (socket says 172.21.15.12:39306) 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2fadd7a700 1 -- 172.21.15.12:0/1489531849 learned_addr learned my addr 172.21.15.12:0/1489531849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.339+0000 7f2fadd7a700 1 -- 172.21.15.12:0/1489531849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8105a30 con 0x7f2fa8076bb0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.340+0000 7f2fadd7a700 1 --2- 172.21.15.12:0/1489531849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098e0 tx=0x7f2fa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee387819c1cb8a5c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.340+0000 7f2facd78700 1 -- 172.21.15.12:0/1489531849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fa400b860 con 0x7f2fa8076bb0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.340+0000 7f2facd78700 1 -- 172.21.15.12:0/1489531849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fa400be90 con 0x7f2fa8076bb0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.340+0000 7f2facd78700 1 -- 172.21.15.12:0/1489531849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fa40103c0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.340+0000 7f2faffde700 1 -- 172.21.15.12:0/1489531849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 msgr2=0x7f2fa81053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 --2- 172.21.15.12:0/1489531849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098e0 tx=0x7f2fa4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 -- 172.21.15.12:0/1489531849 shutdown_connections 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 --2- 172.21.15.12:0/1489531849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa81053b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 -- 172.21.15.12:0/1489531849 >> 172.21.15.12:0/1489531849 conn(0x7f2fa80fce40 msgr2=0x7f2fa80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 -- 172.21.15.12:0/1489531849 shutdown_connections 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 -- 172.21.15.12:0/1489531849 wait complete. 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 Processor -- start 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.341+0000 7f2faffde700 1 -- start start 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2faffde700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2faffde700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8111b50 con 0x7f2fa8076bb0 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2fadd7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2fadd7a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39316/0 (socket says 172.21.15.12:39316) 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2fadd7a700 1 -- 172.21.15.12:0/1847994281 learned_addr learned my addr 172.21.15.12:0/1847994281 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2fadd7a700 1 -- 172.21.15.12:0/1847994281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40094f0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2fadd7a700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40094c0 tx=0x7f2fa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:17.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fa40118f0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fa4010d60 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa8111d50 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.342+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2fa401a430 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.343+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa81128c0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.343+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2fa401a590 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.343+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa805f5f0 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.344+0000 7f2f9e7fc700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 0x7f2f94038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.344+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2fa404a680 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.344+0000 7f2fad579700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 0x7f2f94038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.344+0000 7f2fad579700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 0x7f2f94038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009990 tx=0x7f2f98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.348+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f2fa401a840 con 0x7f2fa8076bb0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.557+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa80fe270 con 0x7f2f940367e0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.562+0000 7f2f9e7fc700 1 -- 172.21.15.12:0/1847994281 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2fa80fe270 con 0x7f2f940367e0 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.563+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 msgr2=0x7f2f94038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.563+0000 7f2faffde700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 0x7f2f94038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009990 tx=0x7f2f98006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:17.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.563+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 msgr2=0x7f2fa8111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.563+0000 7f2faffde700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40094c0 tx=0x7f2fa4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 shutdown_connections 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f2f940367e0 0x7f2f94038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 --2- 172.21.15.12:0/1847994281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa8076bb0 0x7f2fa8111610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 >> 172.21.15.12:0/1847994281 conn(0x7f2fa80fce40 msgr2=0x7f2fa80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 shutdown_connections 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:16.564+0000 7f2faffde700 1 -- 172.21.15.12:0/1847994281 wait complete. 2024-03-18T14:42:17.310 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying node-exporter service with default placement... 2024-03-18T14:42:17.969 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:17 smithi012 ceph-mon[29182]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:17.969 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:17 smithi012 ceph-mon[29182]: Saving service grafana spec with placement count:1 2024-03-18T14:42:17.969 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:17 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:18.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-18T14:42:18.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.757+0000 7f780a875700 1 Processor -- start 2024-03-18T14:42:18.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.757+0000 7f780a875700 1 -- start start 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f780a875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f780a875700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7804103080 con 0x7f7804102730 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7803fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7803fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39330/0 (socket says 172.21.15.12:39330) 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7803fff700 1 -- 172.21.15.12:0/388892418 learned_addr learned my addr 172.21.15.12:0/388892418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7803fff700 1 -- 172.21.15.12:0/388892418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78041031c0 con 0x7f7804102730 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7803fff700 1 --2- 172.21.15.12:0/388892418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0098e0 tx=0x7f77ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e082a7d6c1aaffee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7802ffd700 1 -- 172.21.15.12:0/388892418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77ec00b860 con 0x7f7804102730 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7802ffd700 1 -- 172.21.15.12:0/388892418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77ec00be90 con 0x7f7804102730 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.758+0000 7f7802ffd700 1 -- 172.21.15.12:0/388892418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77ec0103c0 con 0x7f7804102730 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 -- 172.21.15.12:0/388892418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 msgr2=0x7f7804102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 --2- 172.21.15.12:0/388892418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0098e0 tx=0x7f77ec009250 comp rx=0 tx=0).stop 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 -- 172.21.15.12:0/388892418 shutdown_connections 2024-03-18T14:42:18.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 --2- 172.21.15.12:0/388892418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804102b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 -- 172.21.15.12:0/388892418 >> 172.21.15.12:0/388892418 conn(0x7f78040fdf10 msgr2=0x7f7804100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 -- 172.21.15.12:0/388892418 shutdown_connections 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 -- 172.21.15.12:0/388892418 wait complete. 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.759+0000 7f780a875700 1 Processor -- start 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f780a875700 1 -- start start 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f780a875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f780a875700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7804111b70 con 0x7f7804102730 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7803fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7803fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39346/0 (socket says 172.21.15.12:39346) 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7803fff700 1 -- 172.21.15.12:0/624670557 learned_addr learned my addr 172.21.15.12:0/624670557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7803fff700 1 -- 172.21.15.12:0/624670557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec0094f0 con 0x7f7804102730 2024-03-18T14:42:18.540 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7803fff700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0094c0 tx=0x7f77ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77ec010680 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77ec010cb0 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7804111d70 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.760+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77ec01ad40 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.761+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78041128e0 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.761+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f77ec025400 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.761+0000 7f7800ff9700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 0x7f77f0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.761+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f77ec04a3a0 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.762+0000 7f78037fe700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 0x7f77f0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.762+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780405f5f0 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.762+0000 7f78037fe700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 0x7f77f0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009990 tx=0x7f77f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.766+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f77ec020030 con 0x7f7804102730 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.967+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f78040ff2d0 con 0x7f77f00367e0 2024-03-18T14:42:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.971+0000 7f7800ff9700 1 -- 172.21.15.12:0/624670557 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f78040ff2d0 con 0x7f77f00367e0 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 msgr2=0x7f77f0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 0x7f77f0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009990 tx=0x7f77f4006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 msgr2=0x7f7804111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0094c0 tx=0x7f77ec009250 comp rx=0 tx=0).stop 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 shutdown_connections 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f77f00367e0 0x7f77f0038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 --2- 172.21.15.12:0/624670557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7804102730 0x7f7804111630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 >> 172.21.15.12:0/624670557 conn(0x7f78040fdf10 msgr2=0x7f78040feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 shutdown_connections 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:17.973+0000 7f780a875700 1 -- 172.21.15.12:0/624670557 wait complete. 2024-03-18T14:42:18.542 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying alertmanager service with default placement... 2024-03-18T14:42:19.228 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:18 smithi012 ceph-mon[29182]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:19.228 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:18 smithi012 ceph-mon[29182]: Saving service node-exporter spec with placement * 2024-03-18T14:42:19.228 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:18 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:19.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-18T14:42:19.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.014+0000 7f71e1b6d700 1 Processor -- start 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.014+0000 7f71e1b6d700 1 -- start start 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.014+0000 7f71e1b6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.014+0000 7f71e1b6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc103070 con 0x7f71dc102720 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38002/0 (socket says 172.21.15.12:38002) 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71db7fe700 1 -- 172.21.15.12:0/126214324 learned_addr learned my addr 172.21.15.12:0/126214324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71db7fe700 1 -- 172.21.15.12:0/126214324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc1031b0 con 0x7f71dc102720 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71db7fe700 1 --2- 172.21.15.12:0/126214324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0098e0 tx=0x7f71cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5947cdd58f891b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.015+0000 7f71da7fc700 1 -- 172.21.15.12:0/126214324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71cc00b860 con 0x7f71dc102720 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71da7fc700 1 -- 172.21.15.12:0/126214324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71cc00be90 con 0x7f71dc102720 2024-03-18T14:42:19.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71da7fc700 1 -- 172.21.15.12:0/126214324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71cc0103c0 con 0x7f71dc102720 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 -- 172.21.15.12:0/126214324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 msgr2=0x7f71dc102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/126214324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0098e0 tx=0x7f71cc009250 comp rx=0 tx=0).stop 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 -- 172.21.15.12:0/126214324 shutdown_connections 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/126214324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc102b30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 -- 172.21.15.12:0/126214324 >> 172.21.15.12:0/126214324 conn(0x7f71dc0fdee0 msgr2=0x7f71dc1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 -- 172.21.15.12:0/126214324 shutdown_connections 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.016+0000 7f71e1b6d700 1 -- 172.21.15.12:0/126214324 wait complete. 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71e1b6d700 1 Processor -- start 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71e1b6d700 1 -- start start 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71e1b6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71e1b6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc111ac0 con 0x7f71dc102720 2024-03-18T14:42:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38018/0 (socket says 172.21.15.12:38018) 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71db7fe700 1 -- 172.21.15.12:0/420999127 learned_addr learned my addr 172.21.15.12:0/420999127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.017+0000 7f71db7fe700 1 -- 172.21.15.12:0/420999127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0094f0 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71db7fe700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0090b0 tx=0x7f71cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71cc010840 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71cc010e70 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc111cc0 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71cc024e90 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.018+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc112830 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f71cc01a510 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc05f5f0 con 0x7f71dc102720 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71c3fff700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 0x7f71c4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:19.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f71cc04a9c0 con 0x7f71dc102720 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71daffd700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 0x7f71c4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.019+0000 7f71daffd700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 0x7f71c4038bf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009990 tx=0x7f71d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.024+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f71cc018710 con 0x7f71dc102720 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.226+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f71dc0ff190 con 0x7f71c4036740 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.232+0000 7f71c3fff700 1 -- 172.21.15.12:0/420999127 <== mgr.14120 v2:172.21.15.12:6800/3609733177 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f71dc0ff190 con 0x7f71c4036740 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 msgr2=0x7f71c4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 0x7f71c4038bf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009990 tx=0x7f71d0006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 msgr2=0x7f71dc111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0090b0 tx=0x7f71cc009250 comp rx=0 tx=0).stop 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 shutdown_connections 2024-03-18T14:42:19.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f71c4036740 0x7f71c4038bf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:19.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 --2- 172.21.15.12:0/420999127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71dc102720 0x7f71dc111580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:19.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 >> 172.21.15.12:0/420999127 conn(0x7f71dc0fdee0 msgr2=0x7f71dc0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:19.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 shutdown_connections 2024-03-18T14:42:19.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:19.234+0000 7f71e1b6d700 1 -- 172.21.15.12:0/420999127 wait complete. 2024-03-18T14:42:20.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:20 smithi012 ceph-mon[29182]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:20.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:20 smithi012 ceph-mon[29182]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:20.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:20 smithi012 ceph-mon[29182]: Saving service alertmanager spec with placement count:1 2024-03-18T14:42:20.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:20 smithi012 ceph-mon[29182]: from='mgr.14120 172.21.15.12:0/413473860' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:21.000 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448c831700 1 Processor -- start 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448c831700 1 -- start start 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448c831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448c831700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488105820 con 0x7f4488076be0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38030/0 (socket says 172.21.15.12:38030) 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448659c700 1 -- 172.21.15.12:0/579012155 learned_addr learned my addr 172.21.15.12:0/579012155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.200+0000 7f448659c700 1 -- 172.21.15.12:0/579012155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488105960 con 0x7f4488076be0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448659c700 1 --2- 172.21.15.12:0/579012155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44780098e0 tx=0x7f4478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d8e29704a81bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448559a700 1 -- 172.21.15.12:0/579012155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f447800b860 con 0x7f4488076be0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448559a700 1 -- 172.21.15.12:0/579012155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f447800be90 con 0x7f4488076be0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448559a700 1 -- 172.21.15.12:0/579012155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44780103c0 con 0x7f4488076be0 2024-03-18T14:42:21.001 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448c831700 1 -- 172.21.15.12:0/579012155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 msgr2=0x7f44881052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448c831700 1 --2- 172.21.15.12:0/579012155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44780098e0 tx=0x7f4478009250 comp rx=0 tx=0).stop 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448c831700 1 -- 172.21.15.12:0/579012155 shutdown_connections 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448c831700 1 --2- 172.21.15.12:0/579012155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f44881052e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.201+0000 7f448c831700 1 -- 172.21.15.12:0/579012155 >> 172.21.15.12:0/579012155 conn(0x7f44880fcd50 msgr2=0x7f44880ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 -- 172.21.15.12:0/579012155 shutdown_connections 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 -- 172.21.15.12:0/579012155 wait complete. 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 Processor -- start 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 -- start start 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448c831700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488103170 con 0x7f4488076be0 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38038/0 (socket says 172.21.15.12:38038) 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.202+0000 7f448659c700 1 -- 172.21.15.12:0/2529757636 learned_addr learned my addr 172.21.15.12:0/2529757636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:21.002 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f448659c700 1 -- 172.21.15.12:0/2529757636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780094f0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f448659c700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4478006d40 tx=0x7f4478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4478010840 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4478010e70 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44881011e0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4478024ea0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.203+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44881016d0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f44780109a0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448804bed0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f4476ffd700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 0x7f4470038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f447804a6c0 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f4485d9b700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 0x7f4470038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.204+0000 7f4485d9b700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 0x7f4470038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f447c009990 tx=0x7f447c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.209+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f447802c520 con 0x7f4488076be0 2024-03-18T14:42:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.403+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f44881066f0 con 0x7f4488076be0 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.406+0000 7f4476ffd700 1 -- 172.21.15.12:0/2529757636 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f447802c340 con 0x7f4488076be0 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 msgr2=0x7f4470038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 0x7f4470038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f447c009990 tx=0x7f447c006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 msgr2=0x7f4488102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4478006d40 tx=0x7f4478009250 comp rx=0 tx=0).stop 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 shutdown_connections 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f4470036790 0x7f4470038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 --2- 172.21.15.12:0/2529757636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4488076be0 0x7f4488102c30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 >> 172.21.15.12:0/2529757636 conn(0x7f44880fcd50 msgr2=0x7f44880fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 shutdown_connections 2024-03-18T14:42:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:20.408+0000 7f448c831700 1 -- 172.21.15.12:0/2529757636 wait complete. 2024-03-18T14:42:21.664 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:21 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2529757636' entity='client.admin' 2024-03-18T14:42:21.664 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:21 smithi012 ceph-mon[29182]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:22.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.463+0000 7f5d284e7700 1 Processor -- start 2024-03-18T14:42:22.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d284e7700 1 -- start start 2024-03-18T14:42:22.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d284e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:22.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d284e7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d201030c0 con 0x7f5d20102770 2024-03-18T14:42:22.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d26283700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d26283700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38044/0 (socket says 172.21.15.12:38044) 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d26283700 1 -- 172.21.15.12:0/697718729 learned_addr learned my addr 172.21.15.12:0/697718729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d26283700 1 -- 172.21.15.12:0/697718729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20103200 con 0x7f5d20102770 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.464+0000 7f5d26283700 1 --2- 172.21.15.12:0/697718729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0098e0 tx=0x7f5d1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c3d1a45ce328b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d25281700 1 -- 172.21.15.12:0/697718729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d1c00b860 con 0x7f5d20102770 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d25281700 1 -- 172.21.15.12:0/697718729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d1c00be90 con 0x7f5d20102770 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d25281700 1 -- 172.21.15.12:0/697718729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d1c0103c0 con 0x7f5d20102770 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 -- 172.21.15.12:0/697718729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 msgr2=0x7f5d20102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 --2- 172.21.15.12:0/697718729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0098e0 tx=0x7f5d1c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 -- 172.21.15.12:0/697718729 shutdown_connections 2024-03-18T14:42:22.234 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 --2- 172.21.15.12:0/697718729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d20102b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 -- 172.21.15.12:0/697718729 >> 172.21.15.12:0/697718729 conn(0x7f5d200fde90 msgr2=0x7f5d20100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 -- 172.21.15.12:0/697718729 shutdown_connections 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.465+0000 7f5d284e7700 1 -- 172.21.15.12:0/697718729 wait complete. 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d284e7700 1 Processor -- start 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d284e7700 1 -- start start 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d284e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d284e7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d20111b10 con 0x7f5d20102770 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d26283700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d26283700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38056/0 (socket says 172.21.15.12:38056) 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d26283700 1 -- 172.21.15.12:0/1489224606 learned_addr learned my addr 172.21.15.12:0/1489224606 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d26283700 1 -- 172.21.15.12:0/1489224606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0094f0 con 0x7f5d20102770 2024-03-18T14:42:22.235 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.466+0000 7f5d26283700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0090b0 tx=0x7f5d1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d1c010840 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d1c010e70 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d20111d10 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d1c024e40 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d20112880 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.467+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5d1c0109a0 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.468+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2005f5f0 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.468+0000 7f5d16ffd700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 0x7f5d0c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.468+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5d1c04a740 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.468+0000 7f5d25a82700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 0x7f5d0c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.468+0000 7f5d25a82700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 0x7f5d0c038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009990 tx=0x7f5d10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.472+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5d1c02c520 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.662+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f5d2004bed0 con 0x7f5d20102770 2024-03-18T14:42:22.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.668+0000 7f5d16ffd700 1 -- 172.21.15.12:0/1489224606 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f5d1c02c340 con 0x7f5d20102770 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.669+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 msgr2=0x7f5d0c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.669+0000 7f5d284e7700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 0x7f5d0c038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009990 tx=0x7f5d10006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 msgr2=0x7f5d201115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0090b0 tx=0x7f5d1c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 shutdown_connections 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f5d0c036790 0x7f5d0c038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 --2- 172.21.15.12:0/1489224606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d20102770 0x7f5d201115d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 >> 172.21.15.12:0/1489224606 conn(0x7f5d200fde90 msgr2=0x7f5d200feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 shutdown_connections 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:21.670+0000 7f5d284e7700 1 -- 172.21.15.12:0/1489224606 wait complete. 2024-03-18T14:42:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-03-18T14:42:22.897 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:22 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1489224606' entity='client.admin' 2024-03-18T14:42:24.131 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:23 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4014887619' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-18T14:42:24.131 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:23 smithi012 ceph-mon[29182]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f002e964700 1 Processor -- start 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f002e964700 1 -- start start 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f002e964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f002e964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028103090 con 0x7f0028102740 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f0027fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f0027fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38060/0 (socket says 172.21.15.12:38060) 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.641+0000 7f0027fff700 1 -- 172.21.15.12:0/1673059693 learned_addr learned my addr 172.21.15.12:0/1673059693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.642+0000 7f0027fff700 1 -- 172.21.15.12:0/1673059693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00281031d0 con 0x7f0028102740 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.642+0000 7f0027fff700 1 --2- 172.21.15.12:0/1673059693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00180098e0 tx=0x7f0018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fd9ac9f98742b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.642+0000 7f0026ffd700 1 -- 172.21.15.12:0/1673059693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f001800b860 con 0x7f0028102740 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.642+0000 7f0026ffd700 1 -- 172.21.15.12:0/1673059693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f001800be90 con 0x7f0028102740 2024-03-18T14:42:24.473 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.642+0000 7f0026ffd700 1 -- 172.21.15.12:0/1673059693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00180103c0 con 0x7f0028102740 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- 172.21.15.12:0/1673059693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 msgr2=0x7f0028102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 --2- 172.21.15.12:0/1673059693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00180098e0 tx=0x7f0018009250 comp rx=0 tx=0).stop 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- 172.21.15.12:0/1673059693 shutdown_connections 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 --2- 172.21.15.12:0/1673059693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028102b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- 172.21.15.12:0/1673059693 >> 172.21.15.12:0/1673059693 conn(0x7f00280fdea0 msgr2=0x7f00281002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- 172.21.15.12:0/1673059693 shutdown_connections 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- 172.21.15.12:0/1673059693 wait complete. 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 Processor -- start 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.643+0000 7f002e964700 1 -- start start 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f002e964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f002e964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028079110 con 0x7f0028102740 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0027fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0027fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38068/0 (socket says 172.21.15.12:38068) 2024-03-18T14:42:24.474 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0027fff700 1 -- 172.21.15.12:0/4014887619 learned_addr learned my addr 172.21.15.12:0/4014887619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0027fff700 1 -- 172.21.15.12:0/4014887619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180094f0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0027fff700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0018011c10 tx=0x7f0018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0018010890 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0028077180 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0018010ec0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00180243f0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.644+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028077670 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.645+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f001802d3f0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.645+0000 7f0024ff9700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 0x7f0010038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.645+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002804bed0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.645+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f001804a3f0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.646+0000 7f00277fe700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 0x7f0010038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.646+0000 7f00277fe700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 0x7f0010038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f001c009990 tx=0x7f001c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.650+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f001802b6b0 con 0x7f0028102740 2024-03-18T14:42:24.475 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:22.895+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0028104530 con 0x7f0028102740 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.797+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f00180193c0 con 0x7f0028102740 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.798+0000 7f0024ff9700 1 -- 172.21.15.12:0/4014887619 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f001802da60 con 0x7f0028102740 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 msgr2=0x7f0010038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 0x7f0010038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f001c009990 tx=0x7f001c006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 msgr2=0x7f0028078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0018011c10 tx=0x7f0018009250 comp rx=0 tx=0).stop 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 shutdown_connections 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7f0010036790 0x7f0010038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 --2- 172.21.15.12:0/4014887619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0028102740 0x7f0028078bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 >> 172.21.15.12:0/4014887619 conn(0x7f00280fdea0 msgr2=0x7f00280fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 shutdown_connections 2024-03-18T14:42:24.476 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:23.801+0000 7f002e964700 1 -- 172.21.15.12:0/4014887619 wait complete. 2024-03-18T14:42:25.081 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:24 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4014887619' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-18T14:42:25.081 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:24 smithi012 ceph-mon[29182]: mgrmap e9: smithi012.tmscgt(active, since 24s) 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "smithi012.tmscgt", 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.873+0000 7ff8da334700 1 Processor -- start 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.873+0000 7ff8da334700 1 -- start start 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8da334700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8da334700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41058d0 con 0x7ff8d4076b40 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38092/0 (socket says 172.21.15.12:38092) 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d3fff700 1 -- 172.21.15.12:0/747325713 learned_addr learned my addr 172.21.15.12:0/747325713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d3fff700 1 -- 172.21.15.12:0/747325713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4105a10 con 0x7ff8d4076b40 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d3fff700 1 --2- 172.21.15.12:0/747325713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098e0 tx=0x7ff8c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2638e644aa4847ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.874+0000 7ff8d2ffd700 1 -- 172.21.15.12:0/747325713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8c400b860 con 0x7ff8d4076b40 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8d2ffd700 1 -- 172.21.15.12:0/747325713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8c400be90 con 0x7ff8d4076b40 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8d2ffd700 1 -- 172.21.15.12:0/747325713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8c40103c0 con 0x7ff8d4076b40 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 -- 172.21.15.12:0/747325713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 msgr2=0x7ff8d4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 --2- 172.21.15.12:0/747325713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098e0 tx=0x7ff8c4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 -- 172.21.15.12:0/747325713 shutdown_connections 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 --2- 172.21.15.12:0/747325713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4105390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 -- 172.21.15.12:0/747325713 >> 172.21.15.12:0/747325713 conn(0x7ff8d40fcd80 msgr2=0x7ff8d40ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 -- 172.21.15.12:0/747325713 shutdown_connections 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.875+0000 7ff8da334700 1 -- 172.21.15.12:0/747325713 wait complete. 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8da334700 1 Processor -- start 2024-03-18T14:42:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8da334700 1 -- start start 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8da334700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8da334700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4103220 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38104/0 (socket says 172.21.15.12:38104) 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8d3fff700 1 -- 172.21.15.12:0/1366047745 learned_addr learned my addr 172.21.15.12:0/1366047745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.876+0000 7ff8d3fff700 1 -- 172.21.15.12:0/1366047745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40094f0 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8d3fff700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4006d40 tx=0x7ff8c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8c4010840 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8c4010e70 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d4101290 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8c4024ea0 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.877+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d4101780 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7ff8c401a4c0 con 0x7ff8d4076b40 2024-03-18T14:42:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d404bed0 con 0x7ff8d4076b40 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8d0ff9700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 0x7ff8bc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff8c4024780 con 0x7ff8d4076b40 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8d37fe700 1 -- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 msgr2=0x7ff8bc038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.878+0000 7ff8d37fe700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 0x7ff8bc038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:24.883+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7ff8c4018940 con 0x7ff8d4076b40 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.079+0000 7ff8d37fe700 1 -- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 msgr2=0x7ff8bc038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.079+0000 7ff8d37fe700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 0x7ff8bc038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.130+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff8d4101e20 con 0x7ff8d4076b40 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.130+0000 7ff8d0ff9700 1 -- 172.21.15.12:0/1366047745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7ff8c4018350 con 0x7ff8d4076b40 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 msgr2=0x7ff8bc038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 0x7ff8bc038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 msgr2=0x7ff8d4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4006d40 tx=0x7ff8c4009250 comp rx=0 tx=0).stop 2024-03-18T14:42:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 shutdown_connections 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7ff8bc0367e0 0x7ff8bc038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 --2- 172.21.15.12:0/1366047745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8d4076b40 0x7ff8d4102ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 >> 172.21.15.12:0/1366047745 conn(0x7ff8d40fcd80 msgr2=0x7ff8d40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 shutdown_connections 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:25.132+0000 7ff8da334700 1 -- 172.21.15.12:0/1366047745 wait complete. 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-03-18T14:42:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-03-18T14:42:25.972 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:25 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1366047745' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T14:42:30.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: Active manager daemon smithi012.tmscgt restarted 2024-03-18T14:42:30.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: osdmap e3: 0 total, 0 up, 0 in 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: mgrmap e10: smithi012.tmscgt(active, starting, since 0.00572062s) 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:30.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:30 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:42:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:31 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:42:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:31 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:42:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:31 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:31 smithi012 ceph-mon[29182]: mgrmap e11: smithi012.tmscgt(active, since 1.00804s) 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75fe38700 1 Processor -- start 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75fe38700 1 -- start start 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75fe38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:32.301 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75fe38700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758103080 con 0x7fc758102730 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75dbd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75dbd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38108/0 (socket says 172.21.15.12:38108) 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.196+0000 7fc75dbd4700 1 -- 172.21.15.12:0/1240159197 learned_addr learned my addr 172.21.15.12:0/1240159197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.197+0000 7fc75dbd4700 1 -- 172.21.15.12:0/1240159197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7581031c0 con 0x7fc758102730 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.197+0000 7fc75dbd4700 1 --2- 172.21.15.12:0/1240159197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7540098e0 tx=0x7fc754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8589d70aa2e4dd0e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.197+0000 7fc75cbd2700 1 -- 172.21.15.12:0/1240159197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc75400b860 con 0x7fc758102730 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.197+0000 7fc75cbd2700 1 -- 172.21.15.12:0/1240159197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc75400be90 con 0x7fc758102730 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.197+0000 7fc75cbd2700 1 -- 172.21.15.12:0/1240159197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7540103c0 con 0x7fc758102730 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 -- 172.21.15.12:0/1240159197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 msgr2=0x7fc758102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 --2- 172.21.15.12:0/1240159197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7540098e0 tx=0x7fc754009250 comp rx=0 tx=0).stop 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 -- 172.21.15.12:0/1240159197 shutdown_connections 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 --2- 172.21.15.12:0/1240159197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758102b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 -- 172.21.15.12:0/1240159197 >> 172.21.15.12:0/1240159197 conn(0x7fc7580fdf10 msgr2=0x7fc758100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 -- 172.21.15.12:0/1240159197 shutdown_connections 2024-03-18T14:42:32.302 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.198+0000 7fc75fe38700 1 -- 172.21.15.12:0/1240159197 wait complete. 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75fe38700 1 Processor -- start 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75fe38700 1 -- start start 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75fe38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75fe38700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758111b70 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75dbd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75dbd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38110/0 (socket says 172.21.15.12:38110) 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75dbd4700 1 -- 172.21.15.12:0/2167480520 learned_addr learned my addr 172.21.15.12:0/2167480520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75dbd4700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540094f0 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.199+0000 7fc75dbd4700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7540094c0 tx=0x7fc754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.200+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc754010680 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.200+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc758111d70 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.200+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc754010cb0 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.200+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc75401ad40 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.200+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581128e0 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fc754025440 con 0x7fc758102730 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc74e7fc700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7440393a0 con 0x7fc7440367e0 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc75404a370 con 0x7fc758102730 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc75d3d3700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.201+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.402+0000 7fc75d3d3700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.402+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.803+0000 7fc75d3d3700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:26.803+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:27.604+0000 7fc75d3d3700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:27.604+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:29.205+0000 7fc75d3d3700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3609733177 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:29.205+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:30.570+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7fc7540257e0 con 0x7fc758102730 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:30.570+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 msgr2=0x7fc744038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:30.570+0000 7fc74e7fc700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.304 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.572+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fc754049900 con 0x7fc758102730 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.572+0000 7fc74e7fc700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 0x7fc74403c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.572+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7440393a0 con 0x7fc744039fd0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.572+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 0x7fc74403c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.573+0000 7fc75d3d3700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 0x7fc74403c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7480039d0 tx=0x7fc7480092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.573+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fc7440393a0 con 0x7fc744039fd0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.579+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc758112200 con 0x7fc744039fd0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc74e7fc700 1 -- 172.21.15.12:0/2167480520 <== mgr.14162 v2:172.21.15.12:6800/1147707934 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc758112200 con 0x7fc744039fd0 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 msgr2=0x7fc74403c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 0x7fc74403c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7480039d0 tx=0x7fc7480092f0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 msgr2=0x7fc758111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7540094c0 tx=0x7fc754009250 comp rx=0 tx=0).stop 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 shutdown_connections 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc744039fd0 0x7fc74403c3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.305 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc758102730 0x7fc758111630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 --2- 172.21.15.12:0/2167480520 >> [v2:172.21.15.12:6800/3609733177,v1:172.21.15.12:6801/3609733177] conn(0x7fc7440367e0 0x7fc744038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 >> 172.21.15.12:0/2167480520 conn(0x7fc7580fdf10 msgr2=0x7fc7580feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 shutdown_connections 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:31.580+0000 7fc75fe38700 1 -- 172.21.15.12:0/2167480520 wait complete. 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-03-18T14:42:32.306 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-03-18T14:42:32.936 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:30] ENGINE Bus STARTING 2024-03-18T14:42:32.936 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:31] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:42:32.936 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: [18/Mar/2024:14:42:31] ENGINE Bus STARTED 2024-03-18T14:42:32.936 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T14:42:32.936 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T14:42:32.937 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:32 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.738+0000 7fc05335f700 1 Processor -- start 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc05335f700 1 -- start start 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc05335f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc05335f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c101fb0 con 0x7fc04c101660 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc0510fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc0510fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55544/0 (socket says 172.21.15.12:55544) 2024-03-18T14:42:33.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc0510fb700 1 -- 172.21.15.12:0/3009539998 learned_addr learned my addr 172.21.15.12:0/3009539998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.739+0000 7fc0510fb700 1 -- 172.21.15.12:0/3009539998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c1020f0 con 0x7fc04c101660 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc0510fb700 1 --2- 172.21.15.12:0/3009539998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0480098e0 tx=0x7fc048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc4ccc39bad95ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc043fff700 1 -- 172.21.15.12:0/3009539998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04800b860 con 0x7fc04c101660 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc043fff700 1 -- 172.21.15.12:0/3009539998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc04800be90 con 0x7fc04c101660 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc043fff700 1 -- 172.21.15.12:0/3009539998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0480103c0 con 0x7fc04c101660 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 -- 172.21.15.12:0/3009539998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 msgr2=0x7fc04c101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 --2- 172.21.15.12:0/3009539998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0480098e0 tx=0x7fc048009250 comp rx=0 tx=0).stop 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 -- 172.21.15.12:0/3009539998 shutdown_connections 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 --2- 172.21.15.12:0/3009539998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c101a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 -- 172.21.15.12:0/3009539998 >> 172.21.15.12:0/3009539998 conn(0x7fc04c0fcde0 msgr2=0x7fc04c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 -- 172.21.15.12:0/3009539998 shutdown_connections 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.740+0000 7fc05335f700 1 -- 172.21.15.12:0/3009539998 wait complete. 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc05335f700 1 Processor -- start 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc05335f700 1 -- start start 2024-03-18T14:42:33.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc05335f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc05335f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c1142a0 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc0510fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc0510fb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55554/0 (socket says 172.21.15.12:55554) 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc0510fb700 1 -- 172.21.15.12:0/1102839849 learned_addr learned my addr 172.21.15.12:0/1102839849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.741+0000 7fc0510fb700 1 -- 172.21.15.12:0/1102839849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480094f0 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc0510fb700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc0480090b0 tx=0x7fc048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc048010840 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc048010e70 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc048024e40 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c1143e0 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.742+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c114870 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.743+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fc048024430 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.743+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c04bed0 con 0x7fc04c101660 2024-03-18T14:42:33.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.743+0000 7fc041ffb700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 0x7fc038038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.743+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc04804a6c0 con 0x7fc04c101660 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.743+0000 7fc0508fa700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 0x7fc038038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.744+0000 7fc0508fa700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 0x7fc038038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009990 tx=0x7fc03c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.748+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc04802c930 con 0x7fc04c101660 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.934+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fc04802e440 con 0x7fc04c101660 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.948+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc04c0fe040 con 0x7fc0380366b0 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.997+0000 7fc041ffb700 1 -- 172.21.15.12:0/1102839849 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc04c0fe040 con 0x7fc0380366b0 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 msgr2=0x7fc038038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 0x7fc038038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009990 tx=0x7fc03c006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 msgr2=0x7fc04c076ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc0480090b0 tx=0x7fc048009250 comp rx=0 tx=0).stop 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 shutdown_connections 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc0380366b0 0x7fc038038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 --2- 172.21.15.12:0/1102839849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc04c101660 0x7fc04c076ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 >> 172.21.15.12:0/1102839849 conn(0x7fc04c0fcde0 msgr2=0x7fc04c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:33.600 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 shutdown_connections 2024-03-18T14:42:33.601 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:32.999+0000 7fc05335f700 1 -- 172.21.15.12:0/1102839849 wait complete. 2024-03-18T14:42:33.601 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-03-18T14:42:33.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:33 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:33.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:33 smithi012 ceph-mon[29182]: mgrmap e12: smithi012.tmscgt(active, since 2s) 2024-03-18T14:42:33.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:33 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:33.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:33 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:35.082 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:34 smithi012 ceph-mon[29182]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:35.082 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:34 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:35.089 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Fnj..NenKvzNi0pa8eyQmusjvsOz2Mh9ckyajZKV8OyXBGKyjan56", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710772954, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f737698f700 1 Processor -- start 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f737698f700 1 -- start start 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f737698f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f737698f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370103080 con 0x7f7370102730 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f736ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f736ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55560/0 (socket says 172.21.15.12:55560) 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f736ffff700 1 -- 172.21.15.12:0/96844836 learned_addr learned my addr 172.21.15.12:0/96844836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f736ffff700 1 -- 172.21.15.12:0/96844836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73701031c0 con 0x7f7370102730 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.037+0000 7f736ffff700 1 --2- 172.21.15.12:0/96844836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73600098e0 tx=0x7f7360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571c69e63868f015 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f736effd700 1 -- 172.21.15.12:0/96844836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736000b860 con 0x7f7370102730 2024-03-18T14:42:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f736effd700 1 -- 172.21.15.12:0/96844836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f736000be90 con 0x7f7370102730 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f736effd700 1 -- 172.21.15.12:0/96844836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73600103c0 con 0x7f7370102730 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 -- 172.21.15.12:0/96844836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 msgr2=0x7f7370102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 --2- 172.21.15.12:0/96844836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73600098e0 tx=0x7f7360009250 comp rx=0 tx=0).stop 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 -- 172.21.15.12:0/96844836 shutdown_connections 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 --2- 172.21.15.12:0/96844836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370102b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 -- 172.21.15.12:0/96844836 >> 172.21.15.12:0/96844836 conn(0x7f73700fdf10 msgr2=0x7f7370100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 -- 172.21.15.12:0/96844836 shutdown_connections 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.038+0000 7f737698f700 1 -- 172.21.15.12:0/96844836 wait complete. 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f737698f700 1 Processor -- start 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f737698f700 1 -- start start 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f737698f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f737698f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370111b50 con 0x7f7370102730 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f736ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:35.091 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f736ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55568/0 (socket says 172.21.15.12:55568) 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f736ffff700 1 -- 172.21.15.12:0/4145138751 learned_addr learned my addr 172.21.15.12:0/4145138751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.039+0000 7f736ffff700 1 -- 172.21.15.12:0/4145138751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600094f0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f736ffff700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f73600094c0 tx=0x7f7360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7360010680 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7360010cb0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736001ad40 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7370111d50 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.040+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701128c0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f73600107e0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737005f5f0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f736cff9700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 0x7f7358038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f736004a3b0 con 0x7f7370102730 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f736f7fe700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 0x7f7358038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.041+0000 7f736f7fe700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 0x7f7358038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7364009990 tx=0x7f7364006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.046+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f73600188a0 con 0x7f7370102730 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.261+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f737005f410 con 0x7f7358036830 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.507+0000 7f736cff9700 1 -- 172.21.15.12:0/4145138751 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f737005f410 con 0x7f7358036830 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 msgr2=0x7f7358038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 0x7f7358038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7364009990 tx=0x7f7364006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 msgr2=0x7f7370111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f73600094c0 tx=0x7f7360009250 comp rx=0 tx=0).stop 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 shutdown_connections 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7358036830 0x7f7358038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 --2- 172.21.15.12:0/4145138751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370102730 0x7f7370111610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.509+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 >> 172.21.15.12:0/4145138751 conn(0x7f73700fdf10 msgr2=0x7f73700feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.510+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 shutdown_connections 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:34.510+0000 7f737698f700 1 -- 172.21.15.12:0/4145138751 wait complete. 2024-03-18T14:42:35.093 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-03-18T14:42:36.122 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:35 smithi012 ceph-mon[29182]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:36.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:36 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2176926630' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-18T14:42:37.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.219+0000 7f0a91964700 1 Processor -- start 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.219+0000 7f0a91964700 1 -- start start 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.219+0000 7f0a91964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.219+0000 7f0a91964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c103090 con 0x7f0a8c102740 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55584/0 (socket says 172.21.15.12:55584) 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a8affd700 1 -- 172.21.15.12:0/2807263709 learned_addr learned my addr 172.21.15.12:0/2807263709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a8affd700 1 -- 172.21.15.12:0/2807263709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c1031d0 con 0x7f0a8c102740 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a8affd700 1 --2- 172.21.15.12:0/2807263709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009e50 tx=0x7f0a84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e57068c8e42b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a89ffb700 1 -- 172.21.15.12:0/2807263709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a840092e0 con 0x7f0a8c102740 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a89ffb700 1 -- 172.21.15.12:0/2807263709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a84012430 con 0x7f0a8c102740 2024-03-18T14:42:37.042 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.220+0000 7f0a89ffb700 1 -- 172.21.15.12:0/2807263709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a840075a0 con 0x7f0a8c102740 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 -- 172.21.15.12:0/2807263709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 msgr2=0x7f0a8c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 --2- 172.21.15.12:0/2807263709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009e50 tx=0x7f0a84009840 comp rx=0 tx=0).stop 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 -- 172.21.15.12:0/2807263709 shutdown_connections 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 --2- 172.21.15.12:0/2807263709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c102b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 -- 172.21.15.12:0/2807263709 >> 172.21.15.12:0/2807263709 conn(0x7f0a8c0fdea0 msgr2=0x7f0a8c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 -- 172.21.15.12:0/2807263709 shutdown_connections 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 -- 172.21.15.12:0/2807263709 wait complete. 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.221+0000 7f0a91964700 1 Processor -- start 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a91964700 1 -- start start 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a91964700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a91964700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c079110 con 0x7f0a8c102740 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a8affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55586/0 (socket says 172.21.15.12:55586) 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a8affd700 1 -- 172.21.15.12:0/2176926630 learned_addr learned my addr 172.21.15.12:0/2176926630 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:37.043 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a8affd700 1 -- 172.21.15.12:0/2176926630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009ae0 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a8affd700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a84012de0 tx=0x7f0a84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a840092e0 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a84007d30 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c077180 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a84025880 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.222+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c077670 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.223+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f0a84019050 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.223+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c04bed0 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.223+0000 7f0a837fe700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 0x7f0a78038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.223+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0a8404a320 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.224+0000 7f0a8a7fc700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 0x7f0a78038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.224+0000 7f0a8a7fc700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 0x7f0a78038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009990 tx=0x7f0a74006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.228+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0a8402d9a0 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.420+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f0a8c104530 con 0x7f0a8c102740 2024-03-18T14:42:37.044 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.420+0000 7f0a837fe700 1 -- 172.21.15.12:0/2176926630 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f0a84017070 con 0x7f0a8c102740 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 msgr2=0x7f0a78038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 0x7f0a78038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009990 tx=0x7f0a74006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 msgr2=0x7f0a8c078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a84012de0 tx=0x7f0a84009840 comp rx=0 tx=0).stop 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 shutdown_connections 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f0a78036830 0x7f0a78038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 --2- 172.21.15.12:0/2176926630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c102740 0x7f0a8c078bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 >> 172.21.15.12:0/2176926630 conn(0x7f0a8c0fdea0 msgr2=0x7f0a8c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 shutdown_connections 2024-03-18T14:42:37.045 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:36.423+0000 7f0a91964700 1 -- 172.21.15.12:0/2176926630 wait complete. 2024-03-18T14:42:37.050 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T14:42:37.050 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-03-18T14:42:37.057 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T14:42:37.057 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-03-18T14:42:37.057 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-03-18T14:42:37.057 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-18T14:42:37.059 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-03-18T14:42:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: Password: doma4mvp1h 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config directory 2024-03-18T14:42:37.060 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-03-18T14:42:37.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:37 smithi012 ceph-mon[29182]: mgrmap e13: smithi012.tmscgt(active, since 6s) 2024-03-18T14:42:38.223 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.517+0000 7f2a423dc700 1 Processor -- start 2024-03-18T14:42:38.223 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.517+0000 7f2a423dc700 1 -- start start 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.517+0000 7f2a423dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.517+0000 7f2a423dc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c103500 con 0x7f2a3c102bb0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55600/0 (socket says 172.21.15.12:55600) 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3bfff700 1 -- 172.21.15.12:0/3882511009 learned_addr learned my addr 172.21.15.12:0/3882511009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3bfff700 1 -- 172.21.15.12:0/3882511009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c103640 con 0x7f2a3c102bb0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3bfff700 1 --2- 172.21.15.12:0/3882511009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098e0 tx=0x7f2a2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6926d549445fb634 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3affd700 1 -- 172.21.15.12:0/3882511009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a2c00b860 con 0x7f2a3c102bb0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3affd700 1 -- 172.21.15.12:0/3882511009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a2c00be90 con 0x7f2a3c102bb0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.518+0000 7f2a3affd700 1 -- 172.21.15.12:0/3882511009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a2c0103c0 con 0x7f2a3c102bb0 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 -- 172.21.15.12:0/3882511009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 msgr2=0x7f2a3c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3882511009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098e0 tx=0x7f2a2c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 -- 172.21.15.12:0/3882511009 shutdown_connections 2024-03-18T14:42:38.224 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3882511009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c102fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 -- 172.21.15.12:0/3882511009 >> 172.21.15.12:0/3882511009 conn(0x7f2a3c0fc4d0 msgr2=0x7f2a3c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 -- 172.21.15.12:0/3882511009 shutdown_connections 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 -- 172.21.15.12:0/3882511009 wait complete. 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.519+0000 7f2a423dc700 1 Processor -- start 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a423dc700 1 -- start start 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a423dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a423dc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c114250 con 0x7f2a3c102bb0 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55610/0 (socket says 172.21.15.12:55610) 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a3bfff700 1 -- 172.21.15.12:0/3418732324 learned_addr learned my addr 172.21.15.12:0/3418732324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a3bfff700 1 -- 172.21.15.12:0/3418732324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0094f0 con 0x7f2a3c102bb0 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a3bfff700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0090b0 tx=0x7f2a2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a2c010840 con 0x7f2a3c102bb0 2024-03-18T14:42:38.225 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a2c010e70 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.520+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c114390 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.521+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a2c024e40 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.521+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c114880 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.521+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f2a2c01a4c0 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.521+0000 7f2a38ff9700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 0x7f2a24038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.521+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c04bed0 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.522+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2a2c04a980 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.522+0000 7f2a3b7fe700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 0x7f2a24038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.522+0000 7f2a3b7fe700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 0x7f2a24038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009990 tx=0x7f2a30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.526+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f2a2c018940 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.718+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2a3c102440 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.718+0000 7f2a38ff9700 1 -- 172.21.15.12:0/3418732324 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2a2c00fba0 con 0x7f2a3c102bb0 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 msgr2=0x7f2a24038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:38.226 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 0x7f2a24038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009990 tx=0x7f2a30006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 msgr2=0x7f2a3c1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0090b0 tx=0x7f2a2c009250 comp rx=0 tx=0).stop 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 shutdown_connections 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f2a24036880 0x7f2a24038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 --2- 172.21.15.12:0/3418732324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a3c102bb0 0x7f2a3c1013c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 >> 172.21.15.12:0/3418732324 conn(0x7f2a3c0fc4d0 msgr2=0x7f2a3c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 shutdown_connections 2024-03-18T14:42:38.227 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:37.720+0000 7f2a423dc700 1 -- 172.21.15.12:0/3418732324 wait complete. 2024-03-18T14:42:39.547 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.626+0000 7fd564d17700 1 Processor -- start 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.626+0000 7fd564d17700 1 -- start start 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.626+0000 7fd564d17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.626+0000 7fd564d17700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560103090 con 0x7fd560102740 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.626+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53402/0 (socket says 172.21.15.12:53402) 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55e59c700 1 -- 172.21.15.12:0/3062700586 learned_addr learned my addr 172.21.15.12:0/3062700586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55e59c700 1 -- 172.21.15.12:0/3062700586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5601031d0 con 0x7fd560102740 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55e59c700 1 --2- 172.21.15.12:0/3062700586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e50 tx=0x7fd558009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f970725ea51eba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55d59a700 1 -- 172.21.15.12:0/3062700586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5580092e0 con 0x7fd560102740 2024-03-18T14:42:39.548 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55d59a700 1 -- 172.21.15.12:0/3062700586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd558012430 con 0x7fd560102740 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.627+0000 7fd55d59a700 1 -- 172.21.15.12:0/3062700586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5580075a0 con 0x7fd560102740 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- 172.21.15.12:0/3062700586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 msgr2=0x7fd560102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 --2- 172.21.15.12:0/3062700586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e50 tx=0x7fd558009840 comp rx=0 tx=0).stop 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- 172.21.15.12:0/3062700586 shutdown_connections 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 --2- 172.21.15.12:0/3062700586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560102b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- 172.21.15.12:0/3062700586 >> 172.21.15.12:0/3062700586 conn(0x7fd5600fdea0 msgr2=0x7fd5601002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- 172.21.15.12:0/3062700586 shutdown_connections 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- 172.21.15.12:0/3062700586 wait complete. 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 Processor -- start 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.628+0000 7fd564d17700 1 -- start start 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd564d17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd564d17700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560111ba0 con 0x7fd560102740 2024-03-18T14:42:39.549 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53404/0 (socket says 172.21.15.12:53404) 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd55e59c700 1 -- 172.21.15.12:0/1106799876 learned_addr learned my addr 172.21.15.12:0/1106799876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd55e59c700 1 -- 172.21.15.12:0/1106799876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558009ae0 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd55e59c700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd558012de0 tx=0x7fd558009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5580092e0 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd558007ce0 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd560111da0 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd558025930 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.629+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd560112910 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.630+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd558019050 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.630+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56005f5f0 con 0x7fd560102740 2024-03-18T14:42:39.550 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.630+0000 7fd556ffd700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 0x7fd54c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.630+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd55804a340 con 0x7fd560102740 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.630+0000 7fd55dd9b700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 0x7fd54c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.631+0000 7fd55dd9b700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 0x7fd54c038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd548009990 tx=0x7fd548006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.635+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fd55802d530 con 0x7fd560102740 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.938+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd56004bed0 con 0x7fd560102740 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.941+0000 7fd556ffd700 1 -- 172.21.15.12:0/1106799876 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd558017070 con 0x7fd560102740 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 msgr2=0x7fd54c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 0x7fd54c038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd548009990 tx=0x7fd548006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 msgr2=0x7fd560111660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd558012de0 tx=0x7fd558009840 comp rx=0 tx=0).stop 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 shutdown_connections 2024-03-18T14:42:39.551 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd54c0368d0 0x7fd54c038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 --2- 172.21.15.12:0/1106799876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd560102740 0x7fd560111660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 >> 172.21.15.12:0/1106799876 conn(0x7fd5600fdea0 msgr2=0x7fd5600feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 shutdown_connections 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-03-18T14:42:38.943+0000 7fd564d17700 1 -- 172.21.15.12:0/1106799876 wait complete. 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-18T14:42:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:39.553 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-03-18T14:42:39.588 INFO:tasks.cephadm:Fetching config... 2024-03-18T14:42:39.588 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:42:39.588 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-18T14:42:39.643 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-18T14:42:39.644 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:42:39.644 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-18T14:42:39.700 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-18T14:42:39.700 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:42:39.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/keyring of=/dev/stdout 2024-03-18T14:42:39.766 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-18T14:42:39.766 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:42:39.766 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-18T14:42:39.821 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-18T14:42:39.821 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4cw4yrZPdCMk2dyJahmh5HTnjntjZTrVE9j2kvxicSIGI85+NMPaOI9YCjYt5jInPhl9fPxHp7u30qPJqOgPhjOxEwdcaFEerTda1+X3j3DA+VNYCAshVwfvU56LRKJdY2LWAuK1tSJZkGxqQb+LAZZSmK865eA2CGYQ7MlzHltTgMoYL9t/zmv51/w/nw0Fc/E1dkUHoIMjsQdjGUEaKhCNhTKp/qoFhT7ogkr4saJEIj7ZwRKuiuIquqQOU3FYHszwu/x/I1AmLycH5IDUiIzBxZS9y03IbfuR36aHetQ/MVyjf8fEwqq8Kv3GoGMmN3NYX4m+s9TsQ9ss3qrYX7nE8z4E8K9OZrAVo1V75YCu72+qa+ypa06Sw3buwEUQo4we0T+p7UawEntIMKe9bZu8UkaLjlSrljBKscYl5Eqdy3W0TKa12j71h1oWb9VOomGJoCjuYsu24IHhaAFrCN1isWVpLZ5nMSNH8stNDiRLu6G0T8VaoeR1FFYErbL8= ceph-74f0a062-e535-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T14:42:39.900 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4cw4yrZPdCMk2dyJahmh5HTnjntjZTrVE9j2kvxicSIGI85+NMPaOI9YCjYt5jInPhl9fPxHp7u30qPJqOgPhjOxEwdcaFEerTda1+X3j3DA+VNYCAshVwfvU56LRKJdY2LWAuK1tSJZkGxqQb+LAZZSmK865eA2CGYQ7MlzHltTgMoYL9t/zmv51/w/nw0Fc/E1dkUHoIMjsQdjGUEaKhCNhTKp/qoFhT7ogkr4saJEIj7ZwRKuiuIquqQOU3FYHszwu/x/I1AmLycH5IDUiIzBxZS9y03IbfuR36aHetQ/MVyjf8fEwqq8Kv3GoGMmN3NYX4m+s9TsQ9ss3qrYX7nE8z4E8K9OZrAVo1V75YCu72+qa+ypa06Sw3buwEUQo4we0T+p7UawEntIMKe9bZu8UkaLjlSrljBKscYl5Eqdy3W0TKa12j71h1oWb9VOomGJoCjuYsu24IHhaAFrCN1isWVpLZ5nMSNH8stNDiRLu6G0T8VaoeR1FFYErbL8= ceph-74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:42:39.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4cw4yrZPdCMk2dyJahmh5HTnjntjZTrVE9j2kvxicSIGI85+NMPaOI9YCjYt5jInPhl9fPxHp7u30qPJqOgPhjOxEwdcaFEerTda1+X3j3DA+VNYCAshVwfvU56LRKJdY2LWAuK1tSJZkGxqQb+LAZZSmK865eA2CGYQ7MlzHltTgMoYL9t/zmv51/w/nw0Fc/E1dkUHoIMjsQdjGUEaKhCNhTKp/qoFhT7ogkr4saJEIj7ZwRKuiuIquqQOU3FYHszwu/x/I1AmLycH5IDUiIzBxZS9y03IbfuR36aHetQ/MVyjf8fEwqq8Kv3GoGMmN3NYX4m+s9TsQ9ss3qrYX7nE8z4E8K9OZrAVo1V75YCu72+qa+ypa06Sw3buwEUQo4we0T+p7UawEntIMKe9bZu8UkaLjlSrljBKscYl5Eqdy3W0TKa12j71h1oWb9VOomGJoCjuYsu24IHhaAFrCN1isWVpLZ5nMSNH8stNDiRLu6G0T8VaoeR1FFYErbL8= ceph-74f0a062-e535-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T14:42:39.959 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4cw4yrZPdCMk2dyJahmh5HTnjntjZTrVE9j2kvxicSIGI85+NMPaOI9YCjYt5jInPhl9fPxHp7u30qPJqOgPhjOxEwdcaFEerTda1+X3j3DA+VNYCAshVwfvU56LRKJdY2LWAuK1tSJZkGxqQb+LAZZSmK865eA2CGYQ7MlzHltTgMoYL9t/zmv51/w/nw0Fc/E1dkUHoIMjsQdjGUEaKhCNhTKp/qoFhT7ogkr4saJEIj7ZwRKuiuIquqQOU3FYHszwu/x/I1AmLycH5IDUiIzBxZS9y03IbfuR36aHetQ/MVyjf8fEwqq8Kv3GoGMmN3NYX4m+s9TsQ9ss3qrYX7nE8z4E8K9OZrAVo1V75YCu72+qa+ypa06Sw3buwEUQo4we0T+p7UawEntIMKe9bZu8UkaLjlSrljBKscYl5Eqdy3W0TKa12j71h1oWb9VOomGJoCjuYsu24IHhaAFrCN1isWVpLZ5nMSNH8stNDiRLu6G0T8VaoeR1FFYErbL8= ceph-74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:42:39.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-18T14:42:40.230 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:42:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:39 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1106799876' entity='client.admin' 2024-03-18T14:42:42.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- 172.21.15.12:0/2880049969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 msgr2=0x7f98ec0fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 --2- 172.21.15.12:0/2880049969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec0fce50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0048c0 tx=0x7f98dc009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- 172.21.15.12:0/2880049969 shutdown_connections 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 --2- 172.21.15.12:0/2880049969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec0fce50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- 172.21.15.12:0/2880049969 >> 172.21.15.12:0/2880049969 conn(0x7f98ec0f81c0 msgr2=0x7f98ec0fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- 172.21.15.12:0/2880049969 shutdown_connections 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- 172.21.15.12:0/2880049969 wait complete. 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 Processor -- start 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.886+0000 7f98f284e700 1 -- start start 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98f284e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98f284e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0722a0 con 0x7f98ec0fca40 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98ebfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53424/0 (socket says 172.21.15.12:53424) 2024-03-18T14:42:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98ebfff700 1 -- 172.21.15.12:0/1854916800 learned_addr learned my addr 172.21.15.12:0/1854916800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98ebfff700 1 -- 172.21.15.12:0/1854916800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc005040 con 0x7f98ec0fca40 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98ebfff700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0096b0 tx=0x7f98dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98dc00bed0 con 0x7f98ec0fca40 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98dc00b5c0 con 0x7f98ec0fca40 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec070370 con 0x7f98ec0fca40 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98dc01c5b0 con 0x7f98ec0fca40 2024-03-18T14:42:42.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.887+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec070860 con 0x7f98ec0fca40 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.888+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f98dc01c820 con 0x7f98ec0fca40 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.888+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec05f5f0 con 0x7f98ec0fca40 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.889+0000 7f98e8ff9700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 0x7f98d4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.889+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f98dc049a70 con 0x7f98ec0fca40 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.889+0000 7f98eb7fe700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 0x7f98d4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:42.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.889+0000 7f98eb7fe700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 0x7f98d4038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009990 tx=0x7f98e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:42.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:42.896+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f98dc023300 con 0x7f98ec0fca40 2024-03-18T14:42:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.091+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f98ec04bed0 con 0x7f98ec0fca40 2024-03-18T14:42:43.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.096+0000 7f98e8ff9700 1 -- 172.21.15.12:0/1854916800 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f98dc011b60 con 0x7f98ec0fca40 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 msgr2=0x7f98d4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 0x7f98d4038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009990 tx=0x7f98e0006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 msgr2=0x7f98ec071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0096b0 tx=0x7f98dc009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 shutdown_connections 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f98d4036880 0x7f98d4038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 --2- 172.21.15.12:0/1854916800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98ec0fca40 0x7f98ec071d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 >> 172.21.15.12:0/1854916800 conn(0x7f98ec0f81c0 msgr2=0x7f98ec103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:43.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 shutdown_connections 2024-03-18T14:42:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:43.099+0000 7f98f284e700 1 -- 172.21.15.12:0/1854916800 wait complete. 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1854916800' entity='client.admin' 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:42:44.192 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-18T14:42:44.193 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:42:44.193 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:44 smithi012 ceph-mon[29182]: Deploying daemon crash.smithi012 on smithi012 2024-03-18T14:42:44.214 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-18T14:42:44.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-18T14:42:45.432 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- 172.21.15.12:0/2262859853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 msgr2=0x7fc6140fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/2262859853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc6140fd450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6100048c0 tx=0x7fc610009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- 172.21.15.12:0/2262859853 shutdown_connections 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/2262859853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc6140fd450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- 172.21.15.12:0/2262859853 >> 172.21.15.12:0/2262859853 conn(0x7fc6140f70c0 msgr2=0x7fc6140f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- 172.21.15.12:0/2262859853 shutdown_connections 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- 172.21.15.12:0/2262859853 wait complete. 2024-03-18T14:42:47.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 Processor -- start 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.188+0000 7fc61b5ce700 1 -- start start 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61b5ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61b5ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61410d5c0 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61936a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61936a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53438/0 (socket says 172.21.15.12:53438) 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61936a700 1 -- 172.21.15.12:0/1445852932 learned_addr learned my addr 172.21.15.12:0/1445852932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61936a700 1 -- 172.21.15.12:0/1445852932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610005040 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61936a700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6100096b0 tx=0x7fc610009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc61000bed0 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61410d7c0 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc61000b5c0 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.189+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc61001c5b0 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.190+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61410e330 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.190+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fc61001c710 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.190+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc614101e60 con 0x7fc6140fd040 2024-03-18T14:42:47.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.191+0000 7fc609ffb700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 0x7fc600038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.191+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc610049ab0 con 0x7fc6140fd040 2024-03-18T14:42:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.191+0000 7fc618b69700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 0x7fc600038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.191+0000 7fc618b69700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 0x7fc600038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc604009990 tx=0x7fc604006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.195+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc610025400 con 0x7fc6140fd040 2024-03-18T14:42:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.400+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc614066010 con 0x7fc6000368d0 2024-03-18T14:42:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.403+0000 7fc609ffb700 1 -- 172.21.15.12:0/1445852932 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc614066010 con 0x7fc6000368d0 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.405+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 msgr2=0x7fc600038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 0x7fc600038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc604009990 tx=0x7fc604006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 msgr2=0x7fc61410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6100096b0 tx=0x7fc610009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 shutdown_connections 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fc6000368d0 0x7fc600038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 --2- 172.21.15.12:0/1445852932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6140fd040 0x7fc61410d080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 >> 172.21.15.12:0/1445852932 conn(0x7fc6140f70c0 msgr2=0x7fc614065900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 shutdown_connections 2024-03-18T14:42:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:47.406+0000 7fc61b5ce700 1 -- 172.21.15.12:0/1445852932 wait complete. 2024-03-18T14:42:48.013 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-03-18T14:42:48.014 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:42:48.014 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-03-18T14:42:48.032 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:42:48.032 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:42:48.090 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-03-18T14:42:48.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch host add smithi100 2024-03-18T14:42:48.473 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: Deploying daemon node-exporter.smithi012 on smithi012 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:48.596 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:48 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 -- 172.21.15.12:0/1492397280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 msgr2=0x7f72a40ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 --2- 172.21.15.12:0/1492397280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a40ff650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f72940048c0 tx=0x7f7294009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 -- 172.21.15.12:0/1492397280 shutdown_connections 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 --2- 172.21.15.12:0/1492397280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a40ff650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 -- 172.21.15.12:0/1492397280 >> 172.21.15.12:0/1492397280 conn(0x7f72a40f70c0 msgr2=0x7f72a40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 -- 172.21.15.12:0/1492397280 shutdown_connections 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 -- 172.21.15.12:0/1492397280 wait complete. 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.024+0000 7f72aa056700 1 Processor -- start 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72aa056700 1 -- start start 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72aa056700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72aa056700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a410d620 con 0x7f72a40669e0 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72a37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41970/0 (socket says 172.21.15.12:41970) 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72a37fe700 1 -- 172.21.15.12:0/2761089905 learned_addr learned my addr 172.21.15.12:0/2761089905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72a37fe700 1 -- 172.21.15.12:0/2761089905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294005040 con 0x7f72a40669e0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72a37fe700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7294009680 tx=0x7f7294009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f729400bed0 con 0x7f72a40669e0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f729400b5c0 con 0x7f72a40669e0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a410d820 con 0x7f72a40669e0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f729401c5b0 con 0x7f72a40669e0 2024-03-18T14:42:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.025+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a410e390 con 0x7f72a40669e0 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.026+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f729401c710 con 0x7f72a40669e0 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.026+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a4102f60 con 0x7f72a40669e0 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.026+0000 7f728bfff700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 0x7f728c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.026+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7294049950 con 0x7f72a40669e0 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.026+0000 7f72a2ffd700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 0x7f728c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.027+0000 7f72a2ffd700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 0x7f728c038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7298009990 tx=0x7f7298006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:50.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.031+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f7294011ba0 con 0x7f72a40669e0 2024-03-18T14:42:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:50.227+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f72a410e5d0 con 0x7f728c0368d0 2024-03-18T14:42:51.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:51 smithi012 ceph-mon[29182]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:42:51.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:51 smithi012 ceph-mon[29182]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:51.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:51 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:52.420 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-03-18T14:42:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.418+0000 7f728bfff700 1 -- 172.21.15.12:0/2761089905 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f72a410e5d0 con 0x7f728c0368d0 2024-03-18T14:42:52.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 msgr2=0x7f728c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:52.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 0x7f728c038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7298009990 tx=0x7f7298006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:52.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 msgr2=0x7f72a410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7294009680 tx=0x7f7294009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 shutdown_connections 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f728c0368d0 0x7f728c038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 --2- 172.21.15.12:0/2761089905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72a40669e0 0x7f72a410d0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 >> 172.21.15.12:0/2761089905 conn(0x7f72a40f70c0 msgr2=0x7f72a4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 shutdown_connections 2024-03-18T14:42:52.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:52.420+0000 7f72aa056700 1 -- 172.21.15.12:0/2761089905 wait complete. 2024-03-18T14:42:52.738 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:52 smithi012 ceph-mon[29182]: Deploying cephadm binary to smithi100 2024-03-18T14:42:52.738 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:52 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:53.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T14:42:53.348 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:42:54.212 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:53 smithi012 ceph-mon[29182]: Added host smithi100 2024-03-18T14:42:54.212 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:53 smithi012 ceph-mon[29182]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 -- 172.21.15.12:0/3891511242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 msgr2=0x7f083c0fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 --2- 172.21.15.12:0/3891511242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c0fbd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08380048c0 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 -- 172.21.15.12:0/3891511242 shutdown_connections 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 --2- 172.21.15.12:0/3891511242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c0fbd60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 -- 172.21.15.12:0/3891511242 >> 172.21.15.12:0/3891511242 conn(0x7f083c0f7090 msgr2=0x7f083c0f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:55.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 -- 172.21.15.12:0/3891511242 shutdown_connections 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.098+0000 7f0843277700 1 -- 172.21.15.12:0/3891511242 wait complete. 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0843277700 1 Processor -- start 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0843277700 1 -- start start 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0843277700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0843277700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083c10d5d0 con 0x7f083c0fb950 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0841013700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0841013700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41988/0 (socket says 172.21.15.12:41988) 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0841013700 1 -- 172.21.15.12:0/3752338477 learned_addr learned my addr 172.21.15.12:0/3752338477 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0841013700 1 -- 172.21.15.12:0/3752338477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838005040 con 0x7f083c0fb950 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.099+0000 7f0841013700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08380096b0 tx=0x7f0838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.100+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f083800bed0 con 0x7f083c0fb950 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.100+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f083800b5c0 con 0x7f083c0fb950 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.100+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083c10d7d0 con 0x7f083c0fb950 2024-03-18T14:42:55.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.100+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f083801c5b0 con 0x7f083c0fb950 2024-03-18T14:42:55.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.100+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083c10e340 con 0x7f083c0fb950 2024-03-18T14:42:55.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.101+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f083801c710 con 0x7f083c0fb950 2024-03-18T14:42:55.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.101+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083c101e60 con 0x7f083c0fb950 2024-03-18T14:42:55.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.101+0000 7f0831ffb700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 0x7f0828038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:55.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.101+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0838049ac0 con 0x7f083c0fb950 2024-03-18T14:42:55.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.102+0000 7f0840812700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 0x7f0828038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:55.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.102+0000 7f0840812700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 0x7f0828038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f082c009990 tx=0x7f082c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.108+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f083804be60 con 0x7f083c0fb950 2024-03-18T14:42:55.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.308+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f083c05f690 con 0x7f08280368d0 2024-03-18T14:42:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.309+0000 7f0831ffb700 1 -- 172.21.15.12:0/3752338477 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f083c05f690 con 0x7f08280368d0 2024-03-18T14:42:55.311 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:42:55.311 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-03-18T14:42:55.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 msgr2=0x7f0828038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 0x7f0828038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f082c009990 tx=0x7f082c006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 msgr2=0x7f083c10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08380096b0 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 shutdown_connections 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f08280368d0 0x7f0828038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 --2- 172.21.15.12:0/3752338477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f083c0fb950 0x7f083c10d090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 >> 172.21.15.12:0/3752338477 conn(0x7f083c0f7090 msgr2=0x7f083c065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 shutdown_connections 2024-03-18T14:42:55.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:55.311+0000 7f0843277700 1 -- 172.21.15.12:0/3752338477 wait complete. 2024-03-18T14:42:55.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:55 smithi012 ceph-mon[29182]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:56.222 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-18T14:42:56.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-18T14:42:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:42:57.075 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:56 smithi012 ceph-mon[29182]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T14:42:57.075 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:56 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:57.075 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:56 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:57.075 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:56 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:57.075 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:56 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:42:58.072 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:57 smithi012 ceph-mon[29182]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:42:58.072 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:57 smithi012 ceph-mon[29182]: Deploying daemon alertmanager.smithi012 on smithi012 2024-03-18T14:42:58.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- 172.21.15.12:0/1987634315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 msgr2=0x7f568c0fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:58.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 --2- 172.21.15.12:0/1987634315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c0fce60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5684009c60 tx=0x7f5684004d10 comp rx=0 tx=0).stop 2024-03-18T14:42:58.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- 172.21.15.12:0/1987634315 shutdown_connections 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 --2- 172.21.15.12:0/1987634315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c0fce60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- 172.21.15.12:0/1987634315 >> 172.21.15.12:0/1987634315 conn(0x7f568c0f8190 msgr2=0x7f568c0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- 172.21.15.12:0/1987634315 shutdown_connections 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- 172.21.15.12:0/1987634315 wait complete. 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 Processor -- start 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.266+0000 7f5693be2700 1 -- start start 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f5693be2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f5693be2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c10d5f0 con 0x7f568c0fca50 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f569197e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f569197e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42002/0 (socket says 172.21.15.12:42002) 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f569197e700 1 -- 172.21.15.12:0/3655536668 learned_addr learned my addr 172.21.15.12:0/3655536668 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f569197e700 1 -- 172.21.15.12:0/3655536668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009940 con 0x7f568c0fca50 2024-03-18T14:42:58.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f569197e700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5684009500 tx=0x7f5684004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5684007c80 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f568401d910 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c10d7f0 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56840129d0 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.267+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c10e360 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.268+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f5684019040 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.268+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c102f60 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.268+0000 7f568a7fc700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 0x7f5674038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.268+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f568404a420 con 0x7f568c0fca50 2024-03-18T14:42:58.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.269+0000 7f569117d700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 0x7f5674038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:42:58.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.269+0000 7f569117d700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 0x7f5674038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5680009990 tx=0x7f5680006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:42:58.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.273+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5684024310 con 0x7f568c0fca50 2024-03-18T14:42:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.477+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f568c10ecc0 con 0x7f568c0fca50 2024-03-18T14:42:58.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.811+0000 7f568a7fc700 1 -- 172.21.15.12:0/3655536668 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5684017090 con 0x7f568c0fca50 2024-03-18T14:42:58.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 msgr2=0x7f5674038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:58.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 0x7f5674038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5680009990 tx=0x7f5680006e40 comp rx=0 tx=0).stop 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 msgr2=0x7f568c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5684009500 tx=0x7f5684004d10 comp rx=0 tx=0).stop 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 shutdown_connections 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f56740368d0 0x7f5674038d80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 --2- 172.21.15.12:0/3655536668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f568c0fca50 0x7f568c10d0b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.813+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 >> 172.21.15.12:0/3655536668 conn(0x7f568c0f8190 msgr2=0x7f568c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.814+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 shutdown_connections 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:42:58.814+0000 7f5693be2700 1 -- 172.21.15.12:0/3655536668 wait complete. 2024-03-18T14:42:58.816 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-03-18T14:42:59.117 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:58 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3655536668' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T14:42:59.407 INFO:tasks.cephadm:Adding mon.smithi012 on smithi012 2024-03-18T14:42:59.407 INFO:tasks.cephadm:Adding mon.smithi100 on smithi100 2024-03-18T14:42:59.408 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch apply mon '2;smithi012:172.21.15.12=smithi012;smithi100:172.21.15.100=smithi100' 2024-03-18T14:42:59.656 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:42:59.703 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:59 smithi012 ceph-mon[29182]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:00.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:59 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3655536668' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T14:43:00.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:42:59 smithi012 ceph-mon[29182]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- 172.21.15.100:0/1348261063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 msgr2=0x7f57780fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 --2- 172.21.15.100:0/1348261063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780fbd60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f57680048c0 tx=0x7f5768009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- 172.21.15.100:0/1348261063 shutdown_connections 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 --2- 172.21.15.100:0/1348261063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780fbd60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- 172.21.15.100:0/1348261063 >> 172.21.15.100:0/1348261063 conn(0x7f57780f7090 msgr2=0x7f57780f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- 172.21.15.100:0/1348261063 shutdown_connections 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- 172.21.15.100:0/1348261063 wait complete. 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 Processor -- start 2024-03-18T14:43:01.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.197+0000 7f577e37e700 1 -- start start 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f577e37e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f577e37e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810fd60 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f5777fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f5777fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:53872/0 (socket says 172.21.15.100:53872) 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f5777fff700 1 -- 172.21.15.100:0/858826703 learned_addr learned my addr 172.21.15.100:0/858826703 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f5777fff700 1 -- 172.21.15.100:0/858826703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768005040 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.198+0000 7f5777fff700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57680096b0 tx=0x7f5768009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.199+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f576800bed0 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.199+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577810ff00 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.199+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f576800b5c0 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.199+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778110390 con 0x7f57780fb950 2024-03-18T14:43:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.199+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f576801c5b0 con 0x7f57780fb950 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.200+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f5768013420 con 0x7f57780fb950 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.200+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640052e0 con 0x7f57780fb950 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.200+0000 7f5774ff9700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 0x7f5760038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.200+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5768049920 con 0x7f57780fb950 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.200+0000 7f57777fe700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 0x7f5760038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.201+0000 7f57777fe700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 0x7f5760038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f576c009990 tx=0x7f576c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.205+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5768004d50 con 0x7f57780fb950 2024-03-18T14:43:01.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.396+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 --> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi012:172.21.15.12=smithi012;smithi100:172.21.15.100=smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f5764000ca0 con 0x7f5760036880 2024-03-18T14:43:01.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.402+0000 7f5774ff9700 1 -- 172.21.15.100:0/858826703 <== mgr.14162 v2:172.21.15.12:6800/1147707934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5764000ca0 con 0x7f5760036880 2024-03-18T14:43:01.404 INFO:teuthology.orchestra.run.smithi100.stdout:Scheduled mon update... 2024-03-18T14:43:01.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 msgr2=0x7f5760038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:01.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 0x7f5760038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f576c009990 tx=0x7f576c006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:01.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 msgr2=0x7f57780668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:01.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57680096b0 tx=0x7f5768009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 shutdown_connections 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f5760036880 0x7f5760038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 --2- 172.21.15.100:0/858826703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57780fb950 0x7f57780668a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 >> 172.21.15.100:0/858826703 conn(0x7f57780f7090 msgr2=0x7f5778102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 shutdown_connections 2024-03-18T14:43:01.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:01.404+0000 7f577e37e700 1 -- 172.21.15.100:0/858826703 wait complete. 2024-03-18T14:43:01.986 DEBUG:teuthology.orchestra.run.smithi100:mon.smithi100> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi100.service 2024-03-18T14:43:01.988 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:01.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:01 smithi012 ceph-mon[29182]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:01.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:01 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:01.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:01 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:02.264 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:02.311 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:02.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:02 smithi012 ceph-mon[29182]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi012:172.21.15.12=smithi012;smithi100:172.21.15.100=smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:43:02.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:02 smithi012 ceph-mon[29182]: Saving service mon spec with placement smithi012:172.21.15.12=smithi012;smithi100:172.21.15.100=smithi100;count:2 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 -- 172.21.15.100:0/394530777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 msgr2=0x7f47700fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 --2- 172.21.15.100:0/394530777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f47700fbd70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f476c0048c0 tx=0x7f476c009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 -- 172.21.15.100:0/394530777 shutdown_connections 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 --2- 172.21.15.100:0/394530777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f47700fbd70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 -- 172.21.15.100:0/394530777 >> 172.21.15.100:0/394530777 conn(0x7f47700f70c0 msgr2=0x7f47700f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:03.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 -- 172.21.15.100:0/394530777 shutdown_connections 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.588+0000 7f477791e700 1 -- 172.21.15.100:0/394530777 wait complete. 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f477791e700 1 Processor -- start 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f477791e700 1 -- start start 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f477791e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f477791e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477010fdb0 con 0x7f47700fb960 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f47756ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f47756ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:53880/0 (socket says 172.21.15.100:53880) 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f47756ba700 1 -- 172.21.15.100:0/1736657610 learned_addr learned my addr 172.21.15.100:0/1736657610 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.589+0000 7f47756ba700 1 -- 172.21.15.100:0/1736657610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c005040 con 0x7f47700fb960 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f47756ba700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f476c009680 tx=0x7f476c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f476c00b5c0 con 0x7f47700fb960 2024-03-18T14:43:03.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477010fef0 con 0x7f47700fb960 2024-03-18T14:43:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47701103e0 con 0x7f47700fb960 2024-03-18T14:43:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f476c00be70 con 0x7f47700fb960 2024-03-18T14:43:03.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.590+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f476c013420 con 0x7f47700fb960 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.591+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47540052e0 con 0x7f47700fb960 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.591+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f476c01c430 con 0x7f47700fb960 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.591+0000 7f4765ffb700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 0x7f475c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.591+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f476c0494f0 con 0x7f47700fb960 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.591+0000 7f4774eb9700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 0x7f475c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.592+0000 7f4774eb9700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 0x7f475c038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4760009990 tx=0x7f4760006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:03.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.596+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f476c0249b0 con 0x7f47700fb960 2024-03-18T14:43:03.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.849+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4754005590 con 0x7f47700fb960 2024-03-18T14:43:03.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.849+0000 7f4765ffb700 1 -- 172.21.15.100:0/1736657610 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f476c0247d0 con 0x7f47700fb960 2024-03-18T14:43:03.851 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:03.851 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 msgr2=0x7f475c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 0x7f475c038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4760009990 tx=0x7f4760006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 msgr2=0x7f4770066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f476c009680 tx=0x7f476c009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 shutdown_connections 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f475c036880 0x7f475c038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 --2- 172.21.15.100:0/1736657610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47700fb960 0x7f4770066860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.851+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 >> 172.21.15.100:0/1736657610 conn(0x7f47700f70c0 msgr2=0x7f4770102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.852+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 shutdown_connections 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:03.852+0000 7f477791e700 1 -- 172.21.15.100:0/1736657610 wait complete. 2024-03-18T14:43:03.854 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:04.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:03 smithi012 ceph-mon[29182]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:05.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:04 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/1736657610' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:05.390 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:05.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:05.641 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:05.689 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:06.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:05 smithi012 ceph-mon[29182]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 -- 172.21.15.100:0/2183904090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 msgr2=0x7fe2480fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2183904090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe2480fbd70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe2440048c0 tx=0x7fe244009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 -- 172.21.15.100:0/2183904090 shutdown_connections 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2183904090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe2480fbd70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 -- 172.21.15.100:0/2183904090 >> 172.21.15.100:0/2183904090 conn(0x7fe2480f70c0 msgr2=0x7fe2480f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 -- 172.21.15.100:0/2183904090 shutdown_connections 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.143+0000 7fe24f36c700 1 -- 172.21.15.100:0/2183904090 wait complete. 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24f36c700 1 Processor -- start 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24f36c700 1 -- start start 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24f36c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24f36c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24810fd90 con 0x7fe2480fb960 2024-03-18T14:43:07.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24d108700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24d108700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:53890/0 (socket says 172.21.15.100:53890) 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24d108700 1 -- 172.21.15.100:0/2793199516 learned_addr learned my addr 172.21.15.100:0/2793199516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24d108700 1 -- 172.21.15.100:0/2793199516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244005040 con 0x7fe2480fb960 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.144+0000 7fe24d108700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe244009680 tx=0x7fe244009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.145+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe24400b5c0 con 0x7fe2480fb960 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.145+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24810fed0 con 0x7fe2480fb960 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.145+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe24400be70 con 0x7fe2480fb960 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.145+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2440135a0 con 0x7fe2480fb960 2024-03-18T14:43:07.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.145+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2481103c0 con 0x7fe2480fb960 2024-03-18T14:43:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.146+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248105920 con 0x7fe2480fb960 2024-03-18T14:43:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.146+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe244004d30 con 0x7fe2480fb960 2024-03-18T14:43:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.146+0000 7fe23dffb700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 0x7fe234038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.146+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe244049430 con 0x7fe2480fb960 2024-03-18T14:43:07.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.146+0000 7fe24c907700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 0x7fe234038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:07.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.147+0000 7fe24c907700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 0x7fe234038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe238009990 tx=0x7fe238006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:07.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.151+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe244011960 con 0x7fe2480fb960 2024-03-18T14:43:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.405+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe24805f5f0 con 0x7fe2480fb960 2024-03-18T14:43:07.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.406+0000 7fe23dffb700 1 -- 172.21.15.100:0/2793199516 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe244021070 con 0x7fe2480fb960 2024-03-18T14:43:07.408 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:07.408 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 msgr2=0x7fe234038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 0x7fe234038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe238009990 tx=0x7fe238006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 msgr2=0x7fe248066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe244009680 tx=0x7fe244009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 shutdown_connections 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fe234036880 0x7fe234038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 --2- 172.21.15.100:0/2793199516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2480fb960 0x7fe248066880 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 >> 172.21.15.100:0/2793199516 conn(0x7fe2480f70c0 msgr2=0x7fe248102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 shutdown_connections 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:07.408+0000 7fe24f36c700 1 -- 172.21.15.100:0/2793199516 wait complete. 2024-03-18T14:43:07.414 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:08.125 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:07 smithi012 ceph-mon[29182]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:08.125 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2793199516' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:08.987 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:08.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:09.236 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:09.285 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:09.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:09 smithi012 ceph-mon[29182]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:10.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- 172.21.15.100:0/2480834768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 msgr2=0x7f44bc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:10.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2480834768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc0fce70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0048c0 tx=0x7f44ac009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- 172.21.15.100:0/2480834768 shutdown_connections 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2480834768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc0fce70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- 172.21.15.100:0/2480834768 >> 172.21.15.100:0/2480834768 conn(0x7f44bc0f81c0 msgr2=0x7f44bc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- 172.21.15.100:0/2480834768 shutdown_connections 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- 172.21.15.100:0/2480834768 wait complete. 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 Processor -- start 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.715+0000 7f44c20ff700 1 -- start start 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44c20ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44c20ff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc10d5d0 con 0x7f44bc0fca60 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44bb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44bb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:45856/0 (socket says 172.21.15.100:45856) 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44bb7fe700 1 -- 172.21.15.100:0/2625946052 learned_addr learned my addr 172.21.15.100:0/2625946052 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44bb7fe700 1 -- 172.21.15.100:0/2625946052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac005040 con 0x7f44bc0fca60 2024-03-18T14:43:10.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44bb7fe700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0096b0 tx=0x7f44ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:10.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac00bed0 con 0x7f44bc0fca60 2024-03-18T14:43:10.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44ac00b5c0 con 0x7f44bc0fca60 2024-03-18T14:43:10.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc10d7d0 con 0x7f44bc0fca60 2024-03-18T14:43:10.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac01c5b0 con 0x7f44bc0fca60 2024-03-18T14:43:10.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.716+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc10e340 con 0x7f44bc0fca60 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.717+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f44ac01c820 con 0x7f44bc0fca60 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.717+0000 7f44a3fff700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 0x7f44a4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.717+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f44ac049b10 con 0x7f44bc0fca60 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.717+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc102f70 con 0x7f44bc0fca60 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.718+0000 7f44baffd700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 0x7f44a4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:10.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.718+0000 7f44baffd700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 0x7f44a4038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009990 tx=0x7f44b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:10.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.722+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f44ac023300 con 0x7f44bc0fca60 2024-03-18T14:43:10.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.980+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f44bc05f5f0 con 0x7f44bc0fca60 2024-03-18T14:43:10.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.980+0000 7f44a3fff700 1 -- 172.21.15.100:0/2625946052 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f44ac011b60 con 0x7f44bc0fca60 2024-03-18T14:43:10.982 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:10.982 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 msgr2=0x7f44a4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 0x7f44a4038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009990 tx=0x7f44b0006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 msgr2=0x7f44bc10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:10.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0096b0 tx=0x7f44ac009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 shutdown_connections 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f44a4036880 0x7f44a4038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 --2- 172.21.15.100:0/2625946052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44bc0fca60 0x7f44bc10d090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.982+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 >> 172.21.15.100:0/2625946052 conn(0x7f44bc0f81c0 msgr2=0x7f44bc103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.983+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 shutdown_connections 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:10.983+0000 7f44c20ff700 1 -- 172.21.15.100:0/2625946052 wait complete. 2024-03-18T14:43:10.985 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:11.979 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: Deploying daemon grafana.smithi012 on smithi012 2024-03-18T14:43:11.980 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:11 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2625946052' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:12.574 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:12.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:12.815 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:12.862 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- 172.21.15.100:0/2640914592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 msgr2=0x7fb9000fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 --2- 172.21.15.100:0/2640914592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb9000fbd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00048c0 tx=0x7fb8f0009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- 172.21.15.100:0/2640914592 shutdown_connections 2024-03-18T14:43:14.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 --2- 172.21.15.100:0/2640914592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb9000fbd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- 172.21.15.100:0/2640914592 >> 172.21.15.100:0/2640914592 conn(0x7fb9000f7090 msgr2=0x7fb9000f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- 172.21.15.100:0/2640914592 shutdown_connections 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- 172.21.15.100:0/2640914592 wait complete. 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 Processor -- start 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.179+0000 7fb905e1c700 1 -- start start 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb905e1c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:14.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb905e1c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90010f7b0 con 0x7fb9000fb950 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:45868/0 (socket says 172.21.15.100:45868) 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8ff7fe700 1 -- 172.21.15.100:0/3959197261 learned_addr learned my addr 172.21.15.100:0/3959197261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8ff7fe700 1 -- 172.21.15.100:0/3959197261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f0005040 con 0x7fb9000fb950 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8ff7fe700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00096b0 tx=0x7fb8f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8f000bed0 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90010f9b0 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8f000b5c0 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8f001c5b0 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.180+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb900110520 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.181+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb8f001c820 con 0x7fb9000fb950 2024-03-18T14:43:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.181+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900101e60 con 0x7fb9000fb950 2024-03-18T14:43:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.181+0000 7fb8e7fff700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 0x7fb8e8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.181+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb8f0049bd0 con 0x7fb9000fb950 2024-03-18T14:43:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.182+0000 7fb8feffd700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 0x7fb8e8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:14.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.182+0000 7fb8feffd700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 0x7fb8e8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.187+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb8f0024b50 con 0x7fb9000fb950 2024-03-18T14:43:14.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:13 smithi012 ceph-mon[29182]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:14.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.441+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb900110760 con 0x7fb9000fb950 2024-03-18T14:43:14.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.441+0000 7fb8e7fff700 1 -- 172.21.15.100:0/3959197261 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb8f0024b50 con 0x7fb9000fb950 2024-03-18T14:43:14.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:14.444 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 msgr2=0x7fb8e8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 0x7fb8e8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 msgr2=0x7fb90010f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00096b0 tx=0x7fb8f0009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 shutdown_connections 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb8e8036880 0x7fb8e8038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:14.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 --2- 172.21.15.100:0/3959197261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000fb950 0x7fb90010f270 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:14.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 >> 172.21.15.100:0/3959197261 conn(0x7fb9000f7090 msgr2=0x7fb9000f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:14.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 shutdown_connections 2024-03-18T14:43:14.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:14.443+0000 7fb905e1c700 1 -- 172.21.15.100:0/3959197261 wait complete. 2024-03-18T14:43:14.447 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:15.143 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:14 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3959197261' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:16.019 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:16.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:15 smithi012 ceph-mon[29182]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:16.267 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:17.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 -- 172.21.15.100:0/3061616816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 msgr2=0x7f80700ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:17.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 --2- 172.21.15.100:0/3061616816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f80700ff650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f80600048c0 tx=0x7f8060009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:17.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 -- 172.21.15.100:0/3061616816 shutdown_connections 2024-03-18T14:43:17.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 --2- 172.21.15.100:0/3061616816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f80700ff650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:17.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 -- 172.21.15.100:0/3061616816 >> 172.21.15.100:0/3061616816 conn(0x7f80700f70c0 msgr2=0x7f80700f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:17.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 -- 172.21.15.100:0/3061616816 shutdown_connections 2024-03-18T14:43:17.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.789+0000 7f8075567700 1 -- 172.21.15.100:0/3061616816 wait complete. 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f8075567700 1 Processor -- start 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f8075567700 1 -- start start 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f8075567700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f8075567700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807010d610 con 0x7f8070066a40 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f806effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f806effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:45890/0 (socket says 172.21.15.100:45890) 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f806effd700 1 -- 172.21.15.100:0/2914848124 learned_addr learned my addr 172.21.15.100:0/2914848124 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.790+0000 7f806effd700 1 -- 172.21.15.100:0/2914848124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060005040 con 0x7f8070066a40 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f806effd700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8060009680 tx=0x7f8060009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f806000bed0 con 0x7f8070066a40 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807010d810 con 0x7f8070066a40 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f806000b5c0 con 0x7f8070066a40 2024-03-18T14:43:17.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f806001c5b0 con 0x7f8070066a40 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.791+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807010e380 con 0x7f8070066a40 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f806001c710 con 0x7f8070066a40 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8070102f60 con 0x7f8070066a40 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f8067fff700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 0x7f805c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8060049a00 con 0x7f8070066a40 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f80667fc700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 0x7f805c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:17.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.792+0000 7f80667fc700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 0x7f805c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8058009990 tx=0x7f8058006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:17.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:17.797+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f8060025400 con 0x7f8070066a40 2024-03-18T14:43:18.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.053+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f807010ee50 con 0x7f8070066a40 2024-03-18T14:43:18.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.054+0000 7f8067fff700 1 -- 172.21.15.100:0/2914848124 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8060023300 con 0x7f8070066a40 2024-03-18T14:43:18.056 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:18.056 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 msgr2=0x7f805c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 0x7f805c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8058009990 tx=0x7f8058006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 msgr2=0x7f807010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8060009680 tx=0x7f8060009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 shutdown_connections 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f805c036880 0x7f805c038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 --2- 172.21.15.100:0/2914848124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8070066a40 0x7f807010d0d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 >> 172.21.15.100:0/2914848124 conn(0x7f80700f70c0 msgr2=0x7f8070103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 shutdown_connections 2024-03-18T14:43:18.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:18.056+0000 7f8075567700 1 -- 172.21.15.100:0/2914848124 wait complete. 2024-03-18T14:43:18.059 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:17 smithi012 ceph-mon[29182]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:18 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2914848124' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:19.547 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:19.548 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:19.794 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:19.840 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:20.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:19 smithi012 ceph-mon[29182]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.217+0000 7fb039a6d700 1 -- 172.21.15.100:0/75190682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 msgr2=0x7fb0340fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.217+0000 7fb039a6d700 1 --2- 172.21.15.100:0/75190682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fe4b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0240048c0 tx=0x7fb024009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.217+0000 7fb039a6d700 1 -- 172.21.15.100:0/75190682 shutdown_connections 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.217+0000 7fb039a6d700 1 --2- 172.21.15.100:0/75190682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fe4b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 -- 172.21.15.100:0/75190682 >> 172.21.15.100:0/75190682 conn(0x7fb0340f6820 msgr2=0x7fb0340f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 -- 172.21.15.100:0/75190682 shutdown_connections 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 -- 172.21.15.100:0/75190682 wait complete. 2024-03-18T14:43:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 Processor -- start 2024-03-18T14:43:21.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 -- start start 2024-03-18T14:43:21.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:21.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.218+0000 7fb039a6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340fdcf0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:21.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47108/0 (socket says 172.21.15.100:47108) 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb0337fe700 1 -- 172.21.15.100:0/849350576 learned_addr learned my addr 172.21.15.100:0/849350576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb0337fe700 1 -- 172.21.15.100:0/849350576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024005040 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb0337fe700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb024009680 tx=0x7fb024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb02400b5c0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb02400be70 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0340fbdc0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0240135a0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.219+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0340fc2b0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.220+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb024013810 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.220+0000 7fb01bfff700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 0x7fb01c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.220+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb0240493e0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.220+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03405f5f0 con 0x7fb0340fe0a0 2024-03-18T14:43:21.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.221+0000 7fb032ffd700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 0x7fb01c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:21.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.221+0000 7fb032ffd700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 0x7fb01c038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb028009990 tx=0x7fb028006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:21.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.226+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb024021020 con 0x7fb0340fe0a0 2024-03-18T14:43:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.479+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb034106440 con 0x7fb0340fe0a0 2024-03-18T14:43:21.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.479+0000 7fb01bfff700 1 -- 172.21.15.100:0/849350576 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb024013b60 con 0x7fb0340fe0a0 2024-03-18T14:43:21.482 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:21.482 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 msgr2=0x7fb01c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 0x7fb01c038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb028009990 tx=0x7fb028006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 msgr2=0x7fb0340fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb024009680 tx=0x7fb024009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 shutdown_connections 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fb01c0368d0 0x7fb01c038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:21.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 --2- 172.21.15.100:0/849350576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0340fe0a0 0x7fb0340fd7b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:21.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 >> 172.21.15.100:0/849350576 conn(0x7fb0340f6820 msgr2=0x7fb034103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:21.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 shutdown_connections 2024-03-18T14:43:21.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:21.482+0000 7fb039a6d700 1 -- 172.21.15.100:0/849350576 wait complete. 2024-03-18T14:43:21.485 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:22.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:21 smithi012 ceph-mon[29182]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:22.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:21 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/849350576' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:23.032 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:23.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:23.282 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:24.100 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:23 smithi012 ceph-mon[29182]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:24.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- 172.21.15.100:0/2125279526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 msgr2=0x7f7e200fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/2125279526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e200fce70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e100048c0 tx=0x7f7e10009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- 172.21.15.100:0/2125279526 shutdown_connections 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/2125279526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e200fce70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- 172.21.15.100:0/2125279526 >> 172.21.15.100:0/2125279526 conn(0x7f7e200f81c0 msgr2=0x7f7e200fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- 172.21.15.100:0/2125279526 shutdown_connections 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- 172.21.15.100:0/2125279526 wait complete. 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 Processor -- start 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.754+0000 7f7e25ba4700 1 -- start start 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e25ba4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e25ba4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010d5d0 con 0x7f7e200fca60 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:24.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47120/0 (socket says 172.21.15.100:47120) 2024-03-18T14:43:24.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1f7fe700 1 -- 172.21.15.100:0/996659122 learned_addr learned my addr 172.21.15.100:0/996659122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:24.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1f7fe700 1 -- 172.21.15.100:0/996659122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10005040 con 0x7f7e200fca60 2024-03-18T14:43:24.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1f7fe700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e100096b0 tx=0x7f7e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e1000bed0 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2010d7d0 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e1000b5c0 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e1001c5b0 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.755+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2010e340 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.756+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e20102f70 con 0x7f7e200fca60 2024-03-18T14:43:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.756+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7e10013a80 con 0x7f7e200fca60 2024-03-18T14:43:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.757+0000 7f7e1cff9700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 0x7f7e0c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.757+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7e10049ad0 con 0x7f7e200fca60 2024-03-18T14:43:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.757+0000 7f7e17fff700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 0x7f7e0c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.757+0000 7f7e17fff700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 0x7f7e0c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009990 tx=0x7f7e08006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:24.762+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f7e10011940 con 0x7f7e200fca60 2024-03-18T14:43:25.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.016+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7e2010ee10 con 0x7f7e200fca60 2024-03-18T14:43:25.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.017+0000 7f7e1cff9700 1 -- 172.21.15.100:0/996659122 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7e10011370 con 0x7f7e200fca60 2024-03-18T14:43:25.018 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:25.019 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:25.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 msgr2=0x7f7e0c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 0x7f7e0c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009990 tx=0x7f7e08006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 msgr2=0x7f7e2010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e100096b0 tx=0x7f7e10009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 shutdown_connections 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7e0c036880 0x7f7e0c038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 --2- 172.21.15.100:0/996659122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e200fca60 0x7f7e2010d090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 >> 172.21.15.100:0/996659122 conn(0x7f7e200f81c0 msgr2=0x7f7e20103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 shutdown_connections 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:25.019+0000 7f7e25ba4700 1 -- 172.21.15.100:0/996659122 wait complete. 2024-03-18T14:43:25.021 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:25.952 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:25 smithi012 ceph-mon[29182]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:25.952 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:25 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/996659122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:26.655 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:26.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:26.902 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:26.948 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: Deploying daemon prometheus.smithi012 on smithi012 2024-03-18T14:43:27.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:27 smithi012 ceph-mon[29182]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 -- 172.21.15.100:0/2316470448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 msgr2=0x7f47400fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 --2- 172.21.15.100:0/2316470448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f47400fce50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47340048c0 tx=0x7f4734009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 -- 172.21.15.100:0/2316470448 shutdown_connections 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 --2- 172.21.15.100:0/2316470448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f47400fce50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 -- 172.21.15.100:0/2316470448 >> 172.21.15.100:0/2316470448 conn(0x7f47400f81c0 msgr2=0x7f47400fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.473+0000 7f474725f700 1 -- 172.21.15.100:0/2316470448 shutdown_connections 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.474+0000 7f474725f700 1 -- 172.21.15.100:0/2316470448 wait complete. 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.474+0000 7f474725f700 1 Processor -- start 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.474+0000 7f474725f700 1 -- start start 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.474+0000 7f474725f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.474+0000 7f474725f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474010d5d0 con 0x7f47400fca40 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f4744ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f4744ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47136/0 (socket says 172.21.15.100:47136) 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f4744ffb700 1 -- 172.21.15.100:0/1990879932 learned_addr learned my addr 172.21.15.100:0/1990879932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f4744ffb700 1 -- 172.21.15.100:0/1990879932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734005040 con 0x7f47400fca40 2024-03-18T14:43:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f4744ffb700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4734009680 tx=0x7f4734009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f473400b5c0 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f473400be70 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474010d7d0 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47340135a0 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.475+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474010e340 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.476+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f473401c430 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.476+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4740102f50 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.476+0000 7f473d7fa700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 0x7f4730038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.476+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4734049420 con 0x7f47400fca40 2024-03-18T14:43:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.476+0000 7f473ffff700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 0x7f4730038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:28.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.477+0000 7f473ffff700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 0x7f4730038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f472c009990 tx=0x7f472c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:28.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.481+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f4734011bd0 con 0x7f47400fca40 2024-03-18T14:43:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.736+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f474010ed50 con 0x7f47400fca40 2024-03-18T14:43:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.737+0000 7f473d7fa700 1 -- 172.21.15.100:0/1990879932 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4734021070 con 0x7f47400fca40 2024-03-18T14:43:28.739 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:28.739 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 msgr2=0x7f4730038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 0x7f4730038d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f472c009990 tx=0x7f472c006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 msgr2=0x7f474010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4734009680 tx=0x7f4734009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 shutdown_connections 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f47300368d0 0x7f4730038d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.738+0000 7f474725f700 1 --2- 172.21.15.100:0/1990879932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47400fca40 0x7f474010d090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.739+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 >> 172.21.15.100:0/1990879932 conn(0x7f47400f81c0 msgr2=0x7f4740103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.739+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 shutdown_connections 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:28.739+0000 7f474725f700 1 -- 172.21.15.100:0/1990879932 wait complete. 2024-03-18T14:43:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:28 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/1990879932' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:29 smithi012 ceph-mon[29182]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:30.290 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:30.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:30.538 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:30.586 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:31 smithi012 ceph-mon[29182]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:31.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:31 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 -- 172.21.15.100:0/1575020062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 msgr2=0x7fea140fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 --2- 172.21.15.100:0/1575020062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea140fce90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea100048c0 tx=0x7fea10009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 -- 172.21.15.100:0/1575020062 shutdown_connections 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 --2- 172.21.15.100:0/1575020062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea140fce90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 -- 172.21.15.100:0/1575020062 >> 172.21.15.100:0/1575020062 conn(0x7fea140f81e0 msgr2=0x7fea140fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 -- 172.21.15.100:0/1575020062 shutdown_connections 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.109+0000 7fea1b91f700 1 -- 172.21.15.100:0/1575020062 wait complete. 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea1b91f700 1 Processor -- start 2024-03-18T14:43:32.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea1b91f700 1 -- start start 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea1b91f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea1b91f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1410d5f0 con 0x7fea140fca80 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea196bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea196bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:58686/0 (socket says 172.21.15.100:58686) 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.110+0000 7fea196bb700 1 -- 172.21.15.100:0/165509527 learned_addr learned my addr 172.21.15.100:0/165509527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea196bb700 1 -- 172.21.15.100:0/165509527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10005040 con 0x7fea140fca80 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea196bb700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea10009680 tx=0x7fea10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea1000b5c0 con 0x7fea140fca80 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea1000be70 con 0x7fea140fca80 2024-03-18T14:43:32.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1410d7f0 con 0x7fea140fca80 2024-03-18T14:43:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea100135a0 con 0x7fea140fca80 2024-03-18T14:43:32.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.111+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1410e360 con 0x7fea140fca80 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.112+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea14102f90 con 0x7fea140fca80 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.112+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fea10004d30 con 0x7fea140fca80 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.112+0000 7fea09ffb700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 0x7fea00038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.112+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fea10049430 con 0x7fea140fca80 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.113+0000 7fea18eba700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 0x7fea00038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:32.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.113+0000 7fea18eba700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 0x7fea00038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea04009990 tx=0x7fea04006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:32.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.117+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fea10011960 con 0x7fea140fca80 2024-03-18T14:43:32.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.370+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fea1410ed70 con 0x7fea140fca80 2024-03-18T14:43:32.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.371+0000 7fea09ffb700 1 -- 172.21.15.100:0/165509527 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fea10021070 con 0x7fea140fca80 2024-03-18T14:43:32.373 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:32.373 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 msgr2=0x7fea00038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 0x7fea00038d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea04009990 tx=0x7fea04006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 msgr2=0x7fea1410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea10009680 tx=0x7fea10009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 shutdown_connections 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fea00036880 0x7fea00038d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 --2- 172.21.15.100:0/165509527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea140fca80 0x7fea1410d0b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 >> 172.21.15.100:0/165509527 conn(0x7fea140f81e0 msgr2=0x7fea14103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 shutdown_connections 2024-03-18T14:43:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:32.373+0000 7fea1b91f700 1 -- 172.21.15.100:0/165509527 wait complete. 2024-03-18T14:43:32.376 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:32.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:32 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/165509527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:33.977 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:33.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:34.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:33 smithi012 ceph-mon[29182]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- 172.21.15.100:0/3750817185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 msgr2=0x7fd8640fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3750817185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd8640fbd70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8600048c0 tx=0x7fd860009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- 172.21.15.100:0/3750817185 shutdown_connections 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3750817185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd8640fbd70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- 172.21.15.100:0/3750817185 >> 172.21.15.100:0/3750817185 conn(0x7fd8640f70c0 msgr2=0x7fd8640f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- 172.21.15.100:0/3750817185 shutdown_connections 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- 172.21.15.100:0/3750817185 wait complete. 2024-03-18T14:43:35.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 Processor -- start 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- start start 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.681+0000 7fd86bc88700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86410d5c0 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd869a24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd869a24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:58704/0 (socket says 172.21.15.100:58704) 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd869a24700 1 -- 172.21.15.100:0/3316082543 learned_addr learned my addr 172.21.15.100:0/3316082543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd869a24700 1 -- 172.21.15.100:0/3316082543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860005040 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd869a24700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8600096b0 tx=0x7fd860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd86000bed0 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86410d7c0 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd86000b5c0 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd86001c5b0 con 0x7fd8640fb960 2024-03-18T14:43:35.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.682+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86410e330 con 0x7fd8640fb960 2024-03-18T14:43:35.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.683+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd864101e70 con 0x7fd8640fb960 2024-03-18T14:43:35.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.683+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd860013a80 con 0x7fd8640fb960 2024-03-18T14:43:35.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.683+0000 7fd85a7fc700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 0x7fd850038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:35.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.683+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd860049a80 con 0x7fd8640fb960 2024-03-18T14:43:35.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.684+0000 7fd869223700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 0x7fd850038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:35.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.684+0000 7fd869223700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 0x7fd850038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd854009990 tx=0x7fd854006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:35.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.689+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fd860011550 con 0x7fd8640fb960 2024-03-18T14:43:35.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.944+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd86410e570 con 0x7fd8640fb960 2024-03-18T14:43:35.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.944+0000 7fd85a7fc700 1 -- 172.21.15.100:0/3316082543 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd860011370 con 0x7fd8640fb960 2024-03-18T14:43:35.946 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:35.946 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 msgr2=0x7fd850038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 0x7fd850038d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd854009990 tx=0x7fd854006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 msgr2=0x7fd86410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8600096b0 tx=0x7fd860009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 shutdown_connections 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7fd850036880 0x7fd850038d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 --2- 172.21.15.100:0/3316082543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8640fb960 0x7fd86410d080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:35.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 >> 172.21.15.100:0/3316082543 conn(0x7fd8640f70c0 msgr2=0x7fd864065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 shutdown_connections 2024-03-18T14:43:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:35.946+0000 7fd86bc88700 1 -- 172.21.15.100:0/3316082543 wait complete. 2024-03-18T14:43:35.949 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:35 smithi012 ceph-mon[29182]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:37.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:36 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3316082543' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:37.512 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:37.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:37 smithi012 ceph-mon[29182]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- 172.21.15.100:0/718762011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 msgr2=0x7f79380fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 --2- 172.21.15.100:0/718762011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f79380fce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7930009c60 tx=0x7f7930004d10 comp rx=0 tx=0).stop 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- 172.21.15.100:0/718762011 shutdown_connections 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 --2- 172.21.15.100:0/718762011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f79380fce50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- 172.21.15.100:0/718762011 >> 172.21.15.100:0/718762011 conn(0x7f79380f81c0 msgr2=0x7f79380fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- 172.21.15.100:0/718762011 shutdown_connections 2024-03-18T14:43:39.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- 172.21.15.100:0/718762011 wait complete. 2024-03-18T14:43:39.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 Processor -- start 2024-03-18T14:43:39.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.107+0000 7f79403c9700 1 -- start start 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f79403c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f79403c9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793810d5d0 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f793e165700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f793e165700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:58730/0 (socket says 172.21.15.100:58730) 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f793e165700 1 -- 172.21.15.100:0/1318485742 learned_addr learned my addr 172.21.15.100:0/1318485742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f793e165700 1 -- 172.21.15.100:0/1318485742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7930009940 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.108+0000 7f793e165700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7930004e60 tx=0x7f7930004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.109+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7930012410 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.109+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7930012a40 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.109+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793810d7d0 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.109+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f793001ba40 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.109+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793810e340 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7930019040 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7938102f50 con 0x7f79380fca40 2024-03-18T14:43:39.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f792effd700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 0x7f7924038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:39.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7930049ca0 con 0x7f79380fca40 2024-03-18T14:43:39.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f793d964700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 0x7f7924038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:39.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.110+0000 7f793d964700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 0x7f7924038d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7928009990 tx=0x7f7928006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:39.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.115+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f793001b3f0 con 0x7f79380fca40 2024-03-18T14:43:39.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.368+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f793810ed50 con 0x7f79380fca40 2024-03-18T14:43:39.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.369+0000 7f792effd700 1 -- 172.21.15.100:0/1318485742 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7930017090 con 0x7f79380fca40 2024-03-18T14:43:39.370 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:39.370 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:39.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 msgr2=0x7f7924038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 0x7f7924038d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7928009990 tx=0x7f7928006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 msgr2=0x7f793810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7930004e60 tx=0x7f7930004d10 comp rx=0 tx=0).stop 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 shutdown_connections 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f7924036880 0x7f7924038d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 --2- 172.21.15.100:0/1318485742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79380fca40 0x7f793810d090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 >> 172.21.15.100:0/1318485742 conn(0x7f79380f81c0 msgr2=0x7f7938103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 shutdown_connections 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:39.371+0000 7f79403c9700 1 -- 172.21.15.100:0/1318485742 wait complete. 2024-03-18T14:43:39.373 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:39 smithi012 ceph-mon[29182]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:40.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:39 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/1318485742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:40.900 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:40.900 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:41.146 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:42.217 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:41 smithi012 ceph-mon[29182]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 -- 172.21.15.100:0/2108318088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 msgr2=0x7f9a080fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 --2- 172.21.15.100:0/2108318088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a080fd860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99f80048c0 tx=0x7f99f8009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 -- 172.21.15.100:0/2108318088 shutdown_connections 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 --2- 172.21.15.100:0/2108318088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a080fd860 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 -- 172.21.15.100:0/2108318088 >> 172.21.15.100:0/2108318088 conn(0x7f9a080f70d0 msgr2=0x7f9a080f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 -- 172.21.15.100:0/2108318088 shutdown_connections 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.631+0000 7f9a106d5700 1 -- 172.21.15.100:0/2108318088 wait complete. 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a106d5700 1 Processor -- start 2024-03-18T14:43:42.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a106d5700 1 -- start start 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a106d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a106d5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08113c20 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a0e471700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a0e471700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:40974/0 (socket says 172.21.15.100:40974) 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a0e471700 1 -- 172.21.15.100:0/1372724139 learned_addr learned my addr 172.21.15.100:0/1372724139 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.632+0000 7f9a0e471700 1 -- 172.21.15.100:0/1372724139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8005040 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f9a0e471700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009680 tx=0x7f99f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99f800bed0 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a08113e20 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a08114990 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99f800b5c0 con 0x7f9a080fb440 2024-03-18T14:43:42.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.633+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99f80255c0 con 0x7f9a080fb440 2024-03-18T14:43:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.634+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f99f801c3f0 con 0x7f9a080fb440 2024-03-18T14:43:42.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.634+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ec0052e0 con 0x7f9a080fb440 2024-03-18T14:43:42.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.634+0000 7f99feffd700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 0x7f99f4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:42.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.634+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f99f8049970 con 0x7f9a080fb440 2024-03-18T14:43:42.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.634+0000 7f9a0dc70700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 0x7f99f4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:42.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.635+0000 7f9a0dc70700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 0x7f99f4038d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:42.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.639+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f99f801a8e0 con 0x7f9a080fb440 2024-03-18T14:43:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.892+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f99ec005590 con 0x7f9a080fb440 2024-03-18T14:43:42.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.893+0000 7f99feffd700 1 -- 172.21.15.100:0/1372724139 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f99f801a2f0 con 0x7f9a080fb440 2024-03-18T14:43:42.895 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:42.895 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 msgr2=0x7f99f4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 0x7f99f4038d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 msgr2=0x7f9a081136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009680 tx=0x7f99f8009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 shutdown_connections 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f99f40368d0 0x7f99f4038d80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 --2- 172.21.15.100:0/1372724139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080fb440 0x7f9a081136e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 >> 172.21.15.100:0/1372724139 conn(0x7f9a080f70d0 msgr2=0x7f9a0810a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 shutdown_connections 2024-03-18T14:43:42.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:42.895+0000 7f9a106d5700 1 -- 172.21.15.100:0/1372724139 wait complete. 2024-03-18T14:43:42.898 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:44.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:43:44.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/1372724139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:44.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:44.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:44.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:44.105 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:43 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-18T14:43:44.538 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:44.538 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:44.791 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:44.838 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:45.316 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:45 smithi012 ceph-mon[29182]: from='mgr.14162 172.21.15.12:0/2903624181' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-18T14:43:45.316 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:45 smithi012 ceph-mon[29182]: mgrmap e14: smithi012.tmscgt(active, since 73s) 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 -- 172.21.15.100:0/748399899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 msgr2=0x7f4afc0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/748399899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc0fd460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4af80048c0 tx=0x7f4af8009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 -- 172.21.15.100:0/748399899 shutdown_connections 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/748399899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc0fd460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 -- 172.21.15.100:0/748399899 >> 172.21.15.100:0/748399899 conn(0x7f4afc0f70d0 msgr2=0x7f4afc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 -- 172.21.15.100:0/748399899 shutdown_connections 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.299+0000 7f4b03ab4700 1 -- 172.21.15.100:0/748399899 wait complete. 2024-03-18T14:43:46.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b03ab4700 1 Processor -- start 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b03ab4700 1 -- start start 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b03ab4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b03ab4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc10fdc0 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b01850700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b01850700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:40986/0 (socket says 172.21.15.100:40986) 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b01850700 1 -- 172.21.15.100:0/555279378 learned_addr learned my addr 172.21.15.100:0/555279378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.300+0000 7f4b01850700 1 -- 172.21.15.100:0/555279378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8005040 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4b01850700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009680 tx=0x7f4af8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4af800b5c0 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af800be70 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc10ff00 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4af80135a0 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.301+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc1103f0 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f4af801c430 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc105920 con 0x7f4afc0fd050 2024-03-18T14:43:46.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4af27fc700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 0x7f4ae8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:46.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4af80494b0 con 0x7f4afc0fd050 2024-03-18T14:43:46.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4b0104f700 1 -- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 msgr2=0x7f4ae8038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1147707934 2024-03-18T14:43:46.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.302+0000 7f4b0104f700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 0x7f4ae8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:43:46.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.307+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f4af80117e0 con 0x7f4afc0fd050 2024-03-18T14:43:46.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.503+0000 7f4b0104f700 1 -- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 msgr2=0x7f4ae8038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1147707934 2024-03-18T14:43:46.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.503+0000 7f4b0104f700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 0x7f4ae8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:43:46.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.561+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4afc101e70 con 0x7f4afc0fd050 2024-03-18T14:43:46.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.562+0000 7f4af27fc700 1 -- 172.21.15.100:0/555279378 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4af8021070 con 0x7f4afc0fd050 2024-03-18T14:43:46.564 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:46.564 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 msgr2=0x7f4ae8038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 0x7f4ae8038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 msgr2=0x7f4afc065bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009680 tx=0x7f4af8009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 shutdown_connections 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f4ae80368d0 0x7f4ae8038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 --2- 172.21.15.100:0/555279378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4afc0fd050 0x7f4afc065bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 >> 172.21.15.100:0/555279378 conn(0x7f4afc0f70d0 msgr2=0x7f4afc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 shutdown_connections 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:46.564+0000 7f4b03ab4700 1 -- 172.21.15.100:0/555279378 wait complete. 2024-03-18T14:43:46.566 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:46.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:46 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/555279378' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:48.166 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:48.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:48.418 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.847+0000 7f317331d700 1 -- 172.21.15.100:0/828401838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 msgr2=0x7f316c0ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.847+0000 7f317331d700 1 --2- 172.21.15.100:0/828401838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c0ff6c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f31640048c0 tx=0x7f3164009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.847+0000 7f317331d700 1 -- 172.21.15.100:0/828401838 shutdown_connections 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.847+0000 7f317331d700 1 --2- 172.21.15.100:0/828401838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c0ff6c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.847+0000 7f317331d700 1 -- 172.21.15.100:0/828401838 >> 172.21.15.100:0/828401838 conn(0x7f316c0f7090 msgr2=0x7f316c0f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 -- 172.21.15.100:0/828401838 shutdown_connections 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 -- 172.21.15.100:0/828401838 wait complete. 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 Processor -- start 2024-03-18T14:43:49.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 -- start start 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.848+0000 7f317331d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10d5e0 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f31710b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f31710b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47250/0 (socket says 172.21.15.100:47250) 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f31710b9700 1 -- 172.21.15.100:0/3096954359 learned_addr learned my addr 172.21.15.100:0/3096954359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f31710b9700 1 -- 172.21.15.100:0/3096954359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164005040 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f31710b9700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f316400b900 tx=0x7f3164009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f316400b5c0 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f316400be70 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c10d7e0 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f316401c590 con 0x7f316c066940 2024-03-18T14:43:49.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.849+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c10e350 con 0x7f316c066940 2024-03-18T14:43:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f3164013420 con 0x7f316c066940 2024-03-18T14:43:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c102f50 con 0x7f316c066940 2024-03-18T14:43:49.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f3169ffb700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 0x7f3154038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3164049480 con 0x7f316c066940 2024-03-18T14:43:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f31708b8700 1 -- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 msgr2=0x7f3154038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1147707934 2024-03-18T14:43:49.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.850+0000 7f31708b8700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 0x7f3154038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:43:49.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:49.855+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3164011550 con 0x7f316c066940 2024-03-18T14:43:50.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.052+0000 7f31708b8700 1 -- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 msgr2=0x7f3154038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1147707934 2024-03-18T14:43:50.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.052+0000 7f31708b8700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 0x7f3154038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T14:43:50.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.112+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f316c10ed60 con 0x7f316c066940 2024-03-18T14:43:50.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.112+0000 7f3169ffb700 1 -- 172.21.15.100:0/3096954359 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3164021020 con 0x7f316c066940 2024-03-18T14:43:50.114 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:50.114 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 msgr2=0x7f3154038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 0x7f3154038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 msgr2=0x7f316c10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f316400b900 tx=0x7f3164009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 shutdown_connections 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:6800/1147707934,v1:172.21.15.12:6801/1147707934] conn(0x7f31540368d0 0x7f3154038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 --2- 172.21.15.100:0/3096954359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f316c066940 0x7f316c10d0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 >> 172.21.15.100:0/3096954359 conn(0x7f316c0f7090 msgr2=0x7f316c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 shutdown_connections 2024-03-18T14:43:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:50.114+0000 7f317331d700 1 -- 172.21.15.100:0/3096954359 wait complete. 2024-03-18T14:43:50.117 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:50.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:50 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3096954359' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: Active manager daemon smithi012.tmscgt restarted 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: osdmap e5: 0 total, 0 up, 0 in 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: mgrmap e15: smithi012.tmscgt(active, starting, since 0.00293059s) 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:43:51.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:43:51.494 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:43:51.637 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:51.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:51.893 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:51.944 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:52.631 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:52.631 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:52.631 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: [18/Mar/2024:14:43:51] ENGINE Bus STARTING 2024-03-18T14:43:52.631 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: mgrmap e16: smithi012.tmscgt(active, since 1.00468s) 2024-03-18T14:43:52.631 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: [18/Mar/2024:14:43:51] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:43:52.632 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: [18/Mar/2024:14:43:51] ENGINE Bus STARTED 2024-03-18T14:43:52.632 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:52 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 -- 172.21.15.100:0/3179386548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 msgr2=0x7f4e4c0fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 --2- 172.21.15.100:0/3179386548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c0fc1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4e480048c0 tx=0x7f4e48009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 -- 172.21.15.100:0/3179386548 shutdown_connections 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 --2- 172.21.15.100:0/3179386548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c0fc1f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 -- 172.21.15.100:0/3179386548 >> 172.21.15.100:0/3179386548 conn(0x7f4e4c0f6850 msgr2=0x7f4e4c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 -- 172.21.15.100:0/3179386548 shutdown_connections 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.398+0000 7f4e53857700 1 -- 172.21.15.100:0/3179386548 wait complete. 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e53857700 1 Processor -- start 2024-03-18T14:43:53.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e53857700 1 -- start start 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e53857700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e53857700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c113c00 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e515f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e515f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47268/0 (socket says 172.21.15.100:47268) 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e515f3700 1 -- 172.21.15.100:0/4116362390 learned_addr learned my addr 172.21.15.100:0/4116362390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.399+0000 7f4e515f3700 1 -- 172.21.15.100:0/4116362390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48005040 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e515f3700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009680 tx=0x7f4e48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e4800b5c0 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e4800be70 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c113e00 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e480135a0 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.400+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c114970 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.401+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f4e48013810 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.401+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c100d50 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.401+0000 7f4e41ffb700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 0x7f4e38038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:53.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.401+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4e48049660 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.401+0000 7f4e50df2700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 0x7f4e38038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:53.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.402+0000 7f4e50df2700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 0x7f4e38038e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009990 tx=0x7f4e3c006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:53.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.407+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f4e48024b50 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.671+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4e4c115440 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.672+0000 7f4e41ffb700 1 -- 172.21.15.100:0/4116362390 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4e48021020 con 0x7f4e4c0fbde0 2024-03-18T14:43:53.674 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:53.674 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:53.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 msgr2=0x7f4e38038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:53.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 0x7f4e38038e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009990 tx=0x7f4e3c006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 msgr2=0x7f4e4c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009680 tx=0x7f4e48009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 shutdown_connections 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4e380369c0 0x7f4e38038e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 --2- 172.21.15.100:0/4116362390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e4c0fbde0 0x7f4e4c1136c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 >> 172.21.15.100:0/4116362390 conn(0x7f4e4c0f6850 msgr2=0x7f4e4c10aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 shutdown_connections 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:53.674+0000 7f4e53857700 1 -- 172.21.15.100:0/4116362390 wait complete. 2024-03-18T14:43:53.677 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:53 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:53 smithi012 ceph-mon[29182]: mgrmap e17: smithi012.tmscgt(active, since 2s) 2024-03-18T14:43:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:53 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/4116362390' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:43:56.009 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:43:56.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:43:56.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:56.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:56.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:56.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:56.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:43:56.677 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:56.724 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:43:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-18T14:43:57.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 -- 172.21.15.100:0/4013702919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 msgr2=0x7fc9d4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 --2- 172.21.15.100:0/4013702919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4102eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0048c0 tx=0x7fc9cc009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 -- 172.21.15.100:0/4013702919 shutdown_connections 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 --2- 172.21.15.100:0/4013702919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4102eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 -- 172.21.15.100:0/4013702919 >> 172.21.15.100:0/4013702919 conn(0x7fc9d40f85e0 msgr2=0x7fc9d40fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:58.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 -- 172.21.15.100:0/4013702919 shutdown_connections 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.404+0000 7fc9db43b700 1 -- 172.21.15.100:0/4013702919 wait complete. 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9db43b700 1 Processor -- start 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9db43b700 1 -- start start 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9db43b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9db43b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d41158d0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9d91d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9d91d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47286/0 (socket says 172.21.15.100:47286) 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.405+0000 7fc9d91d7700 1 -- 172.21.15.100:0/36299136 learned_addr learned my addr 172.21.15.100:0/36299136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9d91d7700 1 -- 172.21.15.100:0/36299136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc005040 con 0x7fc9d4102ae0 2024-03-18T14:43:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9d91d7700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00b820 tx=0x7fc9cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:58.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc0093c0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9cc013420 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc01c5b0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d4115ad0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.406+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4115fc0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.407+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fc9cc013a90 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.407+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d40fe1e0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.407+0000 7fc9c9ffb700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 0x7fc9c0038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.407+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc9cc049b00 con 0x7fc9d4102ae0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.407+0000 7fc9d89d6700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 0x7fc9c0038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:43:58.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.408+0000 7fc9d89d6700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 0x7fc9c0038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009990 tx=0x7fc9c4006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:43:58.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.412+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc9cc0098a0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.677+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc9d405f5f0 con 0x7fc9d4102ae0 2024-03-18T14:43:58.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.678+0000 7fc9c9ffb700 1 -- 172.21.15.100:0/36299136 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc9cc011370 con 0x7fc9d4102ae0 2024-03-18T14:43:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:43:58.680 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 msgr2=0x7fc9c0038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 0x7fc9c0038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009990 tx=0x7fc9c4006e40 comp rx=0 tx=0).stop 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 msgr2=0x7fc9d4115390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00b820 tx=0x7fc9cc009e30 comp rx=0 tx=0).stop 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 shutdown_connections 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc9c00369c0 0x7fc9c0038e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 --2- 172.21.15.100:0/36299136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9d4102ae0 0x7fc9d4115390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:43:58.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 >> 172.21.15.100:0/36299136 conn(0x7fc9d40f85e0 msgr2=0x7fc9d40713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:43:58.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 shutdown_connections 2024-03-18T14:43:58.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:43:58.680+0000 7fc9db43b700 1 -- 172.21.15.100:0/36299136 wait complete. 2024-03-18T14:43:58.683 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:43:59.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:58 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:43:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:58 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:43:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:58 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:43:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:58 smithi012 ceph-mon[29182]: Deploying daemon crash.smithi100 on smithi100 2024-03-18T14:43:59.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:43:58 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/36299136' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:01.066 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:44:01.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:44:01.317 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:02.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:02.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:02.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:02.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:02.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:02 smithi012 ceph-mon[29182]: Deploying daemon node-exporter.smithi100 on smithi100 2024-03-18T14:44:02.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 -- 172.21.15.100:0/1434756287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 msgr2=0x7f0488101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 --2- 172.21.15.100:0/1434756287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f0488101d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f04780048c0 tx=0x7f0478009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 -- 172.21.15.100:0/1434756287 shutdown_connections 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 --2- 172.21.15.100:0/1434756287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f0488101d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 -- 172.21.15.100:0/1434756287 >> 172.21.15.100:0/1434756287 conn(0x7f04880f74f0 msgr2=0x7f04880f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 -- 172.21.15.100:0/1434756287 shutdown_connections 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.852+0000 7f048d3d8700 1 -- 172.21.15.100:0/1434756287 wait complete. 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f048d3d8700 1 Processor -- start 2024-03-18T14:44:02.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f048d3d8700 1 -- start start 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f048d3d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f048d3d8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048810e640 con 0x7f04881019b0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47558/0 (socket says 172.21.15.100:47558) 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f0486ffd700 1 -- 172.21.15.100:0/3671202034 learned_addr learned my addr 172.21.15.100:0/3671202034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.853+0000 7f0486ffd700 1 -- 172.21.15.100:0/3671202034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478005040 con 0x7f04881019b0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f0486ffd700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f047800b820 tx=0x7f0478009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f04780093c0 con 0x7f04881019b0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048810e840 con 0x7f04881019b0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0478013420 con 0x7f04881019b0 2024-03-18T14:44:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047801c5b0 con 0x7f04881019b0 2024-03-18T14:44:02.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.854+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048810ed30 con 0x7f04881019b0 2024-03-18T14:44:02.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.855+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f047801c7f0 con 0x7f04881019b0 2024-03-18T14:44:02.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.855+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048810ef20 con 0x7f04881019b0 2024-03-18T14:44:02.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.855+0000 7f046f7fe700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 0x7f0470038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:02.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.855+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0478049ad0 con 0x7f04881019b0 2024-03-18T14:44:02.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.855+0000 7f04867fc700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 0x7f0470038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:02.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.856+0000 7f04867fc700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 0x7f0470038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f047c009990 tx=0x7f047c006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:02.860+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0478023300 con 0x7f04881019b0 2024-03-18T14:44:03.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.136+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f048804bed0 con 0x7f04881019b0 2024-03-18T14:44:03.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.137+0000 7f046f7fe700 1 -- 172.21.15.100:0/3671202034 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0478011b60 con 0x7f04881019b0 2024-03-18T14:44:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:44:03.139 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:44:03.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 msgr2=0x7f0470038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 0x7f0470038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f047c009990 tx=0x7f047c006e40 comp rx=0 tx=0).stop 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 msgr2=0x7f048810e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f047800b820 tx=0x7f0478009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 shutdown_connections 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0470036970 0x7f0470038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 --2- 172.21.15.100:0/3671202034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04881019b0 0x7f048810e100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 >> 172.21.15.100:0/3671202034 conn(0x7f04880f74f0 msgr2=0x7f04880f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 shutdown_connections 2024-03-18T14:44:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:03.140+0000 7f048d3d8700 1 -- 172.21.15.100:0/3671202034 wait complete. 2024-03-18T14:44:03.143 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:44:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:03 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3671202034' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:04.723 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:44:04.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:44:05.042 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:06.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.856+0000 7fef04883700 1 -- 172.21.15.100:0/3458948014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 msgr2=0x7fef00102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.856+0000 7fef04883700 1 --2- 172.21.15.100:0/3458948014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef00102e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feef8009c60 tx=0x7feef8004d10 comp rx=0 tx=0).stop 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- 172.21.15.100:0/3458948014 shutdown_connections 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 --2- 172.21.15.100:0/3458948014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef00102e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- 172.21.15.100:0/3458948014 >> 172.21.15.100:0/3458948014 conn(0x7fef000f8560 msgr2=0x7fef000fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- 172.21.15.100:0/3458948014 shutdown_connections 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- 172.21.15.100:0/3458948014 wait complete. 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 Processor -- start 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- start start 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7fef04883700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00113620 con 0x7fef00102a60 2024-03-18T14:44:06.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7feefe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7feefe59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:47580/0 (socket says 172.21.15.100:47580) 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.857+0000 7feefe59c700 1 -- 172.21.15.100:0/3236422016 learned_addr learned my addr 172.21.15.100:0/3236422016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7feefe59c700 1 -- 172.21.15.100:0/3236422016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8009940 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7feefe59c700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feef80096d0 tx=0x7feef8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feef8007c00 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef00113820 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feef801d910 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feef8012900 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.858+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef00113d10 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.859+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000fe160 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.859+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7feef8019040 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.859+0000 7feeeeffd700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 0x7feee8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.859+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7feef804a300 con 0x7fef00102a60 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.859+0000 7feefdd9b700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 0x7feee8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:06.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.860+0000 7feefdd9b700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 0x7feee8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feef4009990 tx=0x7feef4006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:06.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:06.864+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7feef802eb40 con 0x7fef00102a60 2024-03-18T14:44:07.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.125+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef0005f5f0 con 0x7fef00102a60 2024-03-18T14:44:07.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.125+0000 7feeeeffd700 1 -- 172.21.15.100:0/3236422016 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7feef8017090 con 0x7fef00102a60 2024-03-18T14:44:07.127 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:44:07.127 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:44:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 msgr2=0x7feee8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 0x7feee8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feef4009990 tx=0x7feef4006e40 comp rx=0 tx=0).stop 2024-03-18T14:44:07.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 msgr2=0x7fef001130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feef80096d0 tx=0x7feef8004d10 comp rx=0 tx=0).stop 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 shutdown_connections 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feee8036970 0x7feee8038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 --2- 172.21.15.100:0/3236422016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef00102a60 0x7fef001130e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 >> 172.21.15.100:0/3236422016 conn(0x7fef000f8560 msgr2=0x7fef000f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 shutdown_connections 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:07.128+0000 7fef04883700 1 -- 172.21.15.100:0/3236422016 wait complete. 2024-03-18T14:44:07.130 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:44:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:07.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3236422016' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:08.971 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:44:08.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:44:09.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:09.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:09 smithi012 ceph-mon[29182]: Deploying daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:44:09.837 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:11.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 -- 172.21.15.100:0/144990168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 msgr2=0x7f1f04101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:11.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/144990168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04101d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f000048c0 tx=0x7f1f00009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:11.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 -- 172.21.15.100:0/144990168 shutdown_connections 2024-03-18T14:44:11.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/144990168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04101d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 -- 172.21.15.100:0/144990168 >> 172.21.15.100:0/144990168 conn(0x7f1f040f7510 msgr2=0x7f1f040f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 -- 172.21.15.100:0/144990168 shutdown_connections 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.427+0000 7f1f0bb65700 1 -- 172.21.15.100:0/144990168 wait complete. 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f0bb65700 1 Processor -- start 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f0bb65700 1 -- start start 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f0bb65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f0bb65700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04107390 con 0x7f1f041019b0 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f09901700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f09901700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:46136/0 (socket says 172.21.15.100:46136) 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.428+0000 7f1f09901700 1 -- 172.21.15.100:0/60725440 learned_addr learned my addr 172.21.15.100:0/60725440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1f09901700 1 -- 172.21.15.100:0/60725440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00005040 con 0x7f1f041019b0 2024-03-18T14:44:11.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1f09901700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b820 tx=0x7f1f00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f000093c0 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f04107590 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f00013420 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0001c5b0 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.429+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f04107a80 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.430+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04107cc0 con 0x7f1f041019b0 2024-03-18T14:44:11.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.430+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1f00009520 con 0x7f1f041019b0 2024-03-18T14:44:11.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.430+0000 7f1efa7fc700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 0x7f1ef0038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:11.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.430+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1f00049a00 con 0x7f1f041019b0 2024-03-18T14:44:11.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.430+0000 7f1f09100700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 0x7f1ef0038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:11.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.431+0000 7f1f09100700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 0x7f1ef0038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009990 tx=0x7f1ef4006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:11.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.435+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f1f04107cc0 con 0x7f1f041019b0 2024-03-18T14:44:11.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.713+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f04107cc0 con 0x7f1f041019b0 2024-03-18T14:44:11.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.714+0000 7f1efa7fc700 1 -- 172.21.15.100:0/60725440 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1f04107cc0 con 0x7f1f041019b0 2024-03-18T14:44:11.716 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:44:11.716 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:41:22.518337Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 msgr2=0x7f1ef0038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 0x7f1ef0038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009990 tx=0x7f1ef4006e40 comp rx=0 tx=0).stop 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 msgr2=0x7f1f04106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b820 tx=0x7f1f00009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 shutdown_connections 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1ef00369c0 0x7f1ef0038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 --2- 172.21.15.100:0/60725440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f041019b0 0x7f1f04106e50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 >> 172.21.15.100:0/60725440 conn(0x7f1f040f7510 msgr2=0x7f1f041029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 shutdown_connections 2024-03-18T14:44:11.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:11.716+0000 7f1f0bb65700 1 -- 172.21.15.100:0/60725440 wait complete. 2024-03-18T14:44:11.719 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 1 2024-03-18T14:44:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:11 smithi012 ceph-mon[29182]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:11 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:12.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:11 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/60725440' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:13.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:13 smithi012 ceph-mon[29182]: Deploying daemon mon.smithi100 on smithi100 2024-03-18T14:44:13.596 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-18T14:44:13.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T14:44:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:14 smithi012 ceph-mon[29182]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:15.049 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:44:16.306 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:16 smithi012 ceph-mon[29182]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:16.306 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:16 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:16.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:16 smithi100 podman[37754]: 2024-03-18 14:44:16.709592405 +0000 UTC m=+0.305314747 container create ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, org.label-schema.schema-version=1.0, RELEASE=quincy-6716c9f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T14:44:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 -- 172.21.15.100:0/1643951754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 msgr2=0x7f9c7c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/1643951754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c102f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c700048c0 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 -- 172.21.15.100:0/1643951754 shutdown_connections 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/1643951754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c102f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 -- 172.21.15.100:0/1643951754 >> 172.21.15.100:0/1643951754 conn(0x7f9c7c0f86f0 msgr2=0x7f9c7c0fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 -- 172.21.15.100:0/1643951754 shutdown_connections 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.054+0000 7f9c82ea3700 1 -- 172.21.15.100:0/1643951754 wait complete. 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c82ea3700 1 Processor -- start 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c82ea3700 1 -- start start 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c82ea3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c82ea3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c070800 con 0x7f9c7c102ba0 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c80c3f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c80c3f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.100:46162/0 (socket says 172.21.15.100:46162) 2024-03-18T14:44:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.055+0000 7f9c80c3f700 1 -- 172.21.15.100:0/2573129478 learned_addr learned my addr 172.21.15.100:0/2573129478 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c80c3f700 1 -- 172.21.15.100:0/2573129478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70005040 con 0x7f9c7c102ba0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c80c3f700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b820 tx=0x7f9c70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c700093c0 con 0x7f9c7c102ba0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c070a00 con 0x7f9c7c102ba0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c70013420 con 0x7f9c7c102ba0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c7001c5b0 con 0x7f9c7c102ba0 2024-03-18T14:44:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.056+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c070ef0 con 0x7f9c7c102ba0 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9c7001c7f0 con 0x7f9c7c102ba0 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7c071130 con 0x7f9c7c102ba0 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c797fa700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 0x7f9c6c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9c70049b40 con 0x7f9c7c102ba0 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c7bfff700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 0x7f9c6c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:17.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.057+0000 7f9c7bfff700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 0x7f9c6c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009990 tx=0x7f9c68006e40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.062+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f9c7c071130 con 0x7f9c7c102ba0 2024-03-18T14:44:17.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.195+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7004e020 con 0x7f9c7c102ba0 2024-03-18T14:44:17.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:17.344+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9c7c071130 con 0x7f9c7c102ba0 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 podman[37754]: 2024-03-18 14:44:17.138422338 +0000 UTC m=+0.734144708 container init ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, RELEASE=quincy-6716c9f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 podman[37754]: 2024-03-18 14:44:17.142071662 +0000 UTC m=+0.737794017 container start ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=quincy-6716c9f, org.label-schema.build-date=20240307) 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable), process ceph-mon, pid 2 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: pidfile_write: ignore empty --pid-file 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: load: jerasure load: lrc 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: RocksDB version: 6.15.5 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Compile date Mar 16 2024 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: DB SUMMARY 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: DB Session ID: TWU4ZWSSBROXK6JKYVOT 2024-03-18T14:44:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: CURRENT file: CURRENT 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi100/store.db dir, Total Num: 0, files: 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi100/store.db: 000004.log size: 511 ; 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.error_if_exists: 0 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.create_if_missing: 0 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.paranoid_checks: 1 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.env: 0x562cba953a80 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.fs: Posix File System 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.info_log: 0x562cbbe81c00 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.statistics: (nil) 2024-03-18T14:44:17.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.use_fsync: 0 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_log_file_size: 0 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_fallocate: 1 2024-03-18T14:44:17.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.use_direct_reads: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.db_log_dir: 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi100/store.db 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T14:44:17.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.write_buffer_manager: 0x562cbcbf6ab0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.unordered_write: 0 2024-03-18T14:44:17.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.row_cache: None 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.wal_filter: None 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.preserve_deletes: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.two_write_queues: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.atomic_flush: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T14:44:17.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.log_readahead_size: 0 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_background_jobs: 2 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_background_compactions: -1 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_subcompactions: 1 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T14:44:17.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_open_files: -1 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_background_flushes: -1 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Compression algorithms supported: 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kZSTD supported: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kXpressCompression supported: 0 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kZlibCompression supported: 1 2024-03-18T14:44:17.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kSnappyCompression supported: 1 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kLZ4Compression supported: 1 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: kBZip2Compression supported: 0 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi100/store.db/MANIFEST-000003 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.merge_operator: 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_filter: None 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562cbbed6f58) 2024-03-18T14:44:17.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_type: 0 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: data_block_index_type: 0 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_shortening: 1 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: hash_index_allow_collision: 1 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: checksum: 1 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: no_block_cache: 0 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache: 0x562cbbf09090 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_options: 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: capacity : 536870912 2024-03-18T14:44:17.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: num_shard_bits : 4 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: strict_capacity_limit : 0 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_compressed: (nil) 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: persistent_cache: (nil) 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_size: 4096 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_size_deviation: 10 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_restart_interval: 16 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_block_restart_interval: 1 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: metadata_block_size: 4096 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: partition_filters: 0 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: use_delta_encoding: 1 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: whole_key_filtering: 1 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: verify_compression: 0 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-03-18T14:44:17.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: format_version: 4 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: enable_index_compression: 1 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_align: 0 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression: NoCompression 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.num_levels: 7 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T14:44:17.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T14:44:17.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T14:44:17.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.arena_block_size: 4194304 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T14:44:17.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.table_properties_collectors: 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.inplace_update_support: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.bloom_locality: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.max_successive_merges: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.ttl: 2592000 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.enable_blob_files: false 2024-03-18T14:44:17.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.min_blob_size: 0 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi100/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773057168256, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773057168562, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1710773057, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "013bb95c-ee2d-4e09-98c4-2b340d1477e0", "db_session_id": "TWU4ZWSSBROXK6JKYVOT"}} 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773057168867, "job": 1, "event": "recovery_finished"} 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi100/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T14:44:17.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x562cbbe9d180 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: DB pointer 0x562cbbf72000 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** DB Stats ** 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:44:17.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:44:17.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100 does not exist in monmap, will attempt to join an existing cluster 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: using public_addr v2:172.21.15.100:0/0 -> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: starting mon.smithi100 rank -1 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi100 fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(???) e0 preinit fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).mds e1 new map 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).mds e1 print_map 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: e1 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: legacy client fscid: -1 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: No filesystems configured 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T14:44:17.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: Deploying daemon mon.smithi100 on smithi100 2024-03-18T14:44:17.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:17.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:17.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:17.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 ceph-mon[37807]: mon.smithi100@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-18T14:44:17.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 bash[37754]: ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 2024-03-18T14:44:17.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:17 smithi100 systemd[1]: Started Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:44:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-18T14:44:22.247 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","modified":"2024-03-18T14:44:17.192582Z","created":"2024-03-18T14:41:22.518337Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi012","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi100","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T14:44:22.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.244+0000 7f9c797fa700 1 -- 172.21.15.100:0/2573129478 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f9c7c071130 con 0x7f9c7c102ba0 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 msgr2=0x7f9c6c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 0x7f9c6c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009990 tx=0x7f9c68006e40 comp rx=0 tx=0).stop 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 msgr2=0x7f9c7c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b820 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 shutdown_connections 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f9c6c036970 0x7f9c6c038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 --2- 172.21.15.100:0/2573129478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c7c102ba0 0x7f9c7c0702c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 >> 172.21.15.100:0/2573129478 conn(0x7f9c7c0f86f0 msgr2=0x7f9c7c0f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:22.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 shutdown_connections 2024-03-18T14:44:22.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:44:22.247+0000 7f9c82ea3700 1 -- 172.21.15.100:0/2573129478 wait complete. 2024-03-18T14:44:22.250 INFO:teuthology.orchestra.run.smithi100.stderr:dumped monmap epoch 2 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: mon.smithi012 calling monitor election 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/2573129478' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: mon.smithi100 calling monitor election 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: monmap e2: 2 mons at {smithi012=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],smithi100=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: fsmap 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: osdmap e5: 0 total, 0 up, 0 in 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: mgrmap e17: smithi012.tmscgt(active, since 31s) 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: overall HEALTH_OK 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: mon.smithi012 calling monitor election 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2573129478' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: mon.smithi100 calling monitor election 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: monmap e2: 2 mons at {smithi012=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],smithi100=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: fsmap 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: osdmap e5: 0 total, 0 up, 0 in 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: mgrmap e17: smithi012.tmscgt(active, since 31s) 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: overall HEALTH_OK 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/1929530608' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:44:22.744 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.745 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.745 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:22.848 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-18T14:44:22.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-18T14:44:23.118 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:44:23.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:23.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: mgrmap e18: smithi012.tmscgt(active, since 31s), standbys: smithi100.rwnrcn 2024-03-18T14:44:23.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:44:23.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:23.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:44:23.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:23.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: mgrmap e18: smithi012.tmscgt(active, since 31s), standbys: smithi100.rwnrcn 2024-03-18T14:44:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:44:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:44:23.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:23.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:44:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 -- 172.21.15.12:0/2430656385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601019d0 msgr2=0x7f3760101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 --2- 172.21.15.12:0/2430656385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601019d0 0x7f3760101da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f375c0048c0 tx=0x7f375c009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 -- 172.21.15.12:0/2430656385 shutdown_connections 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 --2- 172.21.15.12:0/2430656385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601019d0 0x7f3760101da0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 -- 172.21.15.12:0/2430656385 >> 172.21.15.12:0/2430656385 conn(0x7f3760071800 msgr2=0x7f3760071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 -- 172.21.15.12:0/2430656385 shutdown_connections 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.744+0000 7f3767a41700 1 -- 172.21.15.12:0/2430656385 wait complete. 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 Processor -- start 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 -- start start 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760113dc0 con 0x7f37601137b0 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3767a41700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376010d710 con 0x7f37601019d0 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47854/0 (socket says 172.21.15.12:47854) 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 -- 172.21.15.12:0/4055516757 learned_addr learned my addr 172.21.15.12:0/4055516757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f37657dd700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f37657dd700 1 -- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 msgr2=0x7f3760113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f37657dd700 1 -- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 msgr2=0x7f3760113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f37657dd700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f37657dd700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 -- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 msgr2=0x7f3760113270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.745+0000 7f3764fdc700 1 -- 172.21.15.12:0/4055516757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c005040 con 0x7f37601137b0 2024-03-18T14:44:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3764fdc700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f375000b790 tx=0x7f3750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750010870 con 0x7f37601137b0 2024-03-18T14:44:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376010d9f0 con 0x7f37601137b0 2024-03-18T14:44:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f375000f3e0 con 0x7f37601137b0 2024-03-18T14:44:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750010ea0 con 0x7f37601137b0 2024-03-18T14:44:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.746+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376010dfb0 con 0x7f37601137b0 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.747+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37600fd0d0 con 0x7f37601137b0 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.747+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f375000f550 con 0x7f37601137b0 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.748+0000 7f3755ffb700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 0x7f374c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.748+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f37500865d0 con 0x7f37601137b0 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.748+0000 7f37657dd700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 0x7f374c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.748+0000 7f37657dd700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 0x7f374c06b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f375c0043d0 tx=0x7f375c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:24.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.752+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3750057470 con 0x7f37601137b0 2024-03-18T14:44:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.959+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f376005f5f0 con 0x7f37601137b0 2024-03-18T14:44:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.960+0000 7f3755ffb700 1 -- 172.21.15.12:0/4055516757 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f3750014100 con 0x7f37601137b0 2024-03-18T14:44:24.961 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:44:24.961 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-03-18T14:44:24.962 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:44:24.962 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-03-18T14:44:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 msgr2=0x7f374c06b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 0x7f374c06b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f375c0043d0 tx=0x7f375c009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 msgr2=0x7f376010d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f375000b790 tx=0x7f3750009250 comp rx=0 tx=0).stop 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 shutdown_connections 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f374c069370 0x7f374c06b820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37601019d0 0x7f3760113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 --2- 172.21.15.12:0/4055516757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37601137b0 0x7f376010d1d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 >> 172.21.15.12:0/4055516757 conn(0x7f3760071800 msgr2=0x7f37600652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 shutdown_connections 2024-03-18T14:44:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:24.962+0000 7f3767a41700 1 -- 172.21.15.12:0/4055516757 wait complete. 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:25.537 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Reconfiguring mon.smithi012 (unknown last config time)... 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: Reconfiguring daemon mon.smithi012 on smithi012 2024-03-18T14:44:25.538 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:25 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4055516757' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.539 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-18T14:44:25.539 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:44:25.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T14:44:25.570 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:44:25.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:44:25.636 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:44:25.636 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T14:44:25.662 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.662 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.662 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.662 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:44:25.662 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Reconfiguring mon.smithi012 (unknown last config time)... 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.663 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: Reconfiguring daemon mon.smithi012 on smithi012 2024-03-18T14:44:25.664 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:25 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4055516757' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:25.667 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:44:25.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:44:25.735 INFO:tasks.cephadm:Deploying OSDs... 2024-03-18T14:44:25.735 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:44:25.735 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T14:44:25.750 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T14:44:25.750 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-03-18 14:43:42.333009273 +0000 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-03-18 14:37:15.330909801 +0000 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-03-18 14:37:15.330909801 +0000 2024-03-18T14:44:25.806 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-03-18 14:37:15.330909801 +0000 2024-03-18T14:44:25.807 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T14:44:25.873 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-03-18T14:44:25.873 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-03-18T14:44:25.873 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000308966 s, 1.7 MB/s 2024-03-18T14:44:25.874 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T14:44:25.930 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-03-18T14:44:25.984 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-03-18 14:43:42.334009264 +0000 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-03-18 14:37:15.719906262 +0000 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-03-18 14:37:15.719906262 +0000 2024-03-18T14:44:25.985 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-03-18 14:37:15.719906262 +0000 2024-03-18T14:44:25.985 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T14:44:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-03-18T14:44:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-03-18T14:44:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000181021 s, 2.8 MB/s 2024-03-18T14:44:26.051 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T14:44:26.106 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-03-18T14:44:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T14:44:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:26.160 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-18T14:44:26.160 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:26.161 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:26.161 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-03-18 14:43:42.334009264 +0000 2024-03-18T14:44:26.161 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-03-18 14:37:16.077903005 +0000 2024-03-18T14:44:26.161 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-03-18 14:37:16.077903005 +0000 2024-03-18T14:44:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-03-18 14:37:16.077903005 +0000 2024-03-18T14:44:26.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T14:44:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-03-18T14:44:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-03-18T14:44:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000175802 s, 2.9 MB/s 2024-03-18T14:44:26.227 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T14:44:26.283 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-03-18T14:44:26.337 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-03-18 14:43:42.334009264 +0000 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-03-18 14:37:16.444899665 +0000 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-03-18 14:37:16.444899665 +0000 2024-03-18T14:44:26.338 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-03-18 14:37:16.444899665 +0000 2024-03-18T14:44:26.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T14:44:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-03-18T14:44:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-03-18T14:44:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000167093 s, 3.1 MB/s 2024-03-18T14:44:26.403 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T14:44:26.458 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:44:26.458 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T14:44:26.477 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T14:44:26.477 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-18 14:44:15.368363794 +0000 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-18 14:37:11.541672220 +0000 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-18 14:37:11.541672220 +0000 2024-03-18T14:44:26.535 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-18 14:37:11.541672220 +0000 2024-03-18T14:44:26.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T14:44:26.604 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-18T14:44:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-18T14:44:26.605 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000235358 s, 2.2 MB/s 2024-03-18T14:44:26.606 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T14:44:26.664 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-18 14:44:15.369363767 +0000 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-18 14:37:11.897662069 +0000 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-18 14:37:11.897662069 +0000 2024-03-18T14:44:26.721 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-18 14:37:11.897662069 +0000 2024-03-18T14:44:26.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T14:44:26.788 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-18T14:44:26.788 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-18T14:44:26.788 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000237835 s, 2.2 MB/s 2024-03-18T14:44:26.789 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T14:44:26.846 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-18 14:44:15.369363767 +0000 2024-03-18T14:44:26.902 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-18 14:37:12.250652003 +0000 2024-03-18T14:44:26.903 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-18 14:37:12.250652003 +0000 2024-03-18T14:44:26.903 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-18 14:37:12.250652003 +0000 2024-03-18T14:44:26.903 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T14:44:26.966 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.966 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.966 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.966 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:44:26.966 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:44:26.967 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:26.968 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-18T14:44:26.969 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-18T14:44:26.969 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000216489 s, 2.4 MB/s 2024-03-18T14:44:26.970 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T14:44:26.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:26.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:44:26.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:44:26.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:27.026 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-03-18T14:44:27.081 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-18 14:44:15.369363767 +0000 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-18 14:37:12.807636160 +0000 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-18 14:37:12.807636160 +0000 2024-03-18T14:44:27.082 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-18 14:37:12.807636160 +0000 2024-03-18T14:44:27.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T14:44:27.148 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-18T14:44:27.148 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-18T14:44:27.148 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000206259 s, 2.5 MB/s 2024-03-18T14:44:27.149 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T14:44:27.205 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-03-18T14:44:27.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T14:44:28.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:27 smithi012 ceph-mon[29182]: Reconfiguring mgr.smithi012.tmscgt (unknown last config time)... 2024-03-18T14:44:28.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:27 smithi012 ceph-mon[29182]: Reconfiguring daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:44:28.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:27 smithi012 ceph-mon[29182]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:28.113 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:44:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:27 smithi100 ceph-mon[37807]: Reconfiguring mgr.smithi012.tmscgt (unknown last config time)... 2024-03-18T14:44:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:27 smithi100 ceph-mon[37807]: Reconfiguring daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:44:28.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:27 smithi100 ceph-mon[37807]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:29.394 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: Reconfiguring crash.smithi012 (monmap changed)... 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: Reconfiguring daemon crash.smithi012 on smithi012 2024-03-18T14:44:29.395 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:29 smithi012 ceph-mon[29182]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: Reconfiguring crash.smithi012 (monmap changed)... 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: Reconfiguring daemon crash.smithi012 on smithi012 2024-03-18T14:44:29.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:29 smithi100 ceph-mon[37807]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115899 s, 905 MB/s 2024-03-18T14:44:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-03-18T14:44:31.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:31 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:31.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:31 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:31 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:31 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:32.229 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:32 smithi012 ceph-mon[29182]: Reconfiguring alertmanager.smithi012 (dependencies changed)... 2024-03-18T14:44:32.230 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:32 smithi012 ceph-mon[29182]: Reconfiguring daemon alertmanager.smithi012 on smithi012 2024-03-18T14:44:32.230 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:32 smithi012 ceph-mon[29182]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:32.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:32 smithi100 ceph-mon[37807]: Reconfiguring alertmanager.smithi012 (dependencies changed)... 2024-03-18T14:44:32.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:32 smithi100 ceph-mon[37807]: Reconfiguring daemon alertmanager.smithi012 on smithi012 2024-03-18T14:44:32.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:32 smithi100 ceph-mon[37807]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:32.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-03-18T14:44:33.259 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:44:34.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:34 smithi012 ceph-mon[29182]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:34 smithi100 ceph-mon[37807]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 -- 172.21.15.12:0/1069018100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fee60 msgr2=0x7f77680fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 --2- 172.21.15.12:0/1069018100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fee60 0x7f77680fafa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f775c004940 tx=0x7f775c009e30 comp rx=0 tx=0).stop 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 -- 172.21.15.12:0/1069018100 shutdown_connections 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 --2- 172.21.15.12:0/1069018100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fee60 0x7f77680fafa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 --2- 172.21.15.12:0/1069018100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fe510 0x7f77680fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 -- 172.21.15.12:0/1069018100 >> 172.21.15.12:0/1069018100 conn(0x7f77680f6c10 msgr2=0x7f77680f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 -- 172.21.15.12:0/1069018100 shutdown_connections 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.264+0000 7f776e0bf700 1 -- 172.21.15.12:0/1069018100 wait complete. 2024-03-18T14:44:35.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 Processor -- start 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 -- start start 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 0x7f776810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776810e500 con 0x7f77680fe510 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f776e0bf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7768112ea0 con 0x7f77680fee60 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f77677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f77677fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34734/0 (socket says 172.21.15.12:34734) 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.265+0000 7f77677fe700 1 -- 172.21.15.12:0/701731158 learned_addr learned my addr 172.21.15.12:0/701731158 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f7766ffd700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 0x7f776810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f77677fe700 1 -- 172.21.15.12:0/701731158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 msgr2=0x7f776810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f77677fe700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 0x7f776810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f77677fe700 1 -- 172.21.15.12:0/701731158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c005040 con 0x7f77680fe510 2024-03-18T14:44:35.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f7766ffd700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 0x7f776810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f77677fe700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f775800b790 tx=0x7f7758009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775800f770 con 0x7f77680fe510 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7768113100 con 0x7f77680fe510 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f775800fda0 con 0x7f77680fe510 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775800d310 con 0x7f77680fe510 2024-03-18T14:44:35.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.266+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7768113670 con 0x7f77680fe510 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.267+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776804c710 con 0x7f77680fe510 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.268+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f7758010420 con 0x7f77680fe510 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.268+0000 7f774ffff700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 0x7f775006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.268+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7758085fc0 con 0x7f77680fe510 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.268+0000 7f7766ffd700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 0x7f775006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:44:35.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.268+0000 7f7766ffd700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 0x7f775006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f775c00b7a0 tx=0x7f775c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:44:35.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.273+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f7758056f00 con 0x7f77680fe510 2024-03-18T14:44:35.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:44:35.482+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7768102590 con 0x7f7750069210 2024-03-18T14:44:36.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:36 smithi012 ceph-mon[29182]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:36.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:44:36.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:44:36.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:36.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:36.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:36 smithi100 ceph-mon[37807]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:36.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:44:36.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:44:36.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:36.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:37.400 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:37 smithi012 ceph-mon[29182]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:44:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:37 smithi100 ceph-mon[37807]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:44:38.225 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:38 smithi012 ceph-mon[29182]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:38.225 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:38 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:38.225 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:38 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:38.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:38 smithi100 ceph-mon[37807]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:38.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:38 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:38.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:38 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:39.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:39 smithi012 ceph-mon[29182]: Reconfiguring grafana.smithi012 (dependencies changed)... 2024-03-18T14:44:39.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:39 smithi012 ceph-mon[29182]: Reconfiguring daemon grafana.smithi012 on smithi012 2024-03-18T14:44:39.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:39 smithi100 ceph-mon[37807]: Reconfiguring grafana.smithi012 (dependencies changed)... 2024-03-18T14:44:39.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:39 smithi100 ceph-mon[37807]: Reconfiguring daemon grafana.smithi012 on smithi012 2024-03-18T14:44:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/884245048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83bef24b-6305-4fac-971a-7fe5250379b2"}]: dispatch 2024-03-18T14:44:40.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/884245048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83bef24b-6305-4fac-971a-7fe5250379b2"}]': finished 2024-03-18T14:44:40.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T14:44:40.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:44:40.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:40 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4272092028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/884245048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83bef24b-6305-4fac-971a-7fe5250379b2"}]: dispatch 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/884245048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83bef24b-6305-4fac-971a-7fe5250379b2"}]': finished 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:44:40.447 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:40 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4272092028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:44:41.463 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:41 smithi100 ceph-mon[37807]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:41.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:41 smithi012 ceph-mon[29182]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:44.119 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:43 smithi012 ceph-mon[29182]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:43 smithi100 ceph-mon[37807]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:45.897 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:45 smithi012 ceph-mon[29182]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:45 smithi100 ceph-mon[37807]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:48.199 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:47 smithi012 ceph-mon[29182]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:47 smithi100 ceph-mon[37807]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:49.734 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:49.734 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:49.735 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:49 smithi012 ceph-mon[29182]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:44:49.735 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:49 smithi012 ceph-mon[29182]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:44:49.735 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:49 smithi012 ceph-mon[29182]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:50.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:50.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:50.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:49 smithi100 ceph-mon[37807]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:44:50.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:49 smithi100 ceph-mon[37807]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:44:50.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:49 smithi100 ceph-mon[37807]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:50.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:44:50.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:50.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:50 smithi012 ceph-mon[29182]: Deploying daemon osd.0 on smithi012 2024-03-18T14:44:51.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:44:51.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:51.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:50 smithi100 ceph-mon[37807]: Deploying daemon osd.0 on smithi012 2024-03-18T14:44:51.670 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:51 smithi012 ceph-mon[29182]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:51.671 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:51 smithi100 ceph-mon[37807]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:44:53.959 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:53 smithi012 ceph-mon[29182]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:53 smithi100 ceph-mon[37807]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: Reconfiguring crash.smithi100 (monmap changed)... 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: Reconfiguring daemon crash.smithi100 on smithi100 2024-03-18T14:44:55.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:55 smithi012 ceph-mon[29182]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:55.734 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: Reconfiguring crash.smithi100 (monmap changed)... 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: Reconfiguring daemon crash.smithi100 on smithi100 2024-03-18T14:44:55.735 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:55 smithi100 ceph-mon[37807]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:56.715 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:56.715 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:56.716 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: Reconfiguring mgr.smithi100.rwnrcn (monmap changed)... 2024-03-18T14:44:56.716 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:44:56.716 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:44:56.716 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:56.716 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:56 smithi012 ceph-mon[29182]: Reconfiguring daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:44:57.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: Reconfiguring mgr.smithi100.rwnrcn (monmap changed)... 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:57.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:56 smithi100 ceph-mon[37807]: Reconfiguring daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: Reconfiguring mon.smithi100 (monmap changed)... 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:57.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:57 smithi012 ceph-mon[29182]: Reconfiguring daemon mon.smithi100 on smithi100 2024-03-18T14:44:58.092 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:44:58.092 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:58.092 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:58.092 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: Reconfiguring mon.smithi100 (monmap changed)... 2024-03-18T14:44:58.092 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:44:58.093 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:44:58.093 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:44:58.093 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:57 smithi100 ceph-mon[37807]: Reconfiguring daemon mon.smithi100 on smithi100 2024-03-18T14:44:59.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi012.front.sepia.ceph.com:9093"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi012.front.sepia.ceph.com:3000"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi012.front.sepia.ceph.com:9095"}]: dispatch 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:44:59 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:44:59.505 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.505 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.505 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi012.front.sepia.ceph.com:9093"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi012.front.sepia.ceph.com:3000"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi012.front.sepia.ceph.com:9095"}]: dispatch 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:44:59.506 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:44:59 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi012.front.sepia.ceph.com:9093"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi012.front.sepia.ceph.com:3000"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi012.front.sepia.ceph.com:9095"}]: dispatch 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.408 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:45:00.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi012.front.sepia.ceph.com:9093"}]: dispatch 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi012.front.sepia.ceph.com:3000"}]: dispatch 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi012.front.sepia.ceph.com:9095"}]: dispatch 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:00.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:01.431 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:01 smithi012 ceph-mon[29182]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:01.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:01 smithi100 ceph-mon[37807]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:03.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.606+0000 7f774ffff700 1 -- 172.21.15.12:0/701731158 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7768102590 con 0x7f7750069210 2024-03-18T14:45:03.608 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-03-18T14:45:03.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 msgr2=0x7f775006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:03.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 0x7f775006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f775c00b7a0 tx=0x7f775c009e30 comp rx=0 tx=0).stop 2024-03-18T14:45:03.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 msgr2=0x7f776810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f775800b790 tx=0x7f7758009250 comp rx=0 tx=0).stop 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 shutdown_connections 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f7750069210 0x7f775006b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77680fe510 0x7f776810d930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 --2- 172.21.15.12:0/701731158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77680fee60 0x7f776810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 >> 172.21.15.12:0/701731158 conn(0x7f77680f6c10 msgr2=0x7f77680f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 shutdown_connections 2024-03-18T14:45:03.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:03.610+0000 7f776e0bf700 1 -- 172.21.15.12:0/701731158 wait complete. 2024-03-18T14:45:03.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:03 smithi012 ceph-mon[29182]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:03.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:03 smithi012 ceph-mon[29182]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T14:45:03.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:03.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:03 smithi100 ceph-mon[37807]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:03 smithi100 ceph-mon[37807]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T14:45:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:04.408 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0.service 2024-03-18T14:45:04.411 INFO:tasks.cephadm:Deploying osd.1 on smithi012 with /dev/vg_nvme/lv_3... 2024-03-18T14:45:04.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T14:45:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:45:05.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-mon[29182]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T14:45:05.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-mon[29182]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T14:45:05.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:05.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-mon[29182]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:45:05.151 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[47480]: 2024-03-18T14:45:04.836+0000 7f22d48aa700 -1 osd.0 0 waiting for initial osdmap 2024-03-18T14:45:05.151 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:45:04 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[47480]: 2024-03-18T14:45:04.840+0000 7f22cdea2700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:45:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:04 smithi100 ceph-mon[37807]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T14:45:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:04 smithi100 ceph-mon[37807]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T14:45:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:05.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:04 smithi100 ceph-mon[37807]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:45:06.056 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:06.056 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:45:06.056 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: osdmap e8: 1 total, 0 up, 1 in 2024-03-18T14:45:06.057 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.057 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.057 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:06.057 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: from='osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: osdmap e8: 1 total, 0 up, 1 in 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T14:45:06.932 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T14:45:06.933 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-03-18T14:45:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-03-18T14:45:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124434 s, 843 MB/s 2024-03-18T14:45:06.933 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-03-18T14:45:07.013 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:06 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:45:07.013 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:06 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:45:07.013 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:06 smithi012 ceph-mon[29182]: osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] boot 2024-03-18T14:45:07.013 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:06 smithi012 ceph-mon[29182]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T14:45:07.013 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:06 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:45:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:06 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:45:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:06 smithi100 ceph-mon[37807]: osd.0 [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] boot 2024-03-18T14:45:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:06 smithi100 ceph-mon[37807]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T14:45:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:45:07.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:07 smithi012 ceph-mon[29182]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:07.867 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:07 smithi012 ceph-mon[29182]: osdmap e10: 1 total, 1 up, 1 in 2024-03-18T14:45:07.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:07.868 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:07.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-03-18T14:45:08.155 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:45:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:07 smithi100 ceph-mon[37807]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T14:45:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:07 smithi100 ceph-mon[37807]: osdmap e10: 1 total, 1 up, 1 in 2024-03-18T14:45:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:09.979 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:09 smithi012 ceph-mon[29182]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:09 smithi100 ceph-mon[37807]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- 172.21.15.12:0/2373621214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368102360 msgr2=0x7f3368104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2373621214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368102360 0x7f3368104740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f335c004940 tx=0x7f335c009e30 comp rx=0 tx=0).stop 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- 172.21.15.12:0/2373621214 shutdown_connections 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2373621214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368102360 0x7f3368104740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2373621214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3368066f10 0x7f3368101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- 172.21.15.12:0/2373621214 >> 172.21.15.12:0/2373621214 conn(0x7f33680f7430 msgr2=0x7f33680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- 172.21.15.12:0/2373621214 shutdown_connections 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- 172.21.15.12:0/2373621214 wait complete. 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 Processor -- start 2024-03-18T14:45:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 -- start start 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.305+0000 7f336fe2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336fe2d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3368102360 0x7f33681100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336fe2d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368110740 con 0x7f3368066f10 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336fe2d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681150e0 con 0x7f3368102360 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51070/0 (socket says 172.21.15.12:51070) 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 -- 172.21.15.12:0/2115829123 learned_addr learned my addr 172.21.15.12:0/2115829123 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 -- 172.21.15.12:0/2115829123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3368102360 msgr2=0x7f33681100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3368102360 0x7f33681100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 -- 172.21.15.12:0/2115829123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c005040 con 0x7f3368066f10 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336dbc9700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f336000b790 tx=0x7f3360009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:45:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f336000f6f0 con 0x7f3368066f10 2024-03-18T14:45:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f336000fd20 con 0x7f3368066f10 2024-03-18T14:45:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33600103a0 con 0x7f3368066f10 2024-03-18T14:45:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3368115340 con 0x7f3368066f10 2024-03-18T14:45:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.306+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33681158b0 con 0x7f3368066f10 2024-03-18T14:45:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.307+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336804c710 con 0x7f3368066f10 2024-03-18T14:45:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.308+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f3360011400 con 0x7f3368066f10 2024-03-18T14:45:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.308+0000 7f33667fc700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 0x7f335006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.308+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f3360086180 con 0x7f3368066f10 2024-03-18T14:45:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.308+0000 7f336d3c8700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 0x7f335006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:45:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.309+0000 7f336d3c8700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 0x7f335006b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f335c004f90 tx=0x7f335c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:45:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.313+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3360056d40 con 0x7f3368066f10 2024-03-18T14:45:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:10.523+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f33680f8f70 con 0x7f3350069210 2024-03-18T14:45:10.958 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:45:10.958 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:45:10.959 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:11.109 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:45:11.109 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:45:11.110 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:11.966 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:11 smithi012 ceph-mon[29182]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:45:11.966 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:11 smithi012 ceph-mon[29182]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:12.285 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:11 smithi100 ceph-mon[37807]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:45:12.285 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:11 smithi100 ceph-mon[37807]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi012 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi012 to 12869M 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi012 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi012 to 12869M 2024-03-18T14:45:14.494 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.495 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:14.495 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:14.495 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:14.495 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3171230563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3"}]: dispatch 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3171230563' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3"}]': finished 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:15.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:15 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:15.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3171230563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3"}]: dispatch 2024-03-18T14:45:15.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3171230563' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3"}]': finished 2024-03-18T14:45:15.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T14:45:15.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:15.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:15.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:15 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:16 smithi012 ceph-mon[29182]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:16 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1314913373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:45:16.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:16 smithi100 ceph-mon[37807]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:16.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:16 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1314913373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:18.409 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:18.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:18.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:18.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:18.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:18.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:18.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:20.422 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:20 smithi012 ceph-mon[29182]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:20 smithi100 ceph-mon[37807]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:21.454 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:21 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:21 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:22.425 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:22 smithi012 ceph-mon[29182]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:22.425 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:45:22.425 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:22 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:22.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:22 smithi100 ceph-mon[37807]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:45:22.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:22 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:23.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:23 smithi012 ceph-mon[29182]: Deploying daemon osd.1 on smithi012 2024-03-18T14:45:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:23 smithi100 ceph-mon[37807]: Deploying daemon osd.1 on smithi012 2024-03-18T14:45:24.470 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:24 smithi012 ceph-mon[29182]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:24 smithi100 ceph-mon[37807]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:26.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:26 smithi012 ceph-mon[29182]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:26.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:26 smithi100 ceph-mon[37807]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:28.232 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:27 smithi100 ceph-mon[37807]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:28.232 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:28.232 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:28.233 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:27 smithi012 ceph-mon[29182]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:29.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:29 smithi012 ceph-mon[29182]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:29.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:29.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:29 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:29 smithi100 ceph-mon[37807]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:29 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:30.742 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 1 on host 'smithi012' 2024-03-18T14:45:30.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.740+0000 7f33667fc700 1 -- 172.21.15.12:0/2115829123 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f33680f8f70 con 0x7f3350069210 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 msgr2=0x7f335006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 0x7f335006b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f335c004f90 tx=0x7f335c009e30 comp rx=0 tx=0).stop 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 msgr2=0x7f336810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f336000b790 tx=0x7f3360009220 comp rx=0 tx=0).stop 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 shutdown_connections 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3350069210 0x7f335006b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3368066f10 0x7f336810fb70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 --2- 172.21.15.12:0/2115829123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3368102360 0x7f33681100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 >> 172.21.15.12:0/2115829123 conn(0x7f33680f7430 msgr2=0x7f33680f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:45:30.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 shutdown_connections 2024-03-18T14:45:30.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:30.742+0000 7f336fe2d700 1 -- 172.21.15.12:0/2115829123 wait complete. 2024-03-18T14:45:31.977 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:31 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:31.977 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:31 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:31.977 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:31 smithi012 ceph-mon[29182]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:31.977 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:31 smithi012 ceph-mon[29182]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T14:45:32.003 DEBUG:teuthology.orchestra.run.smithi012:osd.1> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.1.service 2024-03-18T14:45:32.006 INFO:tasks.cephadm:Deploying osd.2 on smithi012 with /dev/vg_nvme/lv_2... 2024-03-18T14:45:32.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T14:45:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:31 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:31 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:31 smithi100 ceph-mon[37807]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:31 smithi100 ceph-mon[37807]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T14:45:32.425 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:45:32.914 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-mon[29182]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T14:45:32.914 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-mon[29182]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T14:45:32.915 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:32.915 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-mon[29182]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:45:32.915 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[54754]: 2024-03-18T14:45:32.704+0000 7f1bfc5e9700 -1 osd.1 0 waiting for initial osdmap 2024-03-18T14:45:32.915 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:45:32 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[54754]: 2024-03-18T14:45:32.711+0000 7f1bf6be3700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:45:33.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:32 smithi100 ceph-mon[37807]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T14:45:33.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:32 smithi100 ceph-mon[37807]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T14:45:33.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:32 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:33.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:32 smithi100 ceph-mon[37807]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:45:33.878 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:33 smithi012 ceph-mon[29182]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:45:33.878 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:33 smithi012 ceph-mon[29182]: osdmap e13: 2 total, 1 up, 2 in 2024-03-18T14:45:33.879 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:33 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:33.879 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:33 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:33.879 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:33 smithi012 ceph-mon[29182]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:33 smithi100 ceph-mon[37807]: from='osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:45:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:33 smithi100 ceph-mon[37807]: osdmap e13: 2 total, 1 up, 2 in 2024-03-18T14:45:34.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:33 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:34.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:33 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:34.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:33 smithi100 ceph-mon[37807]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-18T14:45:34.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:45:34.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:45:34.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:34.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] boot 2024-03-18T14:45:34.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T14:45:34.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:34 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:35.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:45:35.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:45:35.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:35.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: osd.1 [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] boot 2024-03-18T14:45:35.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T14:45:35.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:34 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128447 s, 816 MB/s 2024-03-18T14:45:35.388 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-03-18T14:45:35.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:35 smithi012 ceph-mon[29182]: osdmap e15: 2 total, 2 up, 2 in 2024-03-18T14:45:35.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:35 smithi012 ceph-mon[29182]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:35.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:35.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:35 smithi100 ceph-mon[37807]: osdmap e15: 2 total, 2 up, 2 in 2024-03-18T14:45:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:35 smithi100 ceph-mon[37807]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:36.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-03-18T14:45:36.846 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:36.861 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:45:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:37.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:37 smithi012 ceph-mon[29182]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:38.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:37 smithi100 ceph-mon[37807]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 -- 172.21.15.12:0/1844955865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fbcd0 msgr2=0x7fe9400fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 --2- 172.21.15.12:0/1844955865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fbcd0 0x7fe9400fc0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe930004940 tx=0x7fe930009e30 comp rx=0 tx=0).stop 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 -- 172.21.15.12:0/1844955865 shutdown_connections 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 --2- 172.21.15.12:0/1844955865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fcf80 0x7fe9400fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 --2- 172.21.15.12:0/1844955865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fbcd0 0x7fe9400fc0e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 -- 172.21.15.12:0/1844955865 >> 172.21.15.12:0/1844955865 conn(0x7fe9400f7430 msgr2=0x7fe9400f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 -- 172.21.15.12:0/1844955865 shutdown_connections 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.597+0000 7fe945bd4700 1 -- 172.21.15.12:0/1844955865 wait complete. 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 Processor -- start 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 -- start start 2024-03-18T14:45:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fbcd0 0x7fe94010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94010e520 con 0x7fe9400fcf80 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe945bd4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94010ed70 con 0x7fe9400fbcd0 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe93effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe93effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41144/0 (socket says 172.21.15.12:41144) 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe93effd700 1 -- 172.21.15.12:0/988822650 learned_addr learned my addr 172.21.15.12:0/988822650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.598+0000 7fe93f7fe700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fbcd0 0x7fe94010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe93effd700 1 -- 172.21.15.12:0/988822650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fbcd0 msgr2=0x7fe94010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe93effd700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fbcd0 0x7fe94010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe93effd700 1 -- 172.21.15.12:0/988822650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930005040 con 0x7fe9400fcf80 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe93effd700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe928002a50 tx=0x7fe9280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:45:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe928007cc0 con 0x7fe9400fcf80 2024-03-18T14:45:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94010f050 con 0x7fe9400fcf80 2024-03-18T14:45:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe92800f040 con 0x7fe9400fcf80 2024-03-18T14:45:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92800da30 con 0x7fe9400fcf80 2024-03-18T14:45:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.599+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94010f590 con 0x7fe9400fcf80 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.600+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe940101b10 con 0x7fe9400fcf80 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.601+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fe92800dbb0 con 0x7fe9400fcf80 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.601+0000 7fe937fff700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 0x7fe92c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.601+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fe928086240 con 0x7fe9400fcf80 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.601+0000 7fe93f7fe700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 0x7fe92c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:45:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.602+0000 7fe93f7fe700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 0x7fe92c06b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe93000b970 tx=0x7fe930009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:45:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.606+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe928056c50 con 0x7fe9400fcf80 2024-03-18T14:45:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:39.810+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe940104920 con 0x7fe92c069210 2024-03-18T14:45:40.183 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:40.183 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.183 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.183 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:45:40.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:45:40.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:45:40.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:41.085 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:40 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi012 2024-03-18T14:45:41.085 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:40 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi012 to 6434M 2024-03-18T14:45:41.085 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:40 smithi012 ceph-mon[29182]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:45:41.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:40 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi012 2024-03-18T14:45:41.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:40 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi012 to 6434M 2024-03-18T14:45:41.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:40 smithi100 ceph-mon[37807]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:45:41.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:41 smithi012 ceph-mon[29182]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:41 smithi100 ceph-mon[37807]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:42 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1037481773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879ccbfa-0108-49c6-a6d9-c3badfec53f2"}]: dispatch 2024-03-18T14:45:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:42 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1037481773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "879ccbfa-0108-49c6-a6d9-c3badfec53f2"}]': finished 2024-03-18T14:45:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:42 smithi012 ceph-mon[29182]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T14:45:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:45:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:42 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1037481773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879ccbfa-0108-49c6-a6d9-c3badfec53f2"}]: dispatch 2024-03-18T14:45:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:42 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1037481773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "879ccbfa-0108-49c6-a6d9-c3badfec53f2"}]': finished 2024-03-18T14:45:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:42 smithi100 ceph-mon[37807]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T14:45:43.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:45:44.127 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:43 smithi012 ceph-mon[29182]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:44.127 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:43 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1175879389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:45:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:43 smithi100 ceph-mon[37807]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:43 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1175879389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:45:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:46.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:45:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:48.155 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:47 smithi012 ceph-mon[29182]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:47 smithi100 ceph-mon[37807]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:50.163 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:49 smithi012 ceph-mon[29182]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:49 smithi100 ceph-mon[37807]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:45:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:51.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:45:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:45:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:45:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:51 smithi012 ceph-mon[29182]: Deploying daemon osd.2 on smithi012 2024-03-18T14:45:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:51 smithi012 ceph-mon[29182]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:51 smithi100 ceph-mon[37807]: Deploying daemon osd.2 on smithi012 2024-03-18T14:45:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:51 smithi100 ceph-mon[37807]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:53.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:53 smithi012 ceph-mon[29182]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:53 smithi100 ceph-mon[37807]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:56.068 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:55 smithi012 ceph-mon[29182]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:56.068 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:56.068 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:56.068 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:55 smithi100 ceph-mon[37807]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:56.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:45:56.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:45:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:45:57 smithi012 ceph-mon[29182]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:45:57 smithi100 ceph-mon[37807]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:45:59.032 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 2 on host 'smithi012' 2024-03-18T14:45:59.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.030+0000 7fe937fff700 1 -- 172.21.15.12:0/988822650 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe940104920 con 0x7fe92c069210 2024-03-18T14:45:59.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 msgr2=0x7fe92c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:59.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 0x7fe92c06b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe93000b970 tx=0x7fe930009e30 comp rx=0 tx=0).stop 2024-03-18T14:45:59.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 msgr2=0x7fe94010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:45:59.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe928002a50 tx=0x7fe9280046b0 comp rx=0 tx=0).stop 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 shutdown_connections 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe92c069210 0x7fe92c06b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9400fbcd0 0x7fe94010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 --2- 172.21.15.12:0/988822650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9400fcf80 0x7fe94010df20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 >> 172.21.15.12:0/988822650 conn(0x7fe9400f7430 msgr2=0x7fe940065b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 shutdown_connections 2024-03-18T14:45:59.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:45:59.033+0000 7fe945bd4700 1 -- 172.21.15.12:0/988822650 wait complete. 2024-03-18T14:45:59.659 DEBUG:teuthology.orchestra.run.smithi012:osd.2> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.2.service 2024-03-18T14:45:59.661 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_4... 2024-03-18T14:45:59.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T14:45:59.920 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:00 smithi100 ceph-mon[37807]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:00 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:00 smithi100 ceph-mon[37807]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T14:46:00.403 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:00 smithi012 ceph-mon[29182]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:00.403 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:00.403 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:00 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:00.403 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:00 smithi012 ceph-mon[29182]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T14:46:01.187 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:01 smithi100 ceph-mon[37807]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T14:46:01.188 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:01 smithi100 ceph-mon[37807]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T14:46:01.188 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:01 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:01.188 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:01 smithi100 ceph-mon[37807]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:46:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-mon[29182]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T14:46:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-mon[29182]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T14:46:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-mon[29182]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:46:01.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[62266]: 2024-03-18T14:46:01.033+0000 7f8d35489700 -1 osd.2 0 waiting for initial osdmap 2024-03-18T14:46:01.243 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:01 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[62266]: 2024-03-18T14:46:01.038+0000 7f8d30284700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132688 s, 790 MB/s 2024-03-18T14:46:01.717 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-18T14:46:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:02 smithi100 ceph-mon[37807]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:02.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:02 smithi100 ceph-mon[37807]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:46:02.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:02 smithi100 ceph-mon[37807]: osdmap e18: 3 total, 2 up, 3 in 2024-03-18T14:46:02.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:02 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:02.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:02 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:02.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-03-18T14:46:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:02 smithi012 ceph-mon[29182]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:02 smithi012 ceph-mon[29182]: from='osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-03-18T14:46:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:02 smithi012 ceph-mon[29182]: osdmap e18: 3 total, 2 up, 3 in 2024-03-18T14:46:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:02.604 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:46:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] boot 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:03.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:46:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:46:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: osd.2 [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] boot 2024-03-18T14:46:03.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T14:46:03.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:46:03.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:03.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 -- 172.21.15.100:0/1711210872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 msgr2=0x7f44c80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 --2- 172.21.15.100:0/1711210872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fd3f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44bc004940 tx=0x7f44bc009e30 comp rx=0 tx=0).stop 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 -- 172.21.15.100:0/1711210872 shutdown_connections 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 --2- 172.21.15.100:0/1711210872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fd3f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 --2- 172.21.15.100:0/1711210872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 -- 172.21.15.100:0/1711210872 >> 172.21.15.100:0/1711210872 conn(0x7f44c80f7450 msgr2=0x7f44c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 -- 172.21.15.100:0/1711210872 shutdown_connections 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.061+0000 7f44cce7c700 1 -- 172.21.15.100:0/1711210872 wait complete. 2024-03-18T14:46:04.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 Processor -- start 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 -- start start 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8100170 con 0x7f44c80fbcd0 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44cce7c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80fe800 con 0x7f44c80fcf80 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44c5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44c659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44c5d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38032/0 (socket says 172.21.15.100:38032) 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.062+0000 7f44c5d9b700 1 -- 172.21.15.100:0/4277715516 learned_addr learned my addr 172.21.15.100:0/4277715516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44c5d9b700 1 -- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 msgr2=0x7f44c80ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:04.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44c5d9b700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44c5d9b700 1 -- 172.21.15.100:0/4277715516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc005040 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44c659c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44c5d9b700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f44bc013870 tx=0x7f44bc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44bc021070 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c80fea80 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44bc009430 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44bc00e070 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.063+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c80fefc0 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.064+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c804c710 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.064+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f44bc01c750 con 0x7f44c80fcf80 2024-03-18T14:46:04.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.065+0000 7f44b6ffd700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 0x7f44b006b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:04.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.065+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f44bc087a80 con 0x7f44c80fcf80 2024-03-18T14:46:04.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.065+0000 7f44c659c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 0x7f44b006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:04.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.065+0000 7f44c659c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 0x7f44b006b650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009b80 tx=0x7f44b800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:04.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.069+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f44bc0581e0 con 0x7f44c80fcf80 2024-03-18T14:46:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:04 smithi012 ceph-mon[29182]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:04.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-18T14:46:04.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-18T14:46:04.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:04 smithi012 ceph-mon[29182]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T14:46:04.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T14:46:04.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:04.258+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f44c805e400 con 0x7f44b00691a0 2024-03-18T14:46:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:04 smithi100 ceph-mon[37807]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-18T14:46:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-18T14:46:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-18T14:46:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:04 smithi100 ceph-mon[37807]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T14:46:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T14:46:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T14:46:05.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:05 smithi100 ceph-mon[37807]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T14:46:05.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:05.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:05.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:05.356 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T14:46:05.356 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:05 smithi012 ceph-mon[29182]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T14:46:05.356 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:05.356 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:05.357 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:05.632 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T14:46:05.632 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:05.632 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66691]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:05.632 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66691]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:05.921 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66792]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T14:46:05.921 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66792]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:05.921 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66792]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:05.921 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66792]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:06.175 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66870]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T14:46:06.175 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66870]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:06.175 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66870]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:06.175 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:05 smithi012 sudo[66870]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: pgmap v76: 1 pgs: 1 creating+activating; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: osdmap e22: 3 total, 3 up, 3 in 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.175 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: pgmap v76: 1 pgs: 1 creating+activating; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: osdmap e22: 3 total, 3 up, 3 in 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:06.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:06.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:06.449 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T14:46:06.449 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:06.449 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66892]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:06.449 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66892]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:06.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:06.452+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f44bc01c400 con 0x7f44c80fcf80 2024-03-18T14:46:06.721 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T14:46:06.721 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:06.722 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66910]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:06.722 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66910]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:06.977 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T14:46:06.977 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:06.977 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:06.977 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[66995]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[67018]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T14:46:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[67018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[67018]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:06 smithi012 sudo[67018]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:07.508 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi012 2024-03-18T14:46:07.508 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi012 to 4289M 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: mgrmap e19: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/4118782083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]: dispatch 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]: dispatch 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]': finished 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 ceph-mon[37807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41184]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:07.509 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41184]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:07.648 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 sudo[67023]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 sudo[67023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 sudo[67023]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 sudo[67023]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi012 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi012 to 4289M 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: mgrmap e19: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/4118782083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]: dispatch 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]: dispatch 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1efeb93-b505-43ad-854b-c58259129d99"}]': finished 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:07.649 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:07 smithi012 ceph-mon[29182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T14:46:07.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T14:46:07.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T14:46:07.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T14:46:07.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:07 smithi100 sudo[41187]: pam_unix(sudo:session): session closed for user root 2024-03-18T14:46:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/3219582457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T14:46:08.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:08 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/3219582457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:46:08.592 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T14:46:08.593 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:46:08.593 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:46:08.593 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T14:46:08.593 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:08 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:09 smithi012 ceph-mon[29182]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:09 smithi100 ceph-mon[37807]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:10.972 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:10.972 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:10.972 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:10.972 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:10.972 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:10 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:10.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:10.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:10.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:10.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:10.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:10 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:11.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:11 smithi100 ceph-mon[37807]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:11.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:11 smithi012 ceph-mon[29182]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:14.109 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:13 smithi100 ceph-mon[37807]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:13 smithi012 ceph-mon[29182]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:46:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:14 smithi012 ceph-mon[29182]: Deploying daemon osd.3 on smithi100 2024-03-18T14:46:15.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:46:15.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:15.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:14 smithi100 ceph-mon[37807]: Deploying daemon osd.3 on smithi100 2024-03-18T14:46:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:15 smithi100 ceph-mon[37807]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:15 smithi012 ceph-mon[29182]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:18.039 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:17 smithi100 ceph-mon[37807]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:17 smithi012 ceph-mon[29182]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:20.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:19 smithi012 ceph-mon[29182]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:20.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:19 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:20.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:19 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:20.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:19 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:19 smithi100 ceph-mon[37807]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:19 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:19 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:19 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:21.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:20 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:21.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:20 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:21.941 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:21 smithi100 ceph-mon[37807]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:22.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:21 smithi012 ceph-mon[29182]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:22.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.622+0000 7f44b6ffd700 1 -- 172.21.15.100:0/4277715516 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f44c805e400 con 0x7f44b00691a0 2024-03-18T14:46:22.624 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.624+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 msgr2=0x7f44b006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.624+0000 7f44cce7c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 0x7f44b006b650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009b80 tx=0x7f44b800b040 comp rx=0 tx=0).stop 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 msgr2=0x7f44c80fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f44bc013870 tx=0x7f44bc009e30 comp rx=0 tx=0).stop 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 shutdown_connections 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f44b00691a0 0x7f44b006b650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44c80fbcd0 0x7f44c80ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:22.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 --2- 172.21.15.100:0/4277715516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44c80fcf80 0x7f44c80fe2c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:22.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 >> 172.21.15.100:0/4277715516 conn(0x7f44c80f7450 msgr2=0x7f44c8066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:46:22.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 shutdown_connections 2024-03-18T14:46:22.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:22.625+0000 7f44cce7c700 1 -- 172.21.15.100:0/4277715516 wait complete. 2024-03-18T14:46:23.658 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.3.service 2024-03-18T14:46:23.660 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-03-18T14:46:23.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T14:46:23.917 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.918 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:23 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:23.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:46:23.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:46:23.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:23.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:23 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:24.421 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:25.091 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-mon[37807]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T14:46:25.091 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-mon[37807]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T14:46:25.091 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:25.091 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-mon[37807]: from='osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:25.091 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-mon[37807]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:25.091 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[43219]: 2024-03-18T14:46:24.855+0000 7f2e16408700 -1 osd.3 0 waiting for initial osdmap 2024-03-18T14:46:25.091 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:46:24 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[43219]: 2024-03-18T14:46:24.860+0000 7f2e11a04700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:46:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:24 smithi012 ceph-mon[29182]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T14:46:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:24 smithi012 ceph-mon[29182]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T14:46:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:24 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:24 smithi012 ceph-mon[29182]: from='osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:24 smithi012 ceph-mon[29182]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:25 smithi100 ceph-mon[37807]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:25 smithi100 ceph-mon[37807]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:46:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:25 smithi100 ceph-mon[37807]: osdmap e25: 4 total, 3 up, 4 in 2024-03-18T14:46:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:25 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:25 smithi012 ceph-mon[29182]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:25 smithi012 ceph-mon[29182]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:46:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:25 smithi012 ceph-mon[29182]: osdmap e25: 4 total, 3 up, 4 in 2024-03-18T14:46:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:25 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:46:27.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:46:27.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:27.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] boot 2024-03-18T14:46:27.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T14:46:27.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:26 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: osd.3 [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] boot 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T14:46:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:26 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:46:27.826 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T14:46:27.827 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T14:46:27.827 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-18T14:46:27.827 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-18T14:46:27.827 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137025 s, 765 MB/s 2024-03-18T14:46:27.827 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.128 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:28.129 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:27 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-18T14:46:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:28.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:27 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:28.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-03-18T14:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:29.116 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:28 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi100 2024-03-18T14:46:29.116 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:28 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi100 to 15936M 2024-03-18T14:46:29.116 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:28 smithi100 ceph-mon[37807]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T14:46:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:28 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi100 2024-03-18T14:46:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:28 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi100 to 15936M 2024-03-18T14:46:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:28 smithi012 ceph-mon[29182]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T14:46:29.874 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:29 smithi100 ceph-mon[37807]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:30.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- 172.21.15.100:0/1995777650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 msgr2=0x7f92f80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1995777650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f80fd280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f92e8004940 tx=0x7f92e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- 172.21.15.100:0/1995777650 shutdown_connections 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1995777650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 0x7f92f80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1995777650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f80fd280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- 172.21.15.100:0/1995777650 >> 172.21.15.100:0/1995777650 conn(0x7f92f80f8550 msgr2=0x7f92f80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- 172.21.15.100:0/1995777650 shutdown_connections 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- 172.21.15.100:0/1995777650 wait complete. 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 Processor -- start 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.120+0000 7f92fe0da700 1 -- start start 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:30.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 0x7f92f810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f810e520 con 0x7f92f80fdfe0 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8112ec0 con 0x7f92f80fce70 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56360/0 (socket says 172.21.15.100:56360) 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 -- 172.21.15.100:0/1603495143 learned_addr learned my addr 172.21.15.100:0/1603495143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f6ffd700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 0x7f92f810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 -- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 msgr2=0x7f92f810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 0x7f92f810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 -- 172.21.15.100:0/1603495143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8005040 con 0x7f92f80fce70 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92f77fe700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009150 tx=0x7f92e8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e800e040 con 0x7f92f80fce70 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92e800bbf0 con 0x7f92f80fce70 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f81130c0 con 0x7f92f80fce70 2024-03-18T14:46:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e801c630 con 0x7f92f80fce70 2024-03-18T14:46:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.121+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f8113630 con 0x7f92f80fce70 2024-03-18T14:46:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.122+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f804c710 con 0x7f92f80fce70 2024-03-18T14:46:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.123+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f92e800bd60 con 0x7f92f80fce70 2024-03-18T14:46:30.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.123+0000 7f92dffff700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 0x7f92e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:30.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.123+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f92e8087d70 con 0x7f92f80fce70 2024-03-18T14:46:30.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.123+0000 7f92f6ffd700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 0x7f92e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:30.127 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:29 smithi100 ceph-mon[37807]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T14:46:30.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.124+0000 7f92f6ffd700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 0x7f92e006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009bf0 tx=0x7f92ec0091a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:30.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.128+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f92e8058320 con 0x7f92f80fce70 2024-03-18T14:46:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:29 smithi012 ceph-mon[29182]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:29 smithi012 ceph-mon[29182]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T14:46:30.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:30.321+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f81022b0 con 0x7f92e00692a0 2024-03-18T14:46:31.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:30 smithi100 ceph-mon[37807]: osdmap e30: 4 total, 4 up, 4 in 2024-03-18T14:46:31.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:30 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:31.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:30 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:31.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:30 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:31.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:30 smithi012 ceph-mon[29182]: osdmap e30: 4 total, 4 up, 4 in 2024-03-18T14:46:31.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:30 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:31.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:30 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:31.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:30 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:32.007 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:31 smithi100 ceph-mon[37807]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:32.007 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:31 smithi100 ceph-mon[37807]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T14:46:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:31 smithi012 ceph-mon[29182]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:32.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:31 smithi012 ceph-mon[29182]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T14:46:33.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:32 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/2224193297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]: dispatch 2024-03-18T14:46:33.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:32 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]: dispatch 2024-03-18T14:46:33.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:32 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]': finished 2024-03-18T14:46:33.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:32 smithi100 ceph-mon[37807]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T14:46:33.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:32 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:33.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:32 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2224193297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]: dispatch 2024-03-18T14:46:33.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:32 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]: dispatch 2024-03-18T14:46:33.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:32 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d2105a2-ae9a-492a-b5a6-9df98f19131c"}]': finished 2024-03-18T14:46:33.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:32 smithi012 ceph-mon[29182]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T14:46:33.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:32 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:34.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:33 smithi012 ceph-mon[29182]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-18T14:46:34.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:33 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2230892218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:46:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:33 smithi100 ceph-mon[37807]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-18T14:46:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:33 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/2230892218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:46:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:35 smithi012 ceph-mon[29182]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T14:46:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:35 smithi100 ceph-mon[37807]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T14:46:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:38.168 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:37 smithi100 ceph-mon[37807]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-18T14:46:38.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:37 smithi012 ceph-mon[29182]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-18T14:46:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:39 smithi100 ceph-mon[37807]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:46:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:39 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:39 smithi012 ceph-mon[29182]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:46:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:39 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:41.221 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:40 smithi100 ceph-mon[37807]: Deploying daemon osd.4 on smithi100 2024-03-18T14:46:41.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:40 smithi012 ceph-mon[29182]: Deploying daemon osd.4 on smithi100 2024-03-18T14:46:42.221 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:41 smithi100 ceph-mon[37807]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:41 smithi012 ceph-mon[29182]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:43.998 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:43 smithi100 ceph-mon[37807]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:44.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:43 smithi012 ceph-mon[29182]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:46.160 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:45 smithi100 ceph-mon[37807]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:46.160 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:46.160 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:46.160 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:45 smithi012 ceph-mon[29182]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:48.059 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:47 smithi012 ceph-mon[29182]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:48.112 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:47 smithi100 ceph-mon[37807]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:48.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.184+0000 7f92dffff700 1 -- 172.21.15.100:0/1603495143 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f92f81022b0 con 0x7f92e00692a0 2024-03-18T14:46:48.186 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 msgr2=0x7f92e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 0x7f92e006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009bf0 tx=0x7f92ec0091a0 comp rx=0 tx=0).stop 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 msgr2=0x7f92f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009150 tx=0x7f92e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 shutdown_connections 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f92e00692a0 0x7f92e006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f80fce70 0x7f92f810d950 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 --2- 172.21.15.100:0/1603495143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92f80fdfe0 0x7f92f810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:48.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 >> 172.21.15.100:0/1603495143 conn(0x7f92f80f8550 msgr2=0x7f92f80f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:46:48.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 shutdown_connections 2024-03-18T14:46:48.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:48.187+0000 7f92fe0da700 1 -- 172.21.15.100:0/1603495143 wait complete. 2024-03-18T14:46:49.067 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.4.service 2024-03-18T14:46:49.070 INFO:tasks.cephadm:Deploying osd.5 on smithi100 with /dev/vg_nvme/lv_2... 2024-03-18T14:46:49.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T14:46:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:49 smithi100 ceph-mon[37807]: from='osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:46:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:49 smithi100 ceph-mon[37807]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:46:49.371 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:49 smithi012 ceph-mon[29182]: from='osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:46:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:49 smithi012 ceph-mon[29182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:50.408 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:50.409 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[49653]: 2024-03-18T14:46:50.156+0000 7f192001a700 -1 osd.4 0 waiting for initial osdmap 2024-03-18T14:46:50.409 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:46:50 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[49653]: 2024-03-18T14:46:50.163+0000 7f191ae15700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:46:50.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:50.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:51.357 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:51 smithi100 ceph-mon[37807]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:46:51.357 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:51 smithi100 ceph-mon[37807]: osdmap e33: 5 total, 4 up, 5 in 2024-03-18T14:46:51.357 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:51.357 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:51.357 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:51 smithi012 ceph-mon[29182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:46:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:51 smithi012 ceph-mon[29182]: osdmap e33: 5 total, 4 up, 5 in 2024-03-18T14:46:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134819 s, 778 MB/s 2024-03-18T14:46:51.499 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-18T14:46:52.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] boot 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T14:46:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:52 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:52.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:46:52.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:46:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-18T14:46:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: osd.4 [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] boot 2024-03-18T14:46:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T14:46:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:52 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:46:52.638 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:46:53.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:53 smithi012 ceph-mon[29182]: osdmap e35: 5 total, 5 up, 5 in 2024-03-18T14:46:53.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:53 smithi100 ceph-mon[37807]: osdmap e35: 5 total, 5 up, 5 in 2024-03-18T14:46:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:54 smithi100 ceph-mon[37807]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:54 smithi012 ceph-mon[29182]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:55.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.056+0000 7f039dd1e700 1 -- 172.21.15.100:0/4237221819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 msgr2=0x7f03980fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:55.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.056+0000 7f039dd1e700 1 --2- 172.21.15.100:0/4237221819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f03980fc0e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0388004940 tx=0x7f0388009e30 comp rx=0 tx=0).stop 2024-03-18T14:46:55.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- 172.21.15.100:0/4237221819 shutdown_connections 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 --2- 172.21.15.100:0/4237221819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f03980fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 --2- 172.21.15.100:0/4237221819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f03980fc0e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- 172.21.15.100:0/4237221819 >> 172.21.15.100:0/4237221819 conn(0x7f03980f7430 msgr2=0x7f03980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- 172.21.15.100:0/4237221819 shutdown_connections 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- 172.21.15.100:0/4237221819 wait complete. 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 Processor -- start 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- start start 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f039810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039810e470 con 0x7f03980fcf80 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.057+0000 7f039dd1e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039810e5e0 con 0x7f03980fbcd0 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35434/0 (socket says 172.21.15.100:35434) 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 -- 172.21.15.100:0/3677516768 learned_addr learned my addr 172.21.15.100:0/3677516768 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f0396ffd700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f039810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 -- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 msgr2=0x7f039810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f039810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 -- 172.21.15.100:0/3677516768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0388005040 con 0x7f03980fbcd0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f0396ffd700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f039810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f03977fe700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f038800b3b0 tx=0x7f0388009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f038800e040 con 0x7f03980fbcd0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f038800bcc0 con 0x7f03980fbcd0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0398112fe0 con 0x7f03980fbcd0 2024-03-18T14:46:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f038801c630 con 0x7f03980fbcd0 2024-03-18T14:46:55.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.058+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0398113510 con 0x7f03980fbcd0 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.059+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0398101b10 con 0x7f03980fbcd0 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.060+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0388013980 con 0x7f03980fbcd0 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.060+0000 7f037ffff700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 0x7f038006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.060+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f0388087ab0 con 0x7f03980fbcd0 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.061+0000 7f0396ffd700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 0x7f038006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:46:55.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.061+0000 7f0396ffd700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 0x7f038006b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f038c0041d0 tx=0x7f038c009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:46:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.064+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0388057ce0 con 0x7f03980fbcd0 2024-03-18T14:46:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:46:55.253+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f039802b420 con 0x7f03800692a0 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi100 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi100 to 7968M 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.928 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:55.929 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi100 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi100 to 7968M 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T14:46:55.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:46:57.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:56 smithi100 ceph-mon[37807]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:57.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:56 smithi012 ceph-mon[29182]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:46:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/1371430886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]: dispatch 2024-03-18T14:46:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]: dispatch 2024-03-18T14:46:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]': finished 2024-03-18T14:46:58.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T14:46:58.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/1371430886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]: dispatch 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]: dispatch 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "822b9e7b-edd3-49b1-b668-d6c34e799d33"}]': finished 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T14:46:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:46:59.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:58 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/2441389192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:46:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:58 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/2441389192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T14:47:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:46:59 smithi012 ceph-mon[29182]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:46:59 smithi100 ceph-mon[37807]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:02.015 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:01 smithi100 ceph-mon[37807]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:01 smithi012 ceph-mon[29182]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:04.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:03 smithi100 ceph-mon[37807]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:03 smithi012 ceph-mon[29182]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:05.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:47:05.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:04 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:05.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:47:05.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:04 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:06.141 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:05 smithi100 ceph-mon[37807]: Deploying daemon osd.5 on smithi100 2024-03-18T14:47:06.141 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:05 smithi100 ceph-mon[37807]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:06.141 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:05 smithi012 ceph-mon[29182]: Deploying daemon osd.5 on smithi100 2024-03-18T14:47:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:05 smithi012 ceph-mon[29182]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:08.104 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:07 smithi100 ceph-mon[37807]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:07 smithi012 ceph-mon[29182]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:10.181 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:09 smithi100 ceph-mon[37807]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:10.182 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:09 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:10.182 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:09 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:10.182 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:09 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:47:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:09 smithi012 ceph-mon[29182]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:09 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:47:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:11 smithi012 ceph-mon[29182]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:11 smithi100 ceph-mon[37807]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:12.617 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 5 on host 'smithi100' 2024-03-18T14:47:12.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.615+0000 7f037ffff700 1 -- 172.21.15.100:0/3677516768 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f039802b420 con 0x7f03800692a0 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.617+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 msgr2=0x7f038006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.617+0000 7f039dd1e700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 0x7f038006b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f038c0041d0 tx=0x7f038c009040 comp rx=0 tx=0).stop 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.617+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 msgr2=0x7f039810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.617+0000 7f039dd1e700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f038800b3b0 tx=0x7f0388009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 shutdown_connections 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f03800692a0 0x7f038006b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03980fbcd0 0x7f039810d930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 --2- 172.21.15.100:0/3677516768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03980fcf80 0x7f039810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 >> 172.21.15.100:0/3677516768 conn(0x7f03980f7430 msgr2=0x7f0398066850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 shutdown_connections 2024-03-18T14:47:12.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:12.618+0000 7f039dd1e700 1 -- 172.21.15.100:0/3677516768 wait complete. 2024-03-18T14:47:13.198 DEBUG:teuthology.orchestra.run.smithi100:osd.5> sudo journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.5.service 2024-03-18T14:47:13.200 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-18T14:47:13.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T14:47:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:13.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:13.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:13.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-mon[37807]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:13.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-mon[37807]: from='osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T14:47:13.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-mon[37807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T14:47:13.796 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 14:47:13 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T14:47:13.530+0000 7efea7e52540 -1 osd.5 0 log_to_monitors true 2024-03-18T14:47:13.928 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:13.928 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:13 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:13.929 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:13 smithi012 ceph-mon[29182]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:13.929 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:13 smithi012 ceph-mon[29182]: from='osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T14:47:13.929 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:13 smithi012 ceph-mon[29182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:14.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:14 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:15.130 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:15.131 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:15.131 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T14:47:14.853+0000 7efe9e624700 -1 osd.5 0 waiting for initial osdmap 2024-03-18T14:47:15.131 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 14:47:14 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T14:47:14.860+0000 7efe9941f700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 -- 172.21.15.12:0/555597879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fcef0 msgr2=0x7fb9600fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 --2- 172.21.15.12:0/555597879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fcef0 0x7fb9600fd300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb950004940 tx=0x7fb950009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 -- 172.21.15.12:0/555597879 shutdown_connections 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 --2- 172.21.15.12:0/555597879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fe060 0x7fb9600fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 --2- 172.21.15.12:0/555597879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fcef0 0x7fb9600fd300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 -- 172.21.15.12:0/555597879 >> 172.21.15.12:0/555597879 conn(0x7fb9600f85b0 msgr2=0x7fb9600fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 -- 172.21.15.12:0/555597879 shutdown_connections 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 -- 172.21.15.12:0/555597879 wait complete. 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.250+0000 7fb9668fc700 1 Processor -- start 2024-03-18T14:47:15.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 -- start start 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fcef0 0x7fb96010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96010e4f0 con 0x7fb9600fe060 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96010ed40 con 0x7fb9600fcef0 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54136/0 (socket says 172.21.15.12:54136) 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 -- 172.21.15.12:0/1801825008 learned_addr learned my addr 172.21.15.12:0/1801825008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95ffff700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fcef0 0x7fb96010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 -- 172.21.15.12:0/1801825008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fcef0 msgr2=0x7fb96010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fcef0 0x7fb96010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 -- 172.21.15.12:0/1801825008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950005040 con 0x7fb9600fe060 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95f7fe700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb954002a80 tx=0x7fb9540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb954007cc0 con 0x7fb9600fe060 2024-03-18T14:47:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96010f020 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb95400f040 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.251+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95400dac0 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.252+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96010f560 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.252+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96005f5f0 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.253+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb95400dc20 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.253+0000 7fb95cff9700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.253+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fb954086980 con 0x7fb9600fe060 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.253+0000 7fb95ffff700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.254+0000 7fb95ffff700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 0x7fb94806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb95000b3b0 tx=0x7fb950009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.258+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb954056b10 con 0x7fb9600fe060 2024-03-18T14:47:15.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.450+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb96010f820 con 0x7fb9600fe060 2024-03-18T14:47:15.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.450+0000 7fb95cff9700 1 -- 172.21.15.12:0/1801825008 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fb9540566a0 con 0x7fb9600fe060 2024-03-18T14:47:15.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.452+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 msgr2=0x7fb94806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.452+0000 7fb9668fc700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 0x7fb94806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb95000b3b0 tx=0x7fb950009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.452+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 msgr2=0x7fb96010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.452+0000 7fb9668fc700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb954002a80 tx=0x7fb9540046b0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 shutdown_connections 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9600fcef0 0x7fb96010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 --2- 172.21.15.12:0/1801825008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600fe060 0x7fb96010def0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 >> 172.21.15.12:0/1801825008 conn(0x7fb9600f85b0 msgr2=0x7fb9600f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 shutdown_connections 2024-03-18T14:47:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:15.453+0000 7fb9668fc700 1 -- 172.21.15.12:0/1801825008 wait complete. 2024-03-18T14:47:15.995 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:15.995 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:47:15.996 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T14:47:15.996 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:15.996 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:15.996 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:15 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1801825008' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T14:47:16.131 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:16.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-18T14:47:16.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T14:47:16.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:16.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:16.132 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:15 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1801825008' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T14:47:16.158 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1710773211,"num_in_osds":6,"osd_in_since":1710773217,"num_remapped_pgs":0} 2024-03-18T14:47:16.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:16 smithi012 ceph-mon[29182]: purged_snaps scrub starts 2024-03-18T14:47:16.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:16 smithi012 ceph-mon[29182]: purged_snaps scrub ok 2024-03-18T14:47:16.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:16 smithi012 ceph-mon[29182]: osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] boot 2024-03-18T14:47:16.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:16 smithi012 ceph-mon[29182]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T14:47:16.994 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:16 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:16 smithi100 ceph-mon[37807]: purged_snaps scrub starts 2024-03-18T14:47:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:16 smithi100 ceph-mon[37807]: purged_snaps scrub ok 2024-03-18T14:47:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:16 smithi100 ceph-mon[37807]: osd.5 [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] boot 2024-03-18T14:47:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:16 smithi100 ceph-mon[37807]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T14:47:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:16 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:47:17.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T14:47:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:18.238 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:18.238 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.239 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:47:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-18T14:47:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T14:47:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:18.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:47:19.165 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:18 smithi012 ceph-mon[29182]: Detected new or changed devices on smithi100 2024-03-18T14:47:19.165 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:18 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi100 to 5312M 2024-03-18T14:47:19.165 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:19.165 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:47:19.165 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:19.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 -- 172.21.15.12:0/2853939749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 msgr2=0x7ff5f00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:19.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 --2- 172.21.15.12:0/2853939749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00fc0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4004940 tx=0x7ff5e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 -- 172.21.15.12:0/2853939749 shutdown_connections 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 --2- 172.21.15.12:0/2853939749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 --2- 172.21.15.12:0/2853939749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00fc0e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 -- 172.21.15.12:0/2853939749 >> 172.21.15.12:0/2853939749 conn(0x7ff5f00f7430 msgr2=0x7ff5f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 -- 172.21.15.12:0/2853939749 shutdown_connections 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.163+0000 7ff5f7008700 1 -- 172.21.15.12:0/2853939749 wait complete. 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.164+0000 7ff5f7008700 1 Processor -- start 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.164+0000 7ff5f7008700 1 -- start start 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.164+0000 7ff5f7008700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f7008700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f7008700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0100170 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f7008700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00fe800 con 0x7ff5f00fcf80 2024-03-18T14:47:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53266/0 (socket says 172.21.15.12:53266) 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 -- 172.21.15.12:0/3175424169 learned_addr learned my addr 172.21.15.12:0/3175424169 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5effff700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 -- 172.21.15.12:0/3175424169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 msgr2=0x7ff5f00fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 -- 172.21.15.12:0/3175424169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4005040 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5effff700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f4da4700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009150 tx=0x7ff5e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e400e040 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f00fea80 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5e400bbf0 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e401c630 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.165+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f00ff040 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.166+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f004c710 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.167+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff5e400bd60 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.167+0000 7ff5ed7fa700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 0x7ff5e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.167+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff5e4088230 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.167+0000 7ff5effff700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 0x7ff5e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.168+0000 7ff5effff700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 0x7ff5e006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0041d0 tx=0x7ff5dc009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.172+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7ff5e4058330 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:18 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi100 2024-03-18T14:47:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:18 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi100 to 5312M 2024-03-18T14:47:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:47:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:47:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:47:19.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.363+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff5f0101a40 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.363+0000 7ff5ed7fa700 1 -- 172.21.15.12:0/3175424169 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7ff5e40540a0 con 0x7ff5f00fbcd0 2024-03-18T14:47:19.365 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.365+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 msgr2=0x7ff5e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 0x7ff5e006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0041d0 tx=0x7ff5dc009040 comp rx=0 tx=0).stop 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 msgr2=0x7ff5f00ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009150 tx=0x7ff5e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 shutdown_connections 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff5e00692a0 0x7ff5e006b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f00fbcd0 0x7ff5f00ffb70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 --2- 172.21.15.12:0/3175424169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5f00fcf80 0x7ff5f00fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 >> 172.21.15.12:0/3175424169 conn(0x7ff5f00f7430 msgr2=0x7ff5f00668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:19.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 shutdown_connections 2024-03-18T14:47:19.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:19.366+0000 7ff5f7008700 1 -- 172.21.15.12:0/3175424169 wait complete. 2024-03-18T14:47:20.097 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1710773235,"num_in_osds":6,"osd_in_since":1710773217,"num_remapped_pgs":0} 2024-03-18T14:47:20.098 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T14:47:20.123 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:19 smithi012 ceph-mon[29182]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:20.123 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:19 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3175424169' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T14:47:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:19 smithi100 ceph-mon[37807]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:19 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3175424169' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T14:47:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:21.102 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:20 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:21.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:20 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:22.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- 172.21.15.12:0/3678392855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d080fb4a0 msgr2=0x7f0d080fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 --2- 172.21.15.12:0/3678392855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fb910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004940 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- 172.21.15.12:0/3678392855 shutdown_connections 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 --2- 172.21.15.12:0/3678392855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fb910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 --2- 172.21.15.12:0/3678392855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08102e70 0x7f0d08103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- 172.21.15.12:0/3678392855 >> 172.21.15.12:0/3678392855 conn(0x7f0d080714c0 msgr2=0x7f0d080718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- 172.21.15.12:0/3678392855 shutdown_connections 2024-03-18T14:47:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- 172.21.15.12:0/3678392855 wait complete. 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 Processor -- start 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.997+0000 7f0d105d9700 1 -- start start 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080fd150 con 0x7f0d08102e70 2024-03-18T14:47:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080fd290 con 0x7f0d080fb4a0 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53284/0 (socket says 172.21.15.12:53284) 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 -- 172.21.15.12:0/156650107 learned_addr learned my addr 172.21.15.12:0/156650107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0e375700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 -- 172.21.15.12:0/156650107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d080fb4a0 msgr2=0x7f0d080fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 -- 172.21.15.12:0/156650107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc005040 con 0x7f0d08102e70 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d0db74700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3b0 tx=0x7f0cfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cfc00e040 con 0x7f0d08102e70 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cfc013420 con 0x7f0d08102e70 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d080fd4e0 con 0x7f0d08102e70 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cfc01c630 con 0x7f0d08102e70 2024-03-18T14:47:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.998+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0811a240 con 0x7f0d08102e70 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:21.999+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08100170 con 0x7f0d08102e70 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.000+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0cfc013590 con 0x7f0d08102e70 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.000+0000 7f0d06ffd700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 0x7f0cf006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.000+0000 7f0d0e375700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 0x7f0cf006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.000+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0cfc088020 con 0x7f0d08102e70 2024-03-18T14:47:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.001+0000 7f0d0e375700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 0x7f0cf006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8004550 tx=0x7f0cf8007ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.005+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0cfc054690 con 0x7f0d08102e70 2024-03-18T14:47:22.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.198+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d08109f60 con 0x7f0d08102e70 2024-03-18T14:47:22.199 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:21 smithi012 ceph-mon[29182]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:22.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.198+0000 7f0d06ffd700 1 -- 172.21.15.12:0/156650107 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7f0cfc057cb0 con 0x7f0d08102e70 2024-03-18T14:47:22.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:22.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T14:47:16.856906+0000","last_up_change":"2024-03-18T14:47:15.850925+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6803","nonce":1389034439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6805","nonce":1389034439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6809","nonce":1389034439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6807","nonce":1389034439}]},"public_addr":"172.21.15.12:6803/1389034439","cluster_addr":"172.21.15.12:6805/1389034439","heartbeat_back_addr":"172.21.15.12:6809/1389034439","heartbeat_front_addr":"172.21.15.12:6807/1389034439","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6811","nonce":2569125971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6813","nonce":2569125971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6817","nonce":2569125971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6815","nonce":2569125971}]},"public_addr":"172.21.15.12:6811/2569125971","cluster_addr":"172.21.15.12:6813/2569125971","heartbeat_back_addr":"172.21.15.12:6817/2569125971","heartbeat_front_addr":"172.21.15.12:6815/2569125971","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6819","nonce":892771200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6821","nonce":892771200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6825","nonce":892771200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6823","nonce":892771200}]},"public_addr":"172.21.15.12:6819/892771200","cluster_addr":"172.21.15.12:6821/892771200","heartbeat_back_addr":"172.21.15.12:6825/892771200","heartbeat_front_addr":"172.21.15.12:6823/892771200","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6801","nonce":2368724172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6803","nonce":2368724172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6807","nonce":2368724172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6805","nonce":2368724172}]},"public_addr":"172.21.15.100:6801/2368724172","cluster_addr":"172.21.15.100:6803/2368724172","heartbeat_back_addr":"172.21.15.100:6807/2368724172","heartbeat_front_addr":"172.21.15.100:6805/2368724172","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6809","nonce":1429143082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6811","nonce":1429143082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6815","nonce":1429143082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6813","nonce":1429143082}]},"public_addr":"172.21.15.100:6809/1429143082","cluster_addr":"172.21.15.100:6811/1429143082","heartbeat_back_addr":"172.21.15.100:6815/1429143082","heartbeat_front_addr":"172.21.15.100:6813/1429143082","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6817","nonce":3471230267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6819","nonce":3471230267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6823","nonce":3471230267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6821","nonce":3471230267}]},"public_addr":"172.21.15.100:6817/3471230267","cluster_addr":"172.21.15.100:6819/3471230267","heartbeat_back_addr":"172.21.15.100:6823/3471230267","heartbeat_front_addr":"172.21.15.100:6821/3471230267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 msgr2=0x7f0cf006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 0x7f0cf006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8004550 tx=0x7f0cf8007ed0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 msgr2=0x7f0d080fcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3b0 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 shutdown_connections 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0cf00692a0 0x7f0cf006b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d080fb4a0 0x7f0d080fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.200+0000 7f0d105d9700 1 --2- 172.21.15.12:0/156650107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d08102e70 0x7f0d080fcbc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:22.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.201+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 >> 172.21.15.12:0/156650107 conn(0x7f0d080714c0 msgr2=0x7f0d080fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:22.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.201+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 shutdown_connections 2024-03-18T14:47:22.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:22.201+0000 7f0d105d9700 1 -- 172.21.15.12:0/156650107 wait complete. 2024-03-18T14:47:22.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:21 smithi100 ceph-mon[37807]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:22.777 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T14:46:03.095708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-18T14:47:22.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-18T14:47:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:23.193 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:22 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/156650107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:23.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:22 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/156650107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:24.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:23 smithi012 ceph-mon[29182]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:24.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:23 smithi100 ceph-mon[37807]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 -- 172.21.15.12:0/4282543715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c102e20 msgr2=0x7f541c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 --2- 172.21.15.12:0/4282543715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c102e20 0x7f541c1031f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f540c004940 tx=0x7f540c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 -- 172.21.15.12:0/4282543715 shutdown_connections 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 --2- 172.21.15.12:0/4282543715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c1037c0 0x7f541c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 --2- 172.21.15.12:0/4282543715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c102e20 0x7f541c1031f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 -- 172.21.15.12:0/4282543715 >> 172.21.15.12:0/4282543715 conn(0x7f541c0f89b0 msgr2=0x7f541c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 -- 172.21.15.12:0/4282543715 shutdown_connections 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.812+0000 7f5422314700 1 -- 172.21.15.12:0/4282543715 wait complete. 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 Processor -- start 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 -- start start 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c102e20 0x7f541c113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c1144d0 con 0x7f541c1037c0 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f5422314700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c10dad0 con 0x7f541c102e20 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f541b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f541b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53292/0 (socket says 172.21.15.12:53292) 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.813+0000 7f541b7fe700 1 -- 172.21.15.12:0/594763965 learned_addr learned my addr 172.21.15.12:0/594763965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f541bfff700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c102e20 0x7f541c113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:24.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f541b7fe700 1 -- 172.21.15.12:0/594763965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c102e20 msgr2=0x7f541c113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f541b7fe700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c102e20 0x7f541c113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f541b7fe700 1 -- 172.21.15.12:0/594763965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c005040 con 0x7f541c1037c0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f541b7fe700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f541000c800 tx=0x7f5410004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541000a760 con 0x7f541c1037c0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c10ddb0 con 0x7f541c1037c0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f541000ad90 con 0x7f541c1037c0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54100113a0 con 0x7f541c1037c0 2024-03-18T14:47:24.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.814+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c10e370 con 0x7f541c1037c0 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.815+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c100130 con 0x7f541c1037c0 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.815+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5410012400 con 0x7f541c1037c0 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.816+0000 7f5418ff9700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 0x7f540406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.816+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5410086dd0 con 0x7f541c1037c0 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.816+0000 7f541bfff700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 0x7f540406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.816+0000 7f541bfff700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 0x7f540406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f540c00b3b0 tx=0x7f540c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:24.820+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5410056f70 con 0x7f541c1037c0 2024-03-18T14:47:25.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.010+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f541c10ef90 con 0x7f541c1037c0 2024-03-18T14:47:25.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.011+0000 7f5418ff9700 1 -- 172.21.15.12:0/594763965 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5410056b00 con 0x7f541c1037c0 2024-03-18T14:47:25.012 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-03-18T14:47:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 msgr2=0x7f540406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 0x7f540406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f540c00b3b0 tx=0x7f540c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 msgr2=0x7f541c113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f541000c800 tx=0x7f5410004470 comp rx=0 tx=0).stop 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 shutdown_connections 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f54040692a0 0x7f540406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f541c102e20 0x7f541c113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 --2- 172.21.15.12:0/594763965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f541c1037c0 0x7f541c113e80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 >> 172.21.15.12:0/594763965 conn(0x7f541c0f89b0 msgr2=0x7f541c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 shutdown_connections 2024-03-18T14:47:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:25.013+0000 7f5422314700 1 -- 172.21.15.12:0/594763965 wait complete. 2024-03-18T14:47:25.681 INFO:tasks.cephadm:Setting up client nodes... 2024-03-18T14:47:25.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T14:47:25.946 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:26.164 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:25 smithi012 ceph-mon[29182]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:26.164 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:25 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/594763965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T14:47:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:25 smithi100 ceph-mon[37807]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:25 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/594763965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 -- 172.21.15.12:0/1634898128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fcfd0 msgr2=0x7fe50c0fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 --2- 172.21.15.12:0/1634898128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c0fd440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc004940 tx=0x7fe4fc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 -- 172.21.15.12:0/1634898128 shutdown_connections 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 --2- 172.21.15.12:0/1634898128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c0fd440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 --2- 172.21.15.12:0/1634898128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fbd20 0x7fe50c0fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 -- 172.21.15.12:0/1634898128 >> 172.21.15.12:0/1634898128 conn(0x7fe50c0f7460 msgr2=0x7fe50c0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 -- 172.21.15.12:0/1634898128 shutdown_connections 2024-03-18T14:47:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.534+0000 7fe51403d700 1 -- 172.21.15.12:0/1634898128 wait complete. 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 Processor -- start 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 -- start start 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c112c70 con 0x7fe50c0fbd20 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe51403d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c112de0 con 0x7fe50c0fcfd0 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53300/0 (socket says 172.21.15.12:53300) 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 -- 172.21.15.12:0/1066613703 learned_addr learned my addr 172.21.15.12:0/1066613703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe5115d8700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 -- 172.21.15.12:0/1066613703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fcfd0 msgr2=0x7fe50c112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 -- 172.21.15.12:0/1066613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc005040 con 0x7fe50c0fbd20 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe511dd9700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b790 tx=0x7fe508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.535+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe50800f6f0 con 0x7fe50c0fbd20 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.536+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe50800fd20 con 0x7fe50c0fbd20 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.536+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c1130c0 con 0x7fe50c0fbd20 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.536+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5080103a0 con 0x7fe50c0fbd20 2024-03-18T14:47:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.536+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c113680 con 0x7fe50c0fbd20 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.537+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c04c710 con 0x7fe50c0fbd20 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.537+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe508011400 con 0x7fe50c0fbd20 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.537+0000 7fe5027fc700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 0x7fe4f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.537+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe508086c30 con 0x7fe50c0fbd20 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.537+0000 7fe5115d8700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 0x7fe4f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.538+0000 7fe5115d8700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 0x7fe4f806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b970 tx=0x7fe4fc009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.542+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe508056d30 con 0x7fe50c0fbd20 2024-03-18T14:47:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.810+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe50c104890 con 0x7fe50c0fbd20 2024-03-18T14:47:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.813+0000 7fe5027fc700 1 -- 172.21.15.12:0/1066613703 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe5080568c0 con 0x7fe50c0fbd20 2024-03-18T14:47:27.815 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-03-18T14:47:27.815 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD/U/hl6cpgMBAAOz4HE1+6n/A/hxIROogVxw== 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.815+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 msgr2=0x7fe4f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.815+0000 7fe51403d700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 0x7fe4f806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b970 tx=0x7fe4fc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.815+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 msgr2=0x7fe50c066000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.815+0000 7fe51403d700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b790 tx=0x7fe508009250 comp rx=0 tx=0).stop 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 shutdown_connections 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe4f80692a0 0x7fe4f806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0fbd20 0x7fe50c066000 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 --2- 172.21.15.12:0/1066613703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50c0fcfd0 0x7fe50c112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 >> 172.21.15.12:0/1066613703 conn(0x7fe50c0f7460 msgr2=0x7fe50c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 shutdown_connections 2024-03-18T14:47:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:27.816+0000 7fe51403d700 1 -- 172.21.15.12:0/1066613703 wait complete. 2024-03-18T14:47:28.137 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:27 smithi012 ceph-mon[29182]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:28.137 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:27 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1066613703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:28.137 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:27 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1066613703' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T14:47:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:27 smithi100 ceph-mon[37807]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:27 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1066613703' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:27 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1066613703' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T14:47:28.455 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:47:28.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-18T14:47:28.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T14:47:28.498 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T14:47:28.766 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi100/config 2024-03-18T14:47:30.176 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:29 smithi100 ceph-mon[37807]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:29 smithi012 ceph-mon[29182]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 -- 172.21.15.100:0/2350387230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 msgr2=0x7fa11c0fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 --2- 172.21.15.100:0/2350387230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c0fe4d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa10c004940 tx=0x7fa10c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 -- 172.21.15.100:0/2350387230 shutdown_connections 2024-03-18T14:47:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 --2- 172.21.15.100:0/2350387230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c0fe4d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 --2- 172.21.15.100:0/2350387230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 0x7fa11c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 -- 172.21.15.100:0/2350387230 >> 172.21.15.100:0/2350387230 conn(0x7fa11c0f8570 msgr2=0x7fa11c0fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 -- 172.21.15.100:0/2350387230 shutdown_connections 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.282+0000 7fa12105b700 1 -- 172.21.15.100:0/2350387230 wait complete. 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 Processor -- start 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 -- start start 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 0x7fa11c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10e480 con 0x7fa11c0fce60 2024-03-18T14:47:30.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa12105b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10e5f0 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46518/0 (socket says 172.21.15.100:46518) 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 -- 172.21.15.100:0/4167032474 learned_addr learned my addr 172.21.15.100:0/4167032474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11ad9d700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 0x7fa11c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 -- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 msgr2=0x7fa11c10d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 0x7fa11c10d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 -- 172.21.15.100:0/4167032474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c005040 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.283+0000 7fa11a59c700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b3b0 tx=0x7fa10c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.284+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c00e040 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.284+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c112ff0 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.284+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa10c00bbf0 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.284+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c01c630 con 0x7fa11c0fe060 2024-03-18T14:47:30.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.284+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c113580 con 0x7fa11c0fe060 2024-03-18T14:47:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.285+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c04c710 con 0x7fa11c0fe060 2024-03-18T14:47:30.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.285+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa10c00bd60 con 0x7fa11c0fe060 2024-03-18T14:47:30.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.286+0000 7fa1037fe700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 0x7fa10406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:30.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.286+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa10c087b30 con 0x7fa11c0fe060 2024-03-18T14:47:30.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.286+0000 7fa11ad9d700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 0x7fa10406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:30.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.286+0000 7fa11ad9d700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 0x7fa10406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0fdbd0 tx=0x7fa11000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:30.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.290+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fa10c057c30 con 0x7fa11c0fe060 2024-03-18T14:47:30.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.542+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa11c113770 con 0x7fa11c0fe060 2024-03-18T14:47:30.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.547+0000 7fa1037fe700 1 -- 172.21.15.100:0/4167032474 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa10c0577c0 con 0x7fa11c0fe060 2024-03-18T14:47:30.548 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-03-18T14:47:30.548 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQACVPhleCR1IBAAnrFfprWCv5UDCXQpmLB5nw== 2024-03-18T14:47:30.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 msgr2=0x7fa10406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 0x7fa10406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0fdbd0 tx=0x7fa11000b040 comp rx=0 tx=0).stop 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 msgr2=0x7fa11c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b3b0 tx=0x7fa10c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 shutdown_connections 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa1040692a0 0x7fa10406b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa11c0fce60 0x7fa11c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 --2- 172.21.15.100:0/4167032474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c0fe060 0x7fa11c10de80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 >> 172.21.15.100:0/4167032474 conn(0x7fa11c0f8570 msgr2=0x7fa11c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 shutdown_connections 2024-03-18T14:47:30.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:47:30.549+0000 7fa12105b700 1 -- 172.21.15.100:0/4167032474 wait complete. 2024-03-18T14:47:31.121 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:30 smithi100 ceph-mon[37807]: from='client.? 172.21.15.100:0/4167032474' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:31.121 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:30 smithi100 ceph-mon[37807]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:31.121 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:30 smithi100 ceph-mon[37807]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T14:47:31.122 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:47:31.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-18T14:47:31.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-18T14:47:31.167 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T14:47:31.167 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-18T14:47:31.167 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-18T14:47:31.196 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:30 smithi012 ceph-mon[29182]: from='client.? 172.21.15.100:0/4167032474' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:31.196 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:30 smithi012 ceph-mon[29182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T14:47:31.196 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:30 smithi012 ceph-mon[29182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T14:47:31.436 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:31 smithi012 ceph-mon[29182]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:31 smithi100 ceph-mon[37807]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 -- 172.21.15.12:0/3081125500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3410102e00 msgr2=0x7f34101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 --2- 172.21.15.12:0/3081125500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3410102e00 0x7f34101031d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f340c004940 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 -- 172.21.15.12:0/3081125500 shutdown_connections 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 --2- 172.21.15.12:0/3081125500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34101037a0 0x7f34100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 --2- 172.21.15.12:0/3081125500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3410102e00 0x7f34101031d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.098+0000 7f3417d6f700 1 -- 172.21.15.12:0/3081125500 >> 172.21.15.12:0/3081125500 conn(0x7f34100f89b0 msgr2=0x7f34100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 -- 172.21.15.12:0/3081125500 shutdown_connections 2024-03-18T14:47:33.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 -- 172.21.15.12:0/3081125500 wait complete. 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 Processor -- start 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 -- start start 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3410102e00 0x7f3410113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.099+0000 7f3417d6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f3417d6f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410114550 con 0x7f34101037a0 2024-03-18T14:47:33.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f3417d6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341010dac0 con 0x7f3410102e00 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45078/0 (socket says 172.21.15.12:45078) 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 -- 172.21.15.12:0/4045895459 learned_addr learned my addr 172.21.15.12:0/4045895459 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f3415b0b700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3410102e00 0x7f3410113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 -- 172.21.15.12:0/4045895459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3410102e00 msgr2=0x7f3410113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3410102e00 0x7f3410113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 -- 172.21.15.12:0/4045895459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c005040 con 0x7f34101037a0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f341530a700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f340000b790 tx=0x7f3400009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340000f6f0 con 0x7f34101037a0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f340000fd20 con 0x7f34101037a0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341010dda0 con 0x7f34101037a0 2024-03-18T14:47:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34000103a0 con 0x7f34101037a0 2024-03-18T14:47:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.100+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341010e360 con 0x7f34101037a0 2024-03-18T14:47:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.101+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34100fe550 con 0x7f34101037a0 2024-03-18T14:47:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.102+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3400011400 con 0x7f34101037a0 2024-03-18T14:47:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.102+0000 7f34067fc700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 0x7f33fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.102+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3400086c20 con 0x7f34101037a0 2024-03-18T14:47:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.102+0000 7f3415b0b700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 0x7f33fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.102+0000 7f3415b0b700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 0x7f33fc06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f340c004f90 tx=0x7f340c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.106+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3400056d20 con 0x7f34101037a0 2024-03-18T14:47:33.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.354+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f341004c710 con 0x7f34101037a0 2024-03-18T14:47:33.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.357+0000 7f34067fc700 1 -- 172.21.15.12:0/4045895459 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167166 (secure 0 0 0) 0x7f34000568b0 con 0x7f34101037a0 2024-03-18T14:47:33.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:33.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.359+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 msgr2=0x7f33fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:33.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.359+0000 7f3417d6f700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 0x7f33fc06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f340c004f90 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:33.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 msgr2=0x7f3410113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:33.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f340000b790 tx=0x7f3400009220 comp rx=0 tx=0).stop 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 shutdown_connections 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f33fc0692a0 0x7f33fc06b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3410102e00 0x7f3410113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 --2- 172.21.15.12:0/4045895459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34101037a0 0x7f3410113e70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 >> 172.21.15.12:0/4045895459 conn(0x7f34100f89b0 msgr2=0x7f34100f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 shutdown_connections 2024-03-18T14:47:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:33.360+0000 7f3417d6f700 1 -- 172.21.15.12:0/4045895459 wait complete. 2024-03-18T14:47:34.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:33 smithi012 ceph-mon[29182]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:34.184 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:33 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4045895459' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T14:47:34.213 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"active_gid":14220,"active_name":"smithi012.tmscgt","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":965904248},{"type":"v1","addr":"172.21.15.12:6801","nonce":965904248}]},"active_addr":"172.21.15.12:6801/965904248","active_change":"2024-03-18T14:43:50.763082+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14242,"name":"smithi100.rwnrcn","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/","prometheus":"http://172.21.15.12:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1103543303}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3811799837}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2624983338}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3753470626}]}]} 2024-03-18T14:47:34.216 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-18T14:47:34.216 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-18T14:47:34.216 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T14:47:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:33 smithi100 ceph-mon[37807]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:33 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4045895459' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T14:47:34.516 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:36.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:35 smithi012 ceph-mon[29182]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:36.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 -- 172.21.15.12:0/1350308436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 msgr2=0x7fa1a40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 --2- 172.21.15.12:0/1350308436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a40fce10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa194004940 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 -- 172.21.15.12:0/1350308436 shutdown_connections 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 --2- 172.21.15.12:0/1350308436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a40fce10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 --2- 172.21.15.12:0/1350308436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 0x7fa1a41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 -- 172.21.15.12:0/1350308436 >> 172.21.15.12:0/1350308436 conn(0x7fa1a40f89b0 msgr2=0x7fa1a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 -- 172.21.15.12:0/1350308436 shutdown_connections 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.176+0000 7fa1a9127700 1 -- 172.21.15.12:0/1350308436 wait complete. 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 Processor -- start 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 -- start start 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 0x7fa1a4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4114510 con 0x7fa1a41037a0 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a9127700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a410db10 con 0x7fa1a4102e00 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45100/0 (socket says 172.21.15.12:45100) 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.177+0000 7fa1a259c700 1 -- 172.21.15.12:0/623625353 learned_addr learned my addr 172.21.15.12:0/623625353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a2d9d700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 0x7fa1a4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a259c700 1 -- 172.21.15.12:0/623625353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 msgr2=0x7fa1a4113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a259c700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 0x7fa1a4113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a259c700 1 -- 172.21.15.12:0/623625353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194005040 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a259c700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa194009100 tx=0x7fa194009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19400e040 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a410dd90 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa194013420 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19401c630 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.178+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a410e350 con 0x7fa1a41037a0 2024-03-18T14:47:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.179+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a41052d0 con 0x7fa1a41037a0 2024-03-18T14:47:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.179+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa19401c8f0 con 0x7fa1a41037a0 2024-03-18T14:47:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.180+0000 7fa18b7fe700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 0x7fa18c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.180+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa194087f50 con 0x7fa1a41037a0 2024-03-18T14:47:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.180+0000 7fa1a2d9d700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 0x7fa18c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.180+0000 7fa1a2d9d700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 0x7fa18c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1980041d0 tx=0x7fa1980043c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:36.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.184+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fa1940b9db0 con 0x7fa1a41037a0 2024-03-18T14:47:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:35 smithi100 ceph-mon[37807]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:36.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.376+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1a410efd0 con 0x7fa1a41037a0 2024-03-18T14:47:36.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.377+0000 7fa18b7fe700 1 -- 172.21.15.12:0/623625353 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7fa1940580c0 con 0x7fa1a41037a0 2024-03-18T14:47:36.379 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:36.379 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T14:47:16.856906+0000","last_up_change":"2024-03-18T14:47:15.850925+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6803","nonce":1389034439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6805","nonce":1389034439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6809","nonce":1389034439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6807","nonce":1389034439}]},"public_addr":"172.21.15.12:6803/1389034439","cluster_addr":"172.21.15.12:6805/1389034439","heartbeat_back_addr":"172.21.15.12:6809/1389034439","heartbeat_front_addr":"172.21.15.12:6807/1389034439","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6811","nonce":2569125971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6813","nonce":2569125971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6817","nonce":2569125971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6815","nonce":2569125971}]},"public_addr":"172.21.15.12:6811/2569125971","cluster_addr":"172.21.15.12:6813/2569125971","heartbeat_back_addr":"172.21.15.12:6817/2569125971","heartbeat_front_addr":"172.21.15.12:6815/2569125971","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6819","nonce":892771200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6821","nonce":892771200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6825","nonce":892771200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6823","nonce":892771200}]},"public_addr":"172.21.15.12:6819/892771200","cluster_addr":"172.21.15.12:6821/892771200","heartbeat_back_addr":"172.21.15.12:6825/892771200","heartbeat_front_addr":"172.21.15.12:6823/892771200","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6801","nonce":2368724172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6803","nonce":2368724172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6807","nonce":2368724172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6805","nonce":2368724172}]},"public_addr":"172.21.15.100:6801/2368724172","cluster_addr":"172.21.15.100:6803/2368724172","heartbeat_back_addr":"172.21.15.100:6807/2368724172","heartbeat_front_addr":"172.21.15.100:6805/2368724172","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6809","nonce":1429143082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6811","nonce":1429143082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6815","nonce":1429143082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6813","nonce":1429143082}]},"public_addr":"172.21.15.100:6809/1429143082","cluster_addr":"172.21.15.100:6811/1429143082","heartbeat_back_addr":"172.21.15.100:6815/1429143082","heartbeat_front_addr":"172.21.15.100:6813/1429143082","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6817","nonce":3471230267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6819","nonce":3471230267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6823","nonce":3471230267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6821","nonce":3471230267}]},"public_addr":"172.21.15.100:6817/3471230267","cluster_addr":"172.21.15.100:6819/3471230267","heartbeat_back_addr":"172.21.15.100:6823/3471230267","heartbeat_front_addr":"172.21.15.100:6821/3471230267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T14:47:36.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 msgr2=0x7fa18c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:36.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 0x7fa18c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1980041d0 tx=0x7fa1980043c0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 msgr2=0x7fa1a4113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa194009100 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 shutdown_connections 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa18c0692a0 0x7fa18c06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1a4102e00 0x7fa1a4113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 --2- 172.21.15.12:0/623625353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1a41037a0 0x7fa1a4113ec0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 >> 172.21.15.12:0/623625353 conn(0x7fa1a40f89b0 msgr2=0x7fa1a41021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 shutdown_connections 2024-03-18T14:47:36.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:36.379+0000 7fa1a9127700 1 -- 172.21.15.12:0/623625353 wait complete. 2024-03-18T14:47:37.166 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:36 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/623625353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:37.167 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-18T14:47:37.167 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T14:47:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:36 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/623625353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:37.431 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:38.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:37 smithi012 ceph-mon[29182]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:37 smithi100 ceph-mon[37807]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.124+0000 7fb80e55c700 1 -- 172.21.15.12:0/4261915217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 msgr2=0x7fb8081020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.124+0000 7fb80e55c700 1 --2- 172.21.15.12:0/4261915217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb8081020b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc004940 tx=0x7fb7fc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 -- 172.21.15.12:0/4261915217 shutdown_connections 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 --2- 172.21.15.12:0/4261915217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 0x7fb8080fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 --2- 172.21.15.12:0/4261915217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb8081020b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 -- 172.21.15.12:0/4261915217 >> 172.21.15.12:0/4261915217 conn(0x7fb8080f7890 msgr2=0x7fb8080f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 -- 172.21.15.12:0/4261915217 shutdown_connections 2024-03-18T14:47:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 -- 172.21.15.12:0/4261915217 wait complete. 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 Processor -- start 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.125+0000 7fb80e55c700 1 -- start start 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 0x7fb808113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808114510 con 0x7fb808101ce0 2024-03-18T14:47:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80810db10 con 0x7fb808102680 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41080/0 (socket says 172.21.15.12:41080) 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 -- 172.21.15.12:0/3097496034 learned_addr learned my addr 172.21.15.12:0/3097496034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 -- 172.21.15.12:0/3097496034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 msgr2=0x7fb808113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb8077fe700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 0x7fb808113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 0x7fb808113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 -- 172.21.15.12:0/3097496034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc005040 con 0x7fb808101ce0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb807fff700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00b3b0 tx=0x7fb7fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7fc00e040 con 0x7fb808101ce0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80810dd90 con 0x7fb808101ce0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7fc00bbc0 con 0x7fb808101ce0 2024-03-18T14:47:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7fc01c3f0 con 0x7fb808101ce0 2024-03-18T14:47:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.126+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80810e350 con 0x7fb808101ce0 2024-03-18T14:47:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.127+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8080fd430 con 0x7fb808101ce0 2024-03-18T14:47:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.128+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb7fc00bd30 con 0x7fb808101ce0 2024-03-18T14:47:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.128+0000 7fb804ff9700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 0x7fb7f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.128+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb7fc0880e0 con 0x7fb808101ce0 2024-03-18T14:47:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.128+0000 7fb8077fe700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 0x7fb7f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.128+0000 7fb8077fe700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 0x7fb7f006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80810ecd0 tx=0x7fb7f800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:39.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.133+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb7fc058160 con 0x7fb808101ce0 2024-03-18T14:47:39.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.327+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb808109ed0 con 0x7fb808101ce0 2024-03-18T14:47:39.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.328+0000 7fb804ff9700 1 -- 172.21.15.12:0/3097496034 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11080 (secure 0 0 0) 0x7fb7fc057cf0 con 0x7fb808101ce0 2024-03-18T14:47:39.329 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:47:39.329 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T14:47:16.856906+0000","last_up_change":"2024-03-18T14:47:15.850925+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6803","nonce":1389034439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6805","nonce":1389034439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6809","nonce":1389034439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6807","nonce":1389034439}]},"public_addr":"172.21.15.12:6803/1389034439","cluster_addr":"172.21.15.12:6805/1389034439","heartbeat_back_addr":"172.21.15.12:6809/1389034439","heartbeat_front_addr":"172.21.15.12:6807/1389034439","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6811","nonce":2569125971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6813","nonce":2569125971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6817","nonce":2569125971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6815","nonce":2569125971}]},"public_addr":"172.21.15.12:6811/2569125971","cluster_addr":"172.21.15.12:6813/2569125971","heartbeat_back_addr":"172.21.15.12:6817/2569125971","heartbeat_front_addr":"172.21.15.12:6815/2569125971","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6819","nonce":892771200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6821","nonce":892771200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6825","nonce":892771200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6823","nonce":892771200}]},"public_addr":"172.21.15.12:6819/892771200","cluster_addr":"172.21.15.12:6821/892771200","heartbeat_back_addr":"172.21.15.12:6825/892771200","heartbeat_front_addr":"172.21.15.12:6823/892771200","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6801","nonce":2368724172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6803","nonce":2368724172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6807","nonce":2368724172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6805","nonce":2368724172}]},"public_addr":"172.21.15.100:6801/2368724172","cluster_addr":"172.21.15.100:6803/2368724172","heartbeat_back_addr":"172.21.15.100:6807/2368724172","heartbeat_front_addr":"172.21.15.100:6805/2368724172","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6809","nonce":1429143082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6811","nonce":1429143082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6815","nonce":1429143082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6813","nonce":1429143082}]},"public_addr":"172.21.15.100:6809/1429143082","cluster_addr":"172.21.15.100:6811/1429143082","heartbeat_back_addr":"172.21.15.100:6815/1429143082","heartbeat_front_addr":"172.21.15.100:6813/1429143082","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6817","nonce":3471230267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6819","nonce":3471230267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6823","nonce":3471230267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6821","nonce":3471230267}]},"public_addr":"172.21.15.100:6817/3471230267","cluster_addr":"172.21.15.100:6819/3471230267","heartbeat_back_addr":"172.21.15.100:6823/3471230267","heartbeat_front_addr":"172.21.15.100:6821/3471230267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.329+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 msgr2=0x7fb7f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.329+0000 7fb80e55c700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 0x7fb7f006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80810ecd0 tx=0x7fb7f800b040 comp rx=0 tx=0).stop 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.329+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 msgr2=0x7fb808113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.329+0000 7fb80e55c700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00b3b0 tx=0x7fb7fc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 shutdown_connections 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb7f00692a0 0x7fb7f006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb808101ce0 0x7fb808113980 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 --2- 172.21.15.12:0/3097496034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808102680 0x7fb808113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 >> 172.21.15.12:0/3097496034 conn(0x7fb8080f7890 msgr2=0x7fb8080f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 shutdown_connections 2024-03-18T14:47:39.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:39.330+0000 7fb80e55c700 1 -- 172.21.15.12:0/3097496034 wait complete. 2024-03-18T14:47:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:39 smithi012 ceph-mon[29182]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:39 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3097496034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:40.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-18T14:47:40.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-18T14:47:40.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-18T14:47:40.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-18T14:47:40.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-18T14:47:40.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-18T14:47:40.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:39 smithi100 ceph-mon[37807]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:40.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:39 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3097496034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:47:40.369 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:40.372 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:41.638 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:41.639 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:42.098 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:41 smithi012 ceph-mon[29182]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:41 smithi100 ceph-mon[37807]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- 172.21.15.12:0/1276049097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406f3e0 msgr2=0x7fd23406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 --2- 172.21.15.12:0/1276049097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406f3e0 0x7fd23406f7b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd230004940 tx=0x7fd230009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- 172.21.15.12:0/1276049097 shutdown_connections 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 --2- 172.21.15.12:0/1276049097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406fd80 0x7fd234105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 --2- 172.21.15.12:0/1276049097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406f3e0 0x7fd23406f7b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- 172.21.15.12:0/1276049097 >> 172.21.15.12:0/1276049097 conn(0x7fd2340690e0 msgr2=0x7fd2340694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- 172.21.15.12:0/1276049097 shutdown_connections 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- 172.21.15.12:0/1276049097 wait complete. 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 Processor -- start 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.250+0000 7fd23bca9700 1 -- start start 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd23bca9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 0x7fd2341061a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd23bca9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd23bca9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341080a0 con 0x7fd23406fd80 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd23bca9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd234106c20 con 0x7fd23406f3e0 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239244700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239a45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 0x7fd2341061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239244700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41086/0 (socket says 172.21.15.12:41086) 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239244700 1 -- 172.21.15.12:0/2863749633 learned_addr learned my addr 172.21.15.12:0/2863749633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239a45700 1 -- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 msgr2=0x7fd2341066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239a45700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239a45700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230005040 con 0x7fd23406f3e0 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239244700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:47:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd239a45700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 0x7fd2341061a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd230000c00 tx=0x7fd230009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.251+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23000e040 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.252+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2300093d0 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.252+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd234106ea0 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.252+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd230013ea0 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.252+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2341228d0 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.253+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd23410fdc0 con 0x7fd23406f3e0 2024-03-18T14:47:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.253+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd23001c480 con 0x7fd23406f3e0 2024-03-18T14:47:43.256 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd22a7fc700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 0x7fd22006b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd230088080 con 0x7fd23406f3e0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd239244700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 0x7fd22006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd22a7fc700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 0x7fd220071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd220071a10 con 0x7fd22006ef50 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23008a620 con 0x7fd23406f3e0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd23a246700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 0x7fd220071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd239244700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 0x7fd22006b730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd234107880 tx=0x7fd22400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.254+0000 7fd23a246700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 0x7fd220071360 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.255+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== osd.0 v2:172.21.15.12:6802/1389034439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd220071a10 con 0x7fd22006ef50 2024-03-18T14:47:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.271+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 --> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd23406ec50 con 0x7fd22006ef50 2024-03-18T14:47:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd22a7fc700 1 -- 172.21.15.12:0/2863749633 <== osd.0 v2:172.21.15.12:6802/1389034439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd23406ec50 con 0x7fd22006ef50 2024-03-18T14:47:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 msgr2=0x7fd220071360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 0x7fd220071360 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 msgr2=0x7fd22006b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 0x7fd22006b730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd234107880 tx=0x7fd22400b040 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 msgr2=0x7fd2341061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 0x7fd2341061a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd230000c00 tx=0x7fd230009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 shutdown_connections 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd220069280 0x7fd22006b730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23406f3e0 0x7fd2341061a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:6802/1389034439,v1:172.21.15.12:6803/1389034439] conn(0x7fd22006ef50 0x7fd220071360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.272+0000 7fd23bca9700 1 --2- 172.21.15.12:0/2863749633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd23406fd80 0x7fd2341066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.273+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 >> 172.21.15.12:0/2863749633 conn(0x7fd2340690e0 msgr2=0x7fd23410e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.273+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 shutdown_connections 2024-03-18T14:47:43.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:43.273+0000 7fd23bca9700 1 -- 172.21.15.12:0/2863749633 wait complete. 2024-03-18T14:47:44.091 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:44.181 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:43 smithi012 ceph-mon[29182]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:44.291 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:43 smithi100 ceph-mon[37807]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:44.699 INFO:teuthology.orchestra.run.smithi012.stdout:38654705699 2024-03-18T14:47:44.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-18T14:47:44.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 -- 172.21.15.12:0/1295978735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109db0 msgr2=0x7feecc103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:44.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 --2- 172.21.15.12:0/1295978735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109db0 0x7feecc103420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feebc004940 tx=0x7feebc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:44.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 -- 172.21.15.12:0/1295978735 shutdown_connections 2024-03-18T14:47:44.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 --2- 172.21.15.12:0/1295978735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109db0 0x7feecc103420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 --2- 172.21.15.12:0/1295978735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109410 0x7feecc1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 -- 172.21.15.12:0/1295978735 >> 172.21.15.12:0/1295978735 conn(0x7feecc072c40 msgr2=0x7feecc075050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.928+0000 7feed3a0a700 1 -- 172.21.15.12:0/1295978735 shutdown_connections 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 -- 172.21.15.12:0/1295978735 wait complete. 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 Processor -- start 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 -- start start 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 0x7feecc11a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc11acf0 con 0x7feecc109410 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.929+0000 7feed3a0a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc1140c0 con 0x7feecc109db0 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41104/0 (socket says 172.21.15.12:41104) 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 -- 172.21.15.12:0/228440607 learned_addr learned my addr 172.21.15.12:0/228440607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed0fa5700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 0x7feecc11a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 -- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 msgr2=0x7feecc11a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:44.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 0x7feecc11a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc005040 con 0x7feecc109410 2024-03-18T14:47:44.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed0fa5700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 0x7feecc11a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:47:44.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed17a6700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec800c830 tx=0x7feec8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:44.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec80106f0 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feec8010d20 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feecc1143a0 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.930+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec80113a0 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.931+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc114930 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.932+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feecc114e70 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.932+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7feec8010850 con 0x7feecc109410 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feec1ffb700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 0x7feeb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:44.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feec8086e50 con 0x7feecc109410 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feec1ffb700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 0x7feeb8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feeb8071a30 con 0x7feeb806ef70 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec808c450 con 0x7feecc109410 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feed0fa5700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 0x7feeb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.933+0000 7feed1fa7700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 0x7feeb8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.934+0000 7feed0fa5700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 0x7feeb806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feebc004890 tx=0x7feebc009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:44.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.934+0000 7feed1fa7700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 0x7feeb8071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:44.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.935+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== osd.3 v2:172.21.15.100:6800/2368724172 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7feeb8071a30 con 0x7feeb806ef70 2024-03-18T14:47:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.952+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 --> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feecc104b60 con 0x7feeb806ef70 2024-03-18T14:47:44.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.953+0000 7feec1ffb700 1 -- 172.21.15.12:0/228440607 <== osd.3 v2:172.21.15.100:6800/2368724172 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7feecc104b60 con 0x7feeb806ef70 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 msgr2=0x7feeb8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 0x7feeb8071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 msgr2=0x7feeb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 0x7feeb806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feebc004890 tx=0x7feebc009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 msgr2=0x7feecc11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec800c830 tx=0x7feec8004470 comp rx=0 tx=0).stop 2024-03-18T14:47:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 shutdown_connections 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:6800/2368724172,v1:172.21.15.100:6801/2368724172] conn(0x7feeb806ef70 0x7feeb8071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7feeb80692a0 0x7feeb806b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feecc109410 0x7feecc11a160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 --2- 172.21.15.12:0/228440607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feecc109db0 0x7feecc11a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 >> 172.21.15.12:0/228440607 conn(0x7feecc072c40 msgr2=0x7feecc074c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 shutdown_connections 2024-03-18T14:47:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:44.954+0000 7feed3a0a700 1 -- 172.21.15.12:0/228440607 wait complete. 2024-03-18T14:47:46.123 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:45 smithi012 ceph-mon[29182]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:45 smithi100 ceph-mon[37807]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:47.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:47 smithi012 ceph-mon[29182]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:47 smithi100 ceph-mon[37807]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:48.906 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:48.931 INFO:teuthology.orchestra.run.smithi012.stdout:111669149714 2024-03-18T14:47:48.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-18T14:47:50.097 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:49 smithi012 ceph-mon[29182]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 -- 172.21.15.12:0/2596392045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 msgr2=0x7f3a60109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 --2- 172.21.15.12:0/2596392045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a60109180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a54004970 tx=0x7f3a54009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 -- 172.21.15.12:0/2596392045 shutdown_connections 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 --2- 172.21.15.12:0/2596392045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a60109180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 --2- 172.21.15.12:0/2596392045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 0x7f3a600706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 -- 172.21.15.12:0/2596392045 >> 172.21.15.12:0/2596392045 conn(0x7f3a600f8980 msgr2=0x7f3a600fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 -- 172.21.15.12:0/2596392045 shutdown_connections 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 -- 172.21.15.12:0/2596392045 wait complete. 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.101+0000 7f3a6616c700 1 Processor -- start 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a6616c700 1 -- start start 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a6616c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 0x7f3a60115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a6616c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a6616c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60116920 con 0x7f3a60070c10 2024-03-18T14:47:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a6616c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6010fcf0 con 0x7f3a60070300 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56632/0 (socket says 172.21.15.12:56632) 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 -- 172.21.15.12:0/800509703 learned_addr learned my addr 172.21.15.12:0/800509703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5f7fe700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 0x7f3a60115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 -- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 msgr2=0x7f3a60115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 0x7f3a60115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54005040 con 0x7f3a60070c10 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.102+0000 7f3a5effd700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005010 tx=0x7f3a54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.103+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5400e040 con 0x7f3a60070c10 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.103+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a54009850 con 0x7f3a60070c10 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.103+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6010ff70 con 0x7f3a60070c10 2024-03-18T14:47:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.103+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5401c3f0 con 0x7f3a60070c10 2024-03-18T14:47:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.103+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60110460 con 0x7f3a60070c10 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.104+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a60110990 con 0x7f3a60070c10 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.104+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3a54013420 con 0x7f3a60070c10 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a47fff700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3a54088040 con 0x7f3a60070c10 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a5f7fe700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a47fff700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 0x7f3a48071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a48071a30 con 0x7f3a4806ef70 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a5408a630 con 0x7f3a60070c10 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a5ffff700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 0x7f3a48071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.105+0000 7f3a5f7fe700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 0x7f3a4806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009b80 tx=0x7f3a5000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.106+0000 7f3a5ffff700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 0x7f3a48071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.106+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== osd.4 v2:172.21.15.100:6808/1429143082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3a48071a30 con 0x7f3a4806ef70 2024-03-18T14:47:50.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.121+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 --> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a600fa670 con 0x7f3a4806ef70 2024-03-18T14:47:50.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a47fff700 1 -- 172.21.15.12:0/800509703 <== osd.4 v2:172.21.15.100:6808/1429143082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a600fa670 con 0x7f3a4806ef70 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 msgr2=0x7f3a48071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 0x7f3a48071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 msgr2=0x7f3a4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 0x7f3a4806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009b80 tx=0x7f3a5000b040 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 msgr2=0x7f3a601162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005010 tx=0x7f3a54009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 shutdown_connections 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:6808/1429143082,v1:172.21.15.100:6809/1429143082] conn(0x7f3a4806ef70 0x7f3a48071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a60070300 0x7f3a60115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 --2- 172.21.15.12:0/800509703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a60070c10 0x7f3a601162d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.122+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 >> 172.21.15.12:0/800509703 conn(0x7f3a600f8980 msgr2=0x7f3a600f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:50.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.123+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 shutdown_connections 2024-03-18T14:47:50.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.123+0000 7f3a6616c700 1 -- 172.21.15.12:0/800509703 wait complete. 2024-03-18T14:47:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:49 smithi100 ceph-mon[37807]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:50.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 -- 172.21.15.12:0/3138398691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f00fb3b0 msgr2=0x7f87f00fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 --2- 172.21.15.12:0/3138398691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f00fb3b0 0x7f87f00fb820 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87e4004940 tx=0x7f87e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 -- 172.21.15.12:0/3138398691 shutdown_connections 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 --2- 172.21.15.12:0/3138398691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f00fb3b0 0x7f87f00fb820 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 --2- 172.21.15.12:0/3138398691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f0102f70 0x7f87f0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 -- 172.21.15.12:0/3138398691 >> 172.21.15.12:0/3138398691 conn(0x7f87f0072310 msgr2=0x7f87f0072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 -- 172.21.15.12:0/3138398691 shutdown_connections 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.782+0000 7f87f583d700 1 -- 172.21.15.12:0/3138398691 wait complete. 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 Processor -- start 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 -- start start 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00fb3b0 0x7f87f0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0114590 con 0x7f87f0102f70 2024-03-18T14:47:50.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87f583d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f010db70 con 0x7f87f00fb3b0 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87ee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.783+0000 7f87ee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56656/0 (socket says 172.21.15.12:56656) 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87ee7fc700 1 -- 172.21.15.12:0/655915992 learned_addr learned my addr 172.21.15.12:0/655915992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87eeffd700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00fb3b0 0x7f87f0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87ee7fc700 1 -- 172.21.15.12:0/655915992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00fb3b0 msgr2=0x7f87f0113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87ee7fc700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00fb3b0 0x7f87f0113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87ee7fc700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4005040 con 0x7f87f0102f70 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87ee7fc700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b3b0 tx=0x7f87e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e400e040 con 0x7f87f0102f70 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f010ddf0 con 0x7f87f0102f70 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e400bbc0 con 0x7f87f0102f70 2024-03-18T14:47:50.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e401c3f0 con 0x7f87f0102f70 2024-03-18T14:47:50.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.784+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f010e330 con 0x7f87f0102f70 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.785+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87f010e850 con 0x7f87f0102f70 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f87e4013420 con 0x7f87f0102f70 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 0x7f87d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f87e4088110 con 0x7f87f0102f70 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87eeffd700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 0x7f87d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 0x7f87d8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87d8071a30 con 0x7f87d806ef70 2024-03-18T14:47:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.786+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e408a700 con 0x7f87f0102f70 2024-03-18T14:47:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.787+0000 7f87ef7fe700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 0x7f87d8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.787+0000 7f87eeffd700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 0x7f87d806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006fd0 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.787+0000 7f87ef7fe700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 0x7f87d8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.788+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== osd.2 v2:172.21.15.12:6818/892771200 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f87d8071a30 con 0x7f87d806ef70 2024-03-18T14:47:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 --> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87f010ea10 con 0x7f87d806ef70 2024-03-18T14:47:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87d77fe700 1 -- 172.21.15.12:0/655915992 <== osd.2 v2:172.21.15.12:6818/892771200 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87f010ea10 con 0x7f87d806ef70 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 msgr2=0x7f87d8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 0x7f87d8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 msgr2=0x7f87d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 0x7f87d806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006fd0 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 msgr2=0x7f87f0113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.803+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b3b0 tx=0x7f87e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:50.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 shutdown_connections 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87d80692a0 0x7f87d806b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:6818/892771200,v1:172.21.15.12:6819/892771200] conn(0x7f87d806ef70 0x7f87d8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f00fb3b0 0x7f87f0113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 --2- 172.21.15.12:0/655915992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87f0102f70 0x7f87f0113eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 >> 172.21.15.12:0/655915992 conn(0x7f87f0072310 msgr2=0x7f87f00f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 shutdown_connections 2024-03-18T14:47:50.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.804+0000 7f87f583d700 1 -- 172.21.15.12:0/655915992 wait complete. 2024-03-18T14:47:50.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:47:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:51 smithi012 ceph-mon[29182]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:51 smithi100 ceph-mon[37807]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:52.333 INFO:teuthology.orchestra.run.smithi012.stdout:146028888078 2024-03-18T14:47:52.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 -- 172.21.15.12:0/774056566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fe830 msgr2=0x7f07f4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/774056566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fe830 0x7f07f4106f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07e4004940 tx=0x7f07e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 -- 172.21.15.12:0/774056566 shutdown_connections 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/774056566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fe830 0x7f07f4106f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/774056566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fde90 0x7f07f40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 -- 172.21.15.12:0/774056566 >> 172.21.15.12:0/774056566 conn(0x7f07f4072bc0 msgr2=0x7f07f4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 -- 172.21.15.12:0/774056566 shutdown_connections 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.000+0000 7f07fbe3e700 1 -- 172.21.15.12:0/774056566 wait complete. 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 Processor -- start 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 -- start start 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fe830 0x7f07f41140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f4114730 con 0x7f07f40fde90 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07fbe3e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f410db20 con 0x7f07f40fe830 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07f9bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07f9bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56610/0 (socket says 172.21.15.12:56610) 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.001+0000 7f07f9bda700 1 -- 172.21.15.12:0/2944493207 learned_addr learned my addr 172.21.15.12:0/2944493207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07f93d9700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fe830 0x7f07f41140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07f9bda700 1 -- 172.21.15.12:0/2944493207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fe830 msgr2=0x7f07f41140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07f9bda700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fe830 0x7f07f41140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07f9bda700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4005040 con 0x7f07f40fde90 2024-03-18T14:47:52.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07f9bda700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c830 tx=0x7f07f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f00106f0 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07f0010d20 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f00113a0 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f410de00 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.002+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f410e2f0 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.003+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07f410e920 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.004+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f07f0010850 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07ea7fc700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 0x7f07e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f07f0086e50 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07f93d9700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 0x7f07e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07ea7fc700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 0x7f07e0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07e0071a30 con 0x7f07e006ef70 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f008c4b0 con 0x7f07f40fde90 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07fa3db700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 0x7f07e0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:52.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.005+0000 7f07f93d9700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 0x7f07e006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07f410ece0 tx=0x7f07e4009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.006+0000 7f07fa3db700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 0x7f07e0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.006+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== osd.1 v2:172.21.15.12:6810/2569125971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f07e0071a30 con 0x7f07e006ef70 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.038+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 --> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07f4102480 con 0x7f07e006ef70 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07ea7fc700 1 -- 172.21.15.12:0/2944493207 <== osd.1 v2:172.21.15.12:6810/2569125971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07f4102480 con 0x7f07e006ef70 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 msgr2=0x7f07e0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 0x7f07e0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 msgr2=0x7f07e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 0x7f07e006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07f410ece0 tx=0x7f07e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 msgr2=0x7f07f4113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c830 tx=0x7f07f0004470 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 shutdown_connections 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f07e00692a0 0x7f07e006b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07f40fde90 0x7f07f4113ba0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.12:6810/2569125971,v1:172.21.15.12:6811/2569125971] conn(0x7f07e006ef70 0x7f07e0071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 --2- 172.21.15.12:0/2944493207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f40fe830 0x7f07f41140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 >> 172.21.15.12:0/2944493207 conn(0x7f07f4072bc0 msgr2=0x7f07f41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.039+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 shutdown_connections 2024-03-18T14:47:52.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:50.040+0000 7f07fbe3e700 1 -- 172.21.15.12:0/2944493207 wait complete. 2024-03-18T14:47:52.611 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:53.061 INFO:teuthology.orchestra.run.smithi012.stdout:60129542175 2024-03-18T14:47:53.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 -- 172.21.15.12:0/2033050460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fe850 msgr2=0x7f3160106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2033050460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fe850 0x7f3160106f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3148004970 tx=0x7f3148009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 -- 172.21.15.12:0/2033050460 shutdown_connections 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2033050460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fe850 0x7f3160106f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2033050460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fdeb0 0x7f31600fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 -- 172.21.15.12:0/2033050460 >> 172.21.15.12:0/2033050460 conn(0x7f3160072bc0 msgr2=0x7f3160072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 -- 172.21.15.12:0/2033050460 shutdown_connections 2024-03-18T14:47:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.687+0000 7f3165e1e700 1 -- 172.21.15.12:0/2033050460 wait complete. 2024-03-18T14:47:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.688+0000 7f3165e1e700 1 Processor -- start 2024-03-18T14:47:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.688+0000 7f3165e1e700 1 -- start start 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3165e1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3165e1e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fe850 0x7f31601140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3165e1e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3160114700 con 0x7f31600fdeb0 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3165e1e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316010daf0 con 0x7f31600fe850 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56680/0 (socket says 172.21.15.12:56680) 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 -- 172.21.15.12:0/2815366720 learned_addr learned my addr 172.21.15.12:0/2815366720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315effd700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fe850 0x7f31601140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 -- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fe850 msgr2=0x7f31601140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fe850 0x7f31601140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148005040 con 0x7f31600fdeb0 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f315f7fe700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f315000c8c0 tx=0x7f31500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f315000a760 con 0x7f31600fdeb0 2024-03-18T14:47:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f315000ad90 con 0x7f31600fdeb0 2024-03-18T14:47:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316010ddd0 con 0x7f31600fdeb0 2024-03-18T14:47:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.689+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31500113a0 con 0x7f31600fdeb0 2024-03-18T14:47:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.690+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316010e2e0 con 0x7f31600fdeb0 2024-03-18T14:47:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.691+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f316010e7a0 con 0x7f31600fdeb0 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.691+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f315000a8c0 con 0x7f31600fdeb0 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.691+0000 7f3157fff700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 0x7f314c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.691+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3150086de0 con 0x7f31600fdeb0 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.691+0000 7f315effd700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 0x7f314c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f3157fff700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 0x7f314c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:53.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f314c071a30 con 0x7f314c06ef70 2024-03-18T14:47:53.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3150089450 con 0x7f31600fdeb0 2024-03-18T14:47:53.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f315ffff700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 0x7f314c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:53.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f315effd700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 0x7f314c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3148004f50 tx=0x7f3148009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:53.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.692+0000 7f315ffff700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 0x7f314c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:53.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.693+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== osd.5 v2:172.21.15.100:6816/3471230267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f314c071a30 con 0x7f314c06ef70 2024-03-18T14:47:53.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.715+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 --> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f316005e6c0 con 0x7f314c06ef70 2024-03-18T14:47:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3157fff700 1 -- 172.21.15.12:0/2815366720 <== osd.5 v2:172.21.15.100:6816/3471230267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f316005e6c0 con 0x7f314c06ef70 2024-03-18T14:47:53.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 msgr2=0x7f314c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 0x7f314c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 msgr2=0x7f314c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 0x7f314c06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3148004f50 tx=0x7f3148009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 msgr2=0x7f3160113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f315000c8c0 tx=0x7f31500044c0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 shutdown_connections 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:6816/3471230267,v1:172.21.15.100:6817/3471230267] conn(0x7f314c06ef70 0x7f314c071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f314c0692a0 0x7f314c06b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31600fdeb0 0x7f3160113b70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 --2- 172.21.15.12:0/2815366720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600fe850 0x7f31601140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 >> 172.21.15.12:0/2815366720 conn(0x7f3160072bc0 msgr2=0x7f31600ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:53.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 shutdown_connections 2024-03-18T14:47:53.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:53.716+0000 7f3165e1e700 1 -- 172.21.15.12:0/2815366720 wait complete. 2024-03-18T14:47:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:53 smithi012 ceph-mon[29182]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:53 smithi100 ceph-mon[37807]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- 172.21.15.12:0/378466494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 msgr2=0x7f8ef00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/378466494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef00fce10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8004940 tx=0x7f8ed8009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- 172.21.15.12:0/378466494 shutdown_connections 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/378466494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef00fce10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/378466494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- 172.21.15.12:0/378466494 >> 172.21.15.12:0/378466494 conn(0x7f8ef00f89b0 msgr2=0x7f8ef00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- 172.21.15.12:0/378466494 shutdown_connections 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- 172.21.15.12:0/378466494 wait complete. 2024-03-18T14:47:55.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 Processor -- start 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.026+0000 7f8ef53ad700 1 -- start start 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef01145b0 con 0x7f8ef01037a0 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef010db40 con 0x7f8ef0102e00 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56700/0 (socket says 172.21.15.12:56700) 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 -- 172.21.15.12:0/3495424708 learned_addr learned my addr 172.21.15.12:0/3495424708 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eeeffd700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 -- 172.21.15.12:0/3495424708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 msgr2=0x7f8ef0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef0113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 -- 172.21.15.12:0/3495424708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8005040 con 0x7f8ef01037a0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8eee7fc700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009100 tx=0x7f8ed8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed800e040 con 0x7f8ef01037a0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ed8013420 con 0x7f8ef01037a0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed801c610 con 0x7f8ef01037a0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef010ddc0 con 0x7f8ef01037a0 2024-03-18T14:47:55.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.027+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef010e330 con 0x7f8ef01037a0 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.028+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef0100110 con 0x7f8ef01037a0 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.029+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8ed8013590 con 0x7f8ef01037a0 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.029+0000 7f8ee77fe700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 0x7f8edc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.029+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8ed80880d0 con 0x7f8ef01037a0 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.029+0000 7f8eeeffd700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 0x7f8edc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:55.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.030+0000 7f8eeeffd700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 0x7f8edc06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009f50 tx=0x7f8ee0009590 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:55.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.034+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f8ed8054740 con 0x7f8ef01037a0 2024-03-18T14:47:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.231+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8ef00fe550 con 0x7f8ef01037a0 2024-03-18T14:47:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.232+0000 7f8ee77fe700 1 -- 172.21.15.12:0/3495424708 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ed8057d60 con 0x7f8ef01037a0 2024-03-18T14:47:55.233 INFO:teuthology.orchestra.run.smithi012.stdout:38654705701 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 msgr2=0x7f8edc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 0x7f8edc06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009f50 tx=0x7f8ee0009590 comp rx=0 tx=0).stop 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 msgr2=0x7f8ef0113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009100 tx=0x7f8ed8009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 shutdown_connections 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f8edc0692a0 0x7f8edc06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef0102e00 0x7f8ef0113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 --2- 172.21.15.12:0/3495424708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ef01037a0 0x7f8ef0113ed0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 >> 172.21.15.12:0/3495424708 conn(0x7f8ef00f89b0 msgr2=0x7f8ef00fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 shutdown_connections 2024-03-18T14:47:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:55.234+0000 7f8ef53ad700 1 -- 172.21.15.12:0/3495424708 wait complete. 2024-03-18T14:47:55.545 INFO:teuthology.orchestra.run.smithi012.stdout:81604378649 2024-03-18T14:47:55.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-18T14:47:56.136 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:55 smithi012 ceph-mon[29182]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:56.136 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:55 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3495424708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T14:47:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:55 smithi100 ceph-mon[37807]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:55 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3495424708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T14:47:56.628 INFO:teuthology.orchestra.run.smithi012.stdout:167503724554 2024-03-18T14:47:56.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-18T14:47:57.072 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:57.072 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:57.072 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 -- 172.21.15.12:0/2880857444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e0102de0 msgr2=0x7fa0e01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/2880857444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e0102de0 0x7fa0e01031b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0004940 tx=0x7fa0d0009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 -- 172.21.15.12:0/2880857444 shutdown_connections 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/2880857444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e00fb430 0x7fa0e00fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/2880857444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e0102de0 0x7fa0e01031b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 -- 172.21.15.12:0/2880857444 >> 172.21.15.12:0/2880857444 conn(0x7fa0e0071450 msgr2=0x7fa0e0071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 -- 172.21.15.12:0/2880857444 shutdown_connections 2024-03-18T14:47:57.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.087+0000 7fa0e5f41700 1 -- 172.21.15.12:0/2880857444 wait complete. 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.088+0000 7fa0e5f41700 1 Processor -- start 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.088+0000 7fa0e5f41700 1 -- start start 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0e5f41700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0e5f41700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 0x7fa0e0114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0e5f41700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01146e0 con 0x7fa0e00fb430 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0e5f41700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e010dad0 con 0x7fa0e0102de0 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0deffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 0x7fa0e0114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:57.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:57.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56720/0 (socket says 172.21.15.12:56720) 2024-03-18T14:47:57.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 -- 172.21.15.12:0/4204178964 learned_addr learned my addr 172.21.15.12:0/4204178964 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:47:57.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 -- 172.21.15.12:0/4204178964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 msgr2=0x7fa0e0114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:57.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 0x7fa0e0114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0df7fe700 1 -- 172.21.15.12:0/4204178964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d0005040 con 0x7fa0e00fb430 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.089+0000 7fa0deffd700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 0x7fa0e0114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0df7fe700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000b3b0 tx=0x7fa0d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d000e040 con 0x7fa0e00fb430 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0d0013420 con 0x7fa0e00fb430 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e010dd50 con 0x7fa0e00fb430 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d001c630 con 0x7fa0e00fb430 2024-03-18T14:47:57.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.090+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e010e290 con 0x7fa0e00fb430 2024-03-18T14:47:57.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.091+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e01000a0 con 0x7fa0e00fb430 2024-03-18T14:47:57.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.092+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa0d000baf0 con 0x7fa0e00fb430 2024-03-18T14:47:57.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.092+0000 7fa0c7fff700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 0x7fa0c806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:47:57.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.092+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa0d0088060 con 0x7fa0e00fb430 2024-03-18T14:47:57.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.093+0000 7fa0deffd700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 0x7fa0c806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:47:57.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.093+0000 7fa0deffd700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 0x7fa0c806b820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0e010ec90 tx=0x7fa0d400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:47:57.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.098+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fa0d0058140 con 0x7fa0e00fb430 2024-03-18T14:47:57.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.293+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa0e005f650 con 0x7fa0e00fb430 2024-03-18T14:47:57.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.293+0000 7fa0c7fff700 1 -- 172.21.15.12:0/4204178964 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0d0057cd0 con 0x7fa0e00fb430 2024-03-18T14:47:57.294 INFO:teuthology.orchestra.run.smithi012.stdout:111669149716 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 msgr2=0x7fa0c806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 0x7fa0c806b820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0e010ec90 tx=0x7fa0d400b040 comp rx=0 tx=0).stop 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 msgr2=0x7fa0e0113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000b3b0 tx=0x7fa0d0009e30 comp rx=0 tx=0).stop 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 shutdown_connections 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fa0c8069370 0x7fa0c806b820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e00fb430 0x7fa0e0113b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 --2- 172.21.15.12:0/4204178964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0e0102de0 0x7fa0e0114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 >> 172.21.15.12:0/4204178964 conn(0x7fa0e0071450 msgr2=0x7fa0e00fa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 shutdown_connections 2024-03-18T14:47:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:47:57.295+0000 7fa0e5f41700 1 -- 172.21.15.12:0/4204178964 wait complete. 2024-03-18T14:47:57.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-03-18T14:47:57.829 DEBUG:teuthology.parallel:result is None 2024-03-18T14:47:58.082 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:57 smithi012 ceph-mon[29182]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:58.082 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:57 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4204178964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T14:47:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:57 smithi100 ceph-mon[37807]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:47:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:57 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4204178964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T14:47:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:00.228 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:47:59 smithi012 ceph-mon[29182]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:47:59 smithi100 ceph-mon[37807]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:01.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.745+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/4287460833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 msgr2=0x7f4a88102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:01.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.745+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/4287460833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a88102150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ce0 tx=0x7f4a78004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:01.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/4287460833 shutdown_connections 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/4287460833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 0x7f4a880fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/4287460833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a88102150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/4287460833 >> 172.21.15.12:0/4287460833 conn(0x7f4a880f78f0 msgr2=0x7f4a880f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/4287460833 shutdown_connections 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/4287460833 wait complete. 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 Processor -- start 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.746+0000 7f4a8d7a4700 1 -- start start 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 0x7f4a8810f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810f660 con 0x7f4a88101d80 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810f7a0 con 0x7f4a88102720 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47972/0 (socket says 172.21.15.12:47972) 2024-03-18T14:48:01.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 -- 172.21.15.12:0/2398913108 learned_addr learned my addr 172.21.15.12:0/2398913108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a867fc700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 0x7f4a8810f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 -- 172.21.15.12:0/2398913108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 msgr2=0x7f4a8810f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 0x7f4a8810f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 -- 172.21.15.12:0/2398913108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78009940 con 0x7f4a88101d80 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a86ffd700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800be90 tx=0x7f4a78004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:01.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a7800e040 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a78008ea0 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8810f9d0 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a7801c810 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.747+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a88117ff0 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.748+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880ff090 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.749+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4a78012c00 con 0x7f4a88101d80 2024-03-18T14:48:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.749+0000 7f4a7f7fe700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 0x7f4a7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:01.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.749+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4a78088ac0 con 0x7f4a88101d80 2024-03-18T14:48:01.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.749+0000 7f4a867fc700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 0x7f4a7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:01.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.750+0000 7f4a867fc700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 0x7f4a7406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006fd0 tx=0x7f4a70008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.754+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f4a780550b0 con 0x7f4a88101d80 2024-03-18T14:48:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.953+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4a880667d0 con 0x7f4a88101d80 2024-03-18T14:48:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.953+0000 7f4a7f7fe700 1 -- 172.21.15.12:0/2398913108 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a78004e10 con 0x7f4a88101d80 2024-03-18T14:48:01.955 INFO:teuthology.orchestra.run.smithi012.stdout:146028888080 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 msgr2=0x7f4a7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 0x7f4a7406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006fd0 tx=0x7f4a70008040 comp rx=0 tx=0).stop 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 msgr2=0x7f4a8810ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800be90 tx=0x7f4a78004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 shutdown_connections 2024-03-18T14:48:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4a740692a0 0x7f4a7406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a88101d80 0x7f4a8810ebe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 --2- 172.21.15.12:0/2398913108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a88102720 0x7f4a8810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.955+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 >> 172.21.15.12:0/2398913108 conn(0x7f4a880f78f0 msgr2=0x7f4a880f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.956+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 shutdown_connections 2024-03-18T14:48:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:01.956+0000 7f4a8d7a4700 1 -- 172.21.15.12:0/2398913108 wait complete. 2024-03-18T14:48:01.972 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:01 smithi012 ceph-mon[29182]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:01.972 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:01 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2398913108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T14:48:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:01 smithi100 ceph-mon[37807]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:01 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2398913108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 -- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01701037a0 msgr2=0x7f01700fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01701037a0 0x7f01700fce10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0158004940 tx=0x7f0158009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 -- 172.21.15.12:0/4227384874 shutdown_connections 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01701037a0 0x7f01700fce10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0170102e00 0x7f01701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.794+0000 7f0175079700 1 -- 172.21.15.12:0/4227384874 >> 172.21.15.12:0/4227384874 conn(0x7f01700f89b0 msgr2=0x7f01700fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.795+0000 7f0175079700 1 -- 172.21.15.12:0/4227384874 shutdown_connections 2024-03-18T14:48:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.795+0000 7f0175079700 1 -- 172.21.15.12:0/4227384874 wait complete. 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.795+0000 7f0175079700 1 Processor -- start 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.795+0000 7f0175079700 1 -- start start 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.795+0000 7f0175079700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f0175079700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01701037a0 0x7f0170113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f0175079700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01701145b0 con 0x7f0170102e00 2024-03-18T14:48:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f0175079700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017010db40 con 0x7f01701037a0 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47994/0 (socket says 172.21.15.12:47994) 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 -- 172.21.15.12:0/3112823502 learned_addr learned my addr 172.21.15.12:0/3112823502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016e59c700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01701037a0 0x7f0170113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 -- 172.21.15.12:0/3112823502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01701037a0 msgr2=0x7f0170113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01701037a0 0x7f0170113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 -- 172.21.15.12:0/3112823502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0158005040 con 0x7f0170102e00 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f016ed9d700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f016000bbb0 tx=0x7f0160009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:02.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01600092e0 con 0x7f0170102e00 2024-03-18T14:48:02.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f016000cb20 con 0x7f0170102e00 2024-03-18T14:48:02.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017010de20 con 0x7f0170102e00 2024-03-18T14:48:02.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0160015d70 con 0x7f0170102e00 2024-03-18T14:48:02.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.796+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017010e3b0 con 0x7f0170102e00 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.797+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0170100110 con 0x7f0170102e00 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.798+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0160012070 con 0x7f0170102e00 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.798+0000 7f01677fe700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 0x7f015c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.798+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0160086b90 con 0x7f0170102e00 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.798+0000 7f016e59c700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 0x7f015c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:02.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.799+0000 7f016e59c700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 0x7f015c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0158004890 tx=0x7f0158009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:02.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:02.803+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0160053180 con 0x7f0170102e00 2024-03-18T14:48:03.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.003+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0170109f50 con 0x7f0170102e00 2024-03-18T14:48:03.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.004+0000 7f01677fe700 1 -- 172.21.15.12:0/3112823502 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01600567a0 con 0x7f0170102e00 2024-03-18T14:48:03.005 INFO:teuthology.orchestra.run.smithi012.stdout:167503724556 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 msgr2=0x7f015c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 0x7f015c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0158004890 tx=0x7f0158009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 msgr2=0x7f0170113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f016000bbb0 tx=0x7f0160009840 comp rx=0 tx=0).stop 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 shutdown_connections 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f015c0692a0 0x7f015c06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0170102e00 0x7f0170113990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 --2- 172.21.15.12:0/3112823502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01701037a0 0x7f0170113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 >> 172.21.15.12:0/3112823502 conn(0x7f01700f89b0 msgr2=0x7f01700fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 shutdown_connections 2024-03-18T14:48:03.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:03.006+0000 7f0175079700 1 -- 172.21.15.12:0/3112823502 wait complete. 2024-03-18T14:48:03.619 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724556 for osd.5 2024-03-18T14:48:03.619 DEBUG:teuthology.parallel:result is None 2024-03-18T14:48:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:03 smithi012 ceph-mon[29182]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:03 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3112823502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T14:48:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:03 smithi100 ceph-mon[37807]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:03 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3112823502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T14:48:04.389 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149716 for osd.3 2024-03-18T14:48:04.389 DEBUG:teuthology.parallel:result is None 2024-03-18T14:48:04.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888080 for osd.4 2024-03-18T14:48:04.789 DEBUG:teuthology.parallel:result is None 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.467+0000 7f617e2b4700 1 -- 172.21.15.12:0/3652303192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 msgr2=0x7f61781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.467+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3652303192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61781031d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6168004940 tx=0x7f6168009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 -- 172.21.15.12:0/3652303192 shutdown_connections 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3652303192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 0x7f61780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3652303192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61781031d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 -- 172.21.15.12:0/3652303192 >> 172.21.15.12:0/3652303192 conn(0x7f61780f89b0 msgr2=0x7f61780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 -- 172.21.15.12:0/3652303192 shutdown_connections 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 -- 172.21.15.12:0/3652303192 wait complete. 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 Processor -- start 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.468+0000 7f617e2b4700 1 -- start start 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f617e2b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f617e2b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 0x7f6178070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f617e2b4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780721c0 con 0x7f6178102e00 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f617e2b4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178070d40 con 0x7f61781037a0 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48006/0 (socket says 172.21.15.12:48006) 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 -- 172.21.15.12:0/3460619154 learned_addr learned my addr 172.21.15.12:0/3460619154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f61777fe700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 0x7f6178070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 -- 172.21.15.12:0/3460619154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 msgr2=0x7f6178070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 0x7f6178070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 -- 172.21.15.12:0/3460619154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168005040 con 0x7f6178102e00 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6177fff700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f616800b3b0 tx=0x7f6168009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.469+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f616800e040 con 0x7f6178102e00 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.470+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6168013420 con 0x7f6178102e00 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.470+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6178070fc0 con 0x7f6178102e00 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.470+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f616801c630 con 0x7f6178102e00 2024-03-18T14:48:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.470+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617811a150 con 0x7f6178102e00 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.471+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6178100110 con 0x7f6178102e00 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.471+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6168013590 con 0x7f6178102e00 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.471+0000 7f6174ff9700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 0x7f616006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.472+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6168087e50 con 0x7f6178102e00 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.472+0000 7f61777fe700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 0x7f616006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.472+0000 7f61777fe700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 0x7f616006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61780719a0 tx=0x7f616c0043c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.476+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f6168054560 con 0x7f6178102e00 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 -- 172.21.15.12:0/378453863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970102710 msgr2=0x7f19700fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 --2- 172.21.15.12:0/378453863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970102710 0x7f19700fbd80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f196c004940 tx=0x7f196c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 -- 172.21.15.12:0/378453863 shutdown_connections 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 --2- 172.21.15.12:0/378453863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970102710 0x7f19700fbd80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 --2- 172.21.15.12:0/378453863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970101d70 0x7f1970102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 -- 172.21.15.12:0/378453863 >> 172.21.15.12:0/378453863 conn(0x7f1970072bc0 msgr2=0x7f1970072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 -- 172.21.15.12:0/378453863 shutdown_connections 2024-03-18T14:48:05.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.663+0000 7f1978310700 1 -- 172.21.15.12:0/378453863 wait complete. 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 Processor -- start 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 -- start start 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970102710 0x7f197010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197010d390 con 0x7f1970101d70 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f1978310700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197010d500 con 0x7f1970102710 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48020/0 (socket says 172.21.15.12:48020) 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 -- 172.21.15.12:0/3289980907 learned_addr learned my addr 172.21.15.12:0/3289980907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19758ab700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970102710 0x7f197010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 -- 172.21.15.12:0/3289980907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970102710 msgr2=0x7f197010ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970102710 0x7f197010ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 -- 172.21.15.12:0/3289980907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c005040 con 0x7f1970101d70 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.664+0000 7f19760ac700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1964002a50 tx=0x7f19640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.665+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1964007cc0 con 0x7f1970101d70 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.665+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f196400f040 con 0x7f1970101d70 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.665+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197010d790 con 0x7f1970101d70 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.665+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f196400da30 con 0x7f1970101d70 2024-03-18T14:48:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.665+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197011a320 con 0x7f1970101d70 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.666+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970104240 con 0x7f1970101d70 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.666+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f196400db90 con 0x7f1970101d70 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.666+0000 7f1962ffd700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 0x7f195c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.667+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1964086ab0 con 0x7f1970101d70 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.667+0000 7f19758ab700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 0x7f195c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.667+0000 7f19758ab700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 0x7f195c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f196c004890 tx=0x7f196c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:05.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.671+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6178071810 con 0x7f6178102e00 2024-03-18T14:48:05.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.671+0000 7f6174ff9700 1 -- 172.21.15.12:0/3460619154 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6168013840 con 0x7f6178102e00 2024-03-18T14:48:05.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.671+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f1964056bb0 con 0x7f1970101d70 2024-03-18T14:48:05.672 INFO:teuthology.orchestra.run.smithi012.stdout:60129542178 2024-03-18T14:48:05.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 msgr2=0x7f616006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 0x7f616006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61780719a0 tx=0x7f616c0043c0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 msgr2=0x7f61780702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f616800b3b0 tx=0x7f6168009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 shutdown_connections 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f61600692a0 0x7f616006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6178102e00 0x7f61780702c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 --2- 172.21.15.12:0/3460619154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61781037a0 0x7f6178070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 >> 172.21.15.12:0/3460619154 conn(0x7f61780f89b0 msgr2=0x7f61781073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 shutdown_connections 2024-03-18T14:48:05.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.673+0000 7f617e2b4700 1 -- 172.21.15.12:0/3460619154 wait complete. 2024-03-18T14:48:05.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.868+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19700fd4c0 con 0x7f1970101d70 2024-03-18T14:48:05.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.868+0000 7f1962ffd700 1 -- 172.21.15.12:0/3289980907 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1964056740 con 0x7f1970101d70 2024-03-18T14:48:05.869 INFO:teuthology.orchestra.run.smithi012.stdout:81604378652 2024-03-18T14:48:05.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 msgr2=0x7f195c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 0x7f195c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f196c004890 tx=0x7f196c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 msgr2=0x7f197010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1964002a50 tx=0x7f19640046b0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 shutdown_connections 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f195c0692a0 0x7f195c06b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1970101d70 0x7f197010c910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 --2- 172.21.15.12:0/3289980907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970102710 0x7f197010ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 >> 172.21.15.12:0/3289980907 conn(0x7f1970072bc0 msgr2=0x7f19700fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 shutdown_connections 2024-03-18T14:48:05.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:05.870+0000 7f1978310700 1 -- 172.21.15.12:0/3289980907 wait complete. 2024-03-18T14:48:06.195 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:05 smithi012 ceph-mon[29182]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:06.195 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:05 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3460619154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T14:48:06.195 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:06.195 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:05 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3289980907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T14:48:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:05 smithi100 ceph-mon[37807]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:05 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3460619154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T14:48:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:05 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3289980907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T14:48:06.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378652 for osd.2 2024-03-18T14:48:06.497 DEBUG:teuthology.parallel:result is None 2024-03-18T14:48:06.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542178 for osd.1 2024-03-18T14:48:06.878 DEBUG:teuthology.parallel:result is None 2024-03-18T14:48:06.878 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-18T14:48:06.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T14:48:07.145 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:07 smithi012 ceph-mon[29182]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:07 smithi100 ceph-mon[37807]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 -- 172.21.15.12:0/203170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 msgr2=0x7fec1c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 --2- 172.21.15.12:0/203170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c106dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fec04004940 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 -- 172.21.15.12:0/203170037 shutdown_connections 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 --2- 172.21.15.12:0/203170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c106dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 --2- 172.21.15.12:0/203170037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 0x7fec1c0fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 -- 172.21.15.12:0/203170037 >> 172.21.15.12:0/203170037 conn(0x7fec1c0f7890 msgr2=0x7fec1c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 -- 172.21.15.12:0/203170037 shutdown_connections 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.783+0000 7fec21906700 1 -- 172.21.15.12:0/203170037 wait complete. 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 Processor -- start 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 -- start start 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 0x7fec1c0fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0fdb60 con 0x7fec1c0fe780 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec21906700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0fc690 con 0x7fec1c0fdde0 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35934/0 (socket says 172.21.15.12:35934) 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 -- 172.21.15.12:0/1528382591 learned_addr learned my addr 172.21.15.12:0/1528382591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec1affd700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 0x7fec1c0fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 -- 172.21.15.12:0/1528382591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 msgr2=0x7fec1c0fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 0x7fec1c0fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.784+0000 7fec13fff700 1 -- 172.21.15.12:0/1528382591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04005040 con 0x7fec1c0fe780 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec13fff700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec04004890 tx=0x7fec04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0400e040 con 0x7fec1c0fe780 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0fc910 con 0x7fec1c0fe780 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec04009af0 con 0x7fec1c0fe780 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0401c630 con 0x7fec1c0fe780 2024-03-18T14:48:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.785+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c119ff0 con 0x7fec1c0fe780 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.786+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c066bd0 con 0x7fec1c0fe780 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.786+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fec0400bbc0 con 0x7fec1c0fe780 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.787+0000 7fec137fe700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.787+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fec04088140 con 0x7fec1c0fe780 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.787+0000 7fec1affd700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.787+0000 7fec1affd700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 0x7fec0806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec0c007fd0 tx=0x7fec0c009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.791+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fec040547b0 con 0x7fec1c0fe780 2024-03-18T14:48:08.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.976+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec1c109f10 con 0x7fec080692a0 2024-03-18T14:48:08.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.977+0000 7fec137fe700 1 -- 172.21.15.12:0/1528382591 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28170 (secure 0 0 0) 0x7fec0c004db0 con 0x7fec080692a0 2024-03-18T14:48:08.979 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:08.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 msgr2=0x7fec0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:08.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 0x7fec0806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec0c007fd0 tx=0x7fec0c009040 comp rx=0 tx=0).stop 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 msgr2=0x7fec1c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec04004890 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 shutdown_connections 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fec080692a0 0x7fec0806b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0fdde0 0x7fec1c0fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 --2- 172.21.15.12:0/1528382591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec1c0fe780 0x7fec1c0fc150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 >> 172.21.15.12:0/1528382591 conn(0x7fec1c0f7890 msgr2=0x7fec1c0f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 shutdown_connections 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:08.979+0000 7fec21906700 1 -- 172.21.15.12:0/1528382591 wait complete. 2024-03-18T14:48:08.981 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-03-18T14:48:09.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-03-18T14:48:08.807938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002013"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-18T14:47:15.854126+0000","last_change":"2024-03-18T14:46:51.156056+0000","last_active":"2024-03-18T14:47:15.854126+0000","last_peered":"2024-03-18T14:47:15.854126+0000","last_clean":"2024-03-18T14:47:15.854126+0000","last_became_active":"2024-03-18T14:46:51.155764+0000","last_became_peered":"2024-03-18T14:46:51.155764+0000","last_unstale":"2024-03-18T14:47:15.854126+0000","last_undegraded":"2024-03-18T14:47:15.854126+0000","last_fullsized":"2024-03-18T14:47:15.854126+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T14:46:03.096098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T14:46:03.096098+0000","last_clean_scrub_stamp":"2024-03-18T14:46:03.096098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:37:11.131418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446932,"statfs":{"total":95995035648,"available":95689658368,"internally_reserved":0,"allocated":954368,"data_stored":729238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Mon Mar 18 14:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.35599999999999998,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.42099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38600000000000001}]},{"osd":1,"last update":"Mon Mar 18 14:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.41199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Mon Mar 18 14:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Mar 18 14:47:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 14:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.496,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.251,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Mar 18 14:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.317},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.48999999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Mar 18 14:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.25,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.47499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34100000000000003,"15min":0.33700000000000002},"min":{"1min":0.30099999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Mar 18 14:47:03 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Mar 18 14:47:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Mon Mar 18 14:47:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T14:48:09.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T14:48:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:10.223 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:09 smithi012 ceph-mon[29182]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:10.223 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:09 smithi012 ceph-mon[29182]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T14:48:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:09 smithi100 ceph-mon[37807]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:09 smithi100 ceph-mon[37807]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T14:48:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 -- 172.21.15.12:0/3157173860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 msgr2=0x7fb6840fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 --2- 172.21.15.12:0/3157173860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb6840fbd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb678004940 tx=0x7fb678009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 -- 172.21.15.12:0/3157173860 shutdown_connections 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 --2- 172.21.15.12:0/3157173860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb6840fbd80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 --2- 172.21.15.12:0/3157173860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 0x7fb684102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 -- 172.21.15.12:0/3157173860 >> 172.21.15.12:0/3157173860 conn(0x7fb684072bc0 msgr2=0x7fb684072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 -- 172.21.15.12:0/3157173860 shutdown_connections 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.489+0000 7fb688e69700 1 -- 172.21.15.12:0/3157173860 wait complete. 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 Processor -- start 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 -- start start 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 0x7fb684112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:11.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684113630 con 0x7fb684102710 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb688e69700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68410ca20 con 0x7fb684101d70 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35946/0 (socket says 172.21.15.12:35946) 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 -- 172.21.15.12:0/380973903 learned_addr learned my addr 172.21.15.12:0/380973903 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb68259c700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 0x7fb684112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 -- 172.21.15.12:0/380973903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 msgr2=0x7fb684112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 0x7fb684112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.490+0000 7fb681d9b700 1 -- 172.21.15.12:0/380973903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678005040 con 0x7fb684102710 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb681d9b700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb678009180 tx=0x7fb678009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67800e040 con 0x7fb684102710 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68410cca0 con 0x7fb684102710 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb67800bbc0 con 0x7fb684102710 2024-03-18T14:48:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67801c3f0 con 0x7fb684102710 2024-03-18T14:48:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.491+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68410d260 con 0x7fb684102710 2024-03-18T14:48:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.492+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb684104240 con 0x7fb684102710 2024-03-18T14:48:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.492+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb67800bd30 con 0x7fb684102710 2024-03-18T14:48:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.492+0000 7fb672ffd700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.493+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb678088190 con 0x7fb684102710 2024-03-18T14:48:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.493+0000 7fb68259c700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.493+0000 7fb68259c700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 0x7fb66c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb674007fd0 tx=0x7fb674009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.497+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb678058290 con 0x7fb684102710 2024-03-18T14:48:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.683+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb68405f660 con 0x7fb66c0692a0 2024-03-18T14:48:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.685+0000 7fb672ffd700 1 -- 172.21.15.12:0/380973903 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28228 (secure 0 0 0) 0x7fb68405f660 con 0x7fb66c0692a0 2024-03-18T14:48:11.686 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 msgr2=0x7fb66c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 0x7fb66c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb674007fd0 tx=0x7fb674009040 comp rx=0 tx=0).stop 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 msgr2=0x7fb684112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb678009180 tx=0x7fb678009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 shutdown_connections 2024-03-18T14:48:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb684101d70 0x7fb684112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 --2- 172.21.15.12:0/380973903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684102710 0x7fb684112fe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 >> 172.21.15.12:0/380973903 conn(0x7fb684072bc0 msgr2=0x7fb6840fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 shutdown_connections 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:11.687+0000 7fb688e69700 1 -- 172.21.15.12:0/380973903 wait complete. 2024-03-18T14:48:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-03-18T14:48:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:11 smithi012 ceph-mon[29182]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:11 smithi100 ceph-mon[37807]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:12.422 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-18T14:48:10.808248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002013"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-18T14:47:15.854126+0000","last_change":"2024-03-18T14:46:51.156056+0000","last_active":"2024-03-18T14:47:15.854126+0000","last_peered":"2024-03-18T14:47:15.854126+0000","last_clean":"2024-03-18T14:47:15.854126+0000","last_became_active":"2024-03-18T14:46:51.155764+0000","last_became_peered":"2024-03-18T14:46:51.155764+0000","last_unstale":"2024-03-18T14:47:15.854126+0000","last_undegraded":"2024-03-18T14:47:15.854126+0000","last_fullsized":"2024-03-18T14:47:15.854126+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T14:46:03.096098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T14:46:03.096098+0000","last_clean_scrub_stamp":"2024-03-18T14:46:03.096098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:37:11.131418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446932,"statfs":{"total":95995035648,"available":95689658368,"internally_reserved":0,"allocated":954368,"data_stored":729238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Mar 18 14:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Mon Mar 18 14:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.35599999999999998,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.42099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38600000000000001}]},{"osd":1,"last update":"Mon Mar 18 14:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.41199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Mon Mar 18 14:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Mar 18 14:47:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 14:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.496,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.251,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Mar 18 14:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.504,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.317},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.48999999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Mar 18 14:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Mar 18 14:47:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 14:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.25,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.47499999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34100000000000003,"15min":0.33700000000000002},"min":{"1min":0.30099999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.308}]},{"osd":2,"last update":"Mon Mar 18 14:48:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.376,"15min":0.378},"min":{"1min":0.27600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.27600000000000002}]},{"osd":3,"last update":"Mon Mar 18 14:47:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.505},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Mar 18 14:47:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T14:48:12.423 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-18T14:48:12.423 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T14:48:12.423 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-18T14:48:12.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-18T14:48:12.691 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:13.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:12 smithi012 ceph-mon[29182]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T14:48:13.293 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:12 smithi100 ceph-mon[37807]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T14:48:14.144 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:13 smithi012 ceph-mon[29182]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:13 smithi100 ceph-mon[37807]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:14.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 -- 172.21.15.12:0/1461455975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 msgr2=0x7f14d80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:14.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 --2- 172.21.15.12:0/1461455975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d80fce10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f14cc004940 tx=0x7f14cc009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:14.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 -- 172.21.15.12:0/1461455975 shutdown_connections 2024-03-18T14:48:14.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 --2- 172.21.15.12:0/1461455975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d80fce10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 --2- 172.21.15.12:0/1461455975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 0x7f14d81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 -- 172.21.15.12:0/1461455975 >> 172.21.15.12:0/1461455975 conn(0x7f14d80f89b0 msgr2=0x7f14d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:14.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 -- 172.21.15.12:0/1461455975 shutdown_connections 2024-03-18T14:48:14.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.404+0000 7f14de4be700 1 -- 172.21.15.12:0/1461455975 wait complete. 2024-03-18T14:48:14.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 Processor -- start 2024-03-18T14:48:14.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 -- start start 2024-03-18T14:48:14.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 0x7f14d8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:14.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d8114510 con 0x7f14d81037a0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14de4be700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d810db10 con 0x7f14d8102e00 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14d77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35974/0 (socket says 172.21.15.12:35974) 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14d77fe700 1 -- 172.21.15.12:0/2808217559 learned_addr learned my addr 172.21.15.12:0/2808217559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.405+0000 7f14d7fff700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 0x7f14d8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d77fe700 1 -- 172.21.15.12:0/2808217559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 msgr2=0x7f14d8113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d77fe700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 0x7f14d8113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d77fe700 1 -- 172.21.15.12:0/2808217559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc005040 con 0x7f14d81037a0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d77fe700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009100 tx=0x7f14cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14cc00e040 con 0x7f14d81037a0 2024-03-18T14:48:14.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14cc009af0 con 0x7f14d81037a0 2024-03-18T14:48:14.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d810dd90 con 0x7f14d81037a0 2024-03-18T14:48:14.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14cc01c3f0 con 0x7f14d81037a0 2024-03-18T14:48:14.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.406+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d810e350 con 0x7f14d81037a0 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.407+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d81052d0 con 0x7f14d81037a0 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.407+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f14cc00baf0 con 0x7f14d81037a0 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.408+0000 7f14d4ff9700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 0x7f14c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.408+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f14cc088120 con 0x7f14d81037a0 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.408+0000 7f14d7fff700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 0x7f14c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:14.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.408+0000 7f14d7fff700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 0x7f14c006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14c8004ed0 tx=0x7f14c800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.412+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f14cc058220 con 0x7f14d81037a0 2024-03-18T14:48:14.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.646+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f14d810efd0 con 0x7f14d81037a0 2024-03-18T14:48:14.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.647+0000 7f14d4ff9700 1 -- 172.21.15.12:0/2808217559 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f14cc057db0 con 0x7f14d81037a0 2024-03-18T14:48:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:14.648 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T14:48:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 msgr2=0x7f14c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 0x7f14c006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14c8004ed0 tx=0x7f14c800b040 comp rx=0 tx=0).stop 2024-03-18T14:48:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 msgr2=0x7f14d8113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009100 tx=0x7f14cc009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:14.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 shutdown_connections 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f14c00692a0 0x7f14c006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d8102e00 0x7f14d8113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 --2- 172.21.15.12:0/2808217559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14d81037a0 0x7f14d8113ec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 >> 172.21.15.12:0/2808217559 conn(0x7f14d80f89b0 msgr2=0x7f14d8102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 shutdown_connections 2024-03-18T14:48:14.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:14.649+0000 7f14de4be700 1 -- 172.21.15.12:0/2808217559 wait complete. 2024-03-18T14:48:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:14 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2808217559' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T14:48:15.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:14 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2808217559' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T14:48:15.512 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-18T14:48:15.512 INFO:tasks.cephadm:Setup complete, yielding 2024-03-18T14:48:15.513 INFO:teuthology.run_tasks:Running task print... 2024-03-18T14:48:15.523 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-18T14:48:15.523 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:15.539 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:15.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-18T14:48:15.807 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:15 smithi012 ceph-mon[29182]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:15 smithi100 ceph-mon[37807]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:17.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2372845577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fcf90 msgr2=0x7fc6bc0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2372845577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc0fd400 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4004940 tx=0x7fc6a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2372845577 shutdown_connections 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2372845577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc0fd400 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2372845577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2372845577 >> 172.21.15.12:0/2372845577 conn(0x7fc6bc0f7440 msgr2=0x7fc6bc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2372845577 shutdown_connections 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2372845577 wait complete. 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 Processor -- start 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.389+0000 7fc6c0eaf700 1 -- start start 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc112770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc112cb0 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc112e20 con 0x7fc6bc0fcf90 2024-03-18T14:48:17.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:17.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35982/0 (socket says 172.21.15.12:35982) 2024-03-18T14:48:17.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 -- 172.21.15.12:0/2565196263 learned_addr learned my addr 172.21.15.12:0/2565196263 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:17.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6b9d9b700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc112770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:17.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 -- 172.21.15.12:0/2565196263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fcf90 msgr2=0x7fc6bc112770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:17.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc112770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 -- 172.21.15.12:0/2565196263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4005040 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6ba59c700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00bbb0 tx=0x7fc6ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6ac0092e0 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6ac00cbd0 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc113100 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6ac007400 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.390+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1136c0 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.391+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc04c710 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.392+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc6ac012070 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.392+0000 7fc6b2ffd700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 0x7fc6a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.392+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc6ac086a00 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.392+0000 7fc6b9d9b700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 0x7fc6a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:17.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.393+0000 7fc6b9d9b700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 0x7fc6a806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b3b0 tx=0x7fc6a4009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:17.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.397+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc6ac056b00 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.591+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fc6bc104850 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.595+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2565196263 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc6ac056690 con 0x7fc6bc0fbce0 2024-03-18T14:48:17.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 msgr2=0x7fc6a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 0x7fc6a806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b3b0 tx=0x7fc6a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 msgr2=0x7fc6bc066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00bbb0 tx=0x7fc6ac009840 comp rx=0 tx=0).stop 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 shutdown_connections 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc6a80692a0 0x7fc6a806b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6bc0fbce0 0x7fc6bc066d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 --2- 172.21.15.12:0/2565196263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0fcf90 0x7fc6bc112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.597+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 >> 172.21.15.12:0/2565196263 conn(0x7fc6bc0f7440 msgr2=0x7fc6bc0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.598+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 shutdown_connections 2024-03-18T14:48:17.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:17.598+0000 7fc6c0eaf700 1 -- 172.21.15.12:0/2565196263 wait complete. 2024-03-18T14:48:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:17 smithi012 ceph-mon[29182]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:17 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2565196263' entity='client.admin' 2024-03-18T14:48:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:17 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:18.253 INFO:teuthology.run_tasks:Running task print... 2024-03-18T14:48:18.268 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-18T14:48:18.268 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:18.292 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:18.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-18T14:48:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:17 smithi100 ceph-mon[37807]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:17 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2565196263' entity='client.admin' 2024-03-18T14:48:18.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:17 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:18.550 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:18 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:18 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:20.060 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:20 smithi012 ceph-mon[29182]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:20.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 -- 172.21.15.12:0/171179895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 msgr2=0x7fc6800fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 --2- 172.21.15.12:0/171179895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc6800fe810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc67c004940 tx=0x7fc67c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 -- 172.21.15.12:0/171179895 shutdown_connections 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 --2- 172.21.15.12:0/171179895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc6800fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 --2- 172.21.15.12:0/171179895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc6800fe810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 -- 172.21.15.12:0/171179895 >> 172.21.15.12:0/171179895 conn(0x7fc6800f6ba0 msgr2=0x7fc6800f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 -- 172.21.15.12:0/171179895 shutdown_connections 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.156+0000 7fc687378700 1 -- 172.21.15.12:0/171179895 wait complete. 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 Processor -- start 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 -- start start 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc68010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:20.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68010e530 con 0x7fc6800fe400 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc687378700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68010ed80 con 0x7fc6800fed50 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34172/0 (socket says 172.21.15.12:34172) 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 -- 172.21.15.12:0/340878103 learned_addr learned my addr 172.21.15.12:0/340878103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc684913700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc68010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 -- 172.21.15.12:0/340878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 msgr2=0x7fc68010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc68010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 -- 172.21.15.12:0/340878103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c005040 con 0x7fc6800fe400 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc684913700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc68010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.157+0000 7fc685114700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009150 tx=0x7fc67c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.158+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67c00e040 con 0x7fc6800fe400 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.158+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc67c00bbf0 con 0x7fc6800fe400 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.158+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68010f000 con 0x7fc6800fe400 2024-03-18T14:48:20.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.158+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67c01c630 con 0x7fc6800fe400 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.158+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68010f540 con 0x7fc6800fe400 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.159+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68005f5f0 con 0x7fc6800fe400 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.159+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc67c013420 con 0x7fc6800fe400 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.159+0000 7fc671ffb700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 0x7fc66c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.160+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc67c0882e0 con 0x7fc6800fe400 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.160+0000 7fc684913700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 0x7fc66c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.160+0000 7fc684913700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 0x7fc66c06b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc674007fd0 tx=0x7fc674009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:20.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.165+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc67c0583e0 con 0x7fc6800fe400 2024-03-18T14:48:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:20 smithi100 ceph-mon[37807]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:20.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.375+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc68010f800 con 0x7fc66c0692b0 2024-03-18T14:48:20.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.376+0000 7fc671ffb700 1 -- 172.21.15.12:0/340878103 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc68010f800 con 0x7fc66c0692b0 2024-03-18T14:48:20.377 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-03-18T14:48:20.377 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-03-18T14:48:20.378 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-03-18T14:48:20.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 msgr2=0x7fc66c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 0x7fc66c06b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc674007fd0 tx=0x7fc674009040 comp rx=0 tx=0).stop 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 msgr2=0x7fc68010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009150 tx=0x7fc67c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 shutdown_connections 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc66c0692b0 0x7fc66c06b760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6800fe400 0x7fc68010d9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 --2- 172.21.15.12:0/340878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6800fed50 0x7fc68010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 >> 172.21.15.12:0/340878103 conn(0x7fc6800f6ba0 msgr2=0x7fc6800f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 shutdown_connections 2024-03-18T14:48:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:20.378+0000 7fc687378700 1 -- 172.21.15.12:0/340878103 wait complete. 2024-03-18T14:48:21.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-18T14:48:21.312 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:21.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:21 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:21 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:22.384 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:22 smithi012 ceph-mon[29182]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:22.384 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:22 smithi012 ceph-mon[29182]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:22.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:22 smithi100 ceph-mon[37807]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:22.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:22 smithi100 ceph-mon[37807]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:22.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- 172.21.15.12:0/3811324299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 msgr2=0x7f87fc0fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 --2- 172.21.15.12:0/3811324299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc0fe850 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87f4004940 tx=0x7f87f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- 172.21.15.12:0/3811324299 shutdown_connections 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 --2- 172.21.15.12:0/3811324299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 0x7f87fc0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 --2- 172.21.15.12:0/3811324299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc0fe850 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- 172.21.15.12:0/3811324299 >> 172.21.15.12:0/3811324299 conn(0x7f87fc0f6be0 msgr2=0x7f87fc0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- 172.21.15.12:0/3811324299 shutdown_connections 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- 172.21.15.12:0/3811324299 wait complete. 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 Processor -- start 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.887+0000 7f8800ee2700 1 -- start start 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 0x7f87fc10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:22.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc10e480 con 0x7f87fc0fe440 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc10e5f0 con 0x7f87fc0fed90 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34184/0 (socket says 172.21.15.12:34184) 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 -- 172.21.15.12:0/4012337605 learned_addr learned my addr 172.21.15.12:0/4012337605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87f9d9b700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 0x7f87fc10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 -- 172.21.15.12:0/4012337605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 msgr2=0x7f87fc10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 0x7f87fc10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 -- 172.21.15.12:0/4012337605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4005040 con 0x7f87fc0fe440 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87fa59c700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b3b0 tx=0x7f87f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f400e040 con 0x7f87fc0fe440 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f400bbf0 con 0x7f87fc0fe440 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc112ff0 con 0x7f87fc0fe440 2024-03-18T14:48:22.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f401c630 con 0x7f87fc0fe440 2024-03-18T14:48:22.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.888+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc113520 con 0x7f87fc0fe440 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.889+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc04c710 con 0x7f87fc0fe440 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.890+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f87f400bd60 con 0x7f87fc0fe440 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.890+0000 7f87f2ffd700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 0x7f87e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.890+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f87f4087b00 con 0x7f87fc0fe440 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.890+0000 7f87f9d9b700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 0x7f87e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:22.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.891+0000 7f87f9d9b700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 0x7f87e806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87e4004f40 tx=0x7f87e400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:22.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:22.895+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f87f4057c00 con 0x7f87fc0fe440 2024-03-18T14:48:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.097+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f87fc113740 con 0x7f87e80692a0 2024-03-18T14:48:23.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.105+0000 7f87f2ffd700 1 -- 172.21.15.12:0/4012337605 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f87fc113740 con 0x7f87e80692a0 2024-03-18T14:48:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (3m) 2m ago 5m 25.8M - 0.25.0 c8568f914cd2 3a6187ef0acb 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (5m) 2m ago 5m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (4m) 68s ago 4m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (3m) 2m ago 4m 74.5M - 9.4.7 954c08fa6188 a14a00878205 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (4m) 68s ago 4m 433M - 17.2.7-684-g6716c9f3 19857db3290c 47f9c2dcc3f5 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (6m) 2m ago 6m 40.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (4m) 68s ago 4m 38.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (5m) 2m ago 5m 14.7M - 1.5.0 0da6a335fe13 4ebf4cd84beb 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (4m) 68s ago 4m 17.4M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (3m) 2m ago 3m 38.7M 4289M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (2m) 2m ago 2m 40.2M 4289M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:48:23.107 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (2m) 2m ago 2m 41.0M 4289M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:48:23.108 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (2m) 68s ago 2m 41.5M 5312M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:48:23.108 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (98s) 68s ago 98s 36.5M 5312M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:48:23.108 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (73s) 68s ago 73s 10.3M 5312M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:48:23.108 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (3m) 2m ago 4m 37.7M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 msgr2=0x7f87e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 0x7f87e806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87e4004f40 tx=0x7f87e400b040 comp rx=0 tx=0).stop 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 msgr2=0x7f87fc10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b3b0 tx=0x7f87f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 shutdown_connections 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f87e80692a0 0x7f87e806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87fc0fe440 0x7f87fc10d940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 --2- 172.21.15.12:0/4012337605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0fed90 0x7f87fc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 >> 172.21.15.12:0/4012337605 conn(0x7f87fc0f6be0 msgr2=0x7f87fc0f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 shutdown_connections 2024-03-18T14:48:23.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:23.107+0000 7f8800ee2700 1 -- 172.21.15.12:0/4012337605 wait complete. 2024-03-18T14:48:23.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-18T14:48:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:24.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:24 smithi012 ceph-mon[29182]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:24.237 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:24 smithi012 ceph-mon[29182]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:24 smithi100 ceph-mon[37807]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:24 smithi100 ceph-mon[37807]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 -- 172.21.15.12:0/1945926677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 msgr2=0x7f33500fd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 --2- 172.21.15.12:0/1945926677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f33500fd040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3340009ce0 tx=0x7f3340004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 -- 172.21.15.12:0/1945926677 shutdown_connections 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 --2- 172.21.15.12:0/1945926677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500fdee0 0x7f33500fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 --2- 172.21.15.12:0/1945926677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f33500fd040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 -- 172.21.15.12:0/1945926677 >> 172.21.15.12:0/1945926677 conn(0x7f33500f8390 msgr2=0x7f33500fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 -- 172.21.15.12:0/1945926677 shutdown_connections 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 -- 172.21.15.12:0/1945926677 wait complete. 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.824+0000 7f334f59e700 1 Processor -- start 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 -- start start 2024-03-18T14:48:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500fdee0 0x7f3350070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350070930 con 0x7f33500fcc30 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350070a70 con 0x7f33500fdee0 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34206/0 (socket says 172.21.15.12:34206) 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 -- 172.21.15.12:0/2547865083 learned_addr learned my addr 172.21.15.12:0/2547865083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 -- 172.21.15.12:0/2547865083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500fdee0 msgr2=0x7f3350070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500fdee0 0x7f3350070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 -- 172.21.15.12:0/2547865083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009940 con 0x7f33500fcc30 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334e59c700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3340004740 tx=0x7f3340004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334000e040 con 0x7f33500fcc30 2024-03-18T14:48:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3350070cc0 con 0x7f33500fcc30 2024-03-18T14:48:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33400128e0 con 0x7f33500fcc30 2024-03-18T14:48:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334001ca80 con 0x7f33500fcc30 2024-03-18T14:48:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.825+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501159f0 con 0x7f33500fcc30 2024-03-18T14:48:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.826+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335005f5f0 con 0x7f33500fcc30 2024-03-18T14:48:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.827+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3340012a50 con 0x7f33500fcc30 2024-03-18T14:48:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.827+0000 7f3346ffd700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 0x7f333c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.827+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3340088b30 con 0x7f33500fcc30 2024-03-18T14:48:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.827+0000 7f334dd9b700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 0x7f333c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.828+0000 7f334dd9b700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 0x7f333c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3338006fd0 tx=0x7f3338006db0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:25.832+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3340055120 con 0x7f33500fcc30 2024-03-18T14:48:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.034+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3350102180 con 0x7f333c0692a0 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.039+0000 7f3346ffd700 1 -- 172.21.15.12:0/2547865083 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f3350102180 con 0x7f333c0692a0 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:crash 2/2 2m ago 6m * 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 2/2 2m ago 6m count:2 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:mon 2/2 2m ago 5m smithi012:172.21.15.12=smithi012;smithi100:172.21.15.100=smithi100;count:2 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:osd 6 2m ago - 2024-03-18T14:48:26.040 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-18T14:48:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 msgr2=0x7f333c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 0x7f333c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3338006fd0 tx=0x7f3338006db0 comp rx=0 tx=0).stop 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 msgr2=0x7f3350071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3340004740 tx=0x7f3340004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 shutdown_connections 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f333c0692a0 0x7f333c06b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33500fcc30 0x7f3350071dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 --2- 172.21.15.12:0/2547865083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500fdee0 0x7f3350070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 >> 172.21.15.12:0/2547865083 conn(0x7f33500f8390 msgr2=0x7f33500f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 shutdown_connections 2024-03-18T14:48:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:26.041+0000 7f334f59e700 1 -- 172.21.15.12:0/2547865083 wait complete. 2024-03-18T14:48:26.362 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:26 smithi100 ceph-mon[37807]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:26.414 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:26 smithi012 ceph-mon[29182]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:26.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-18T14:48:27.062 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:27.309 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:27 smithi012 ceph-mon[29182]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:27.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:27 smithi100 ceph-mon[37807]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:28.377 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:28 smithi012 ceph-mon[29182]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:28 smithi100 ceph-mon[37807]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 -- 172.21.15.12:0/3951320680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fcf80 msgr2=0x7ff1fc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 --2- 172.21.15.12:0/3951320680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc0fd3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8004940 tx=0x7ff1e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 -- 172.21.15.12:0/3951320680 shutdown_connections 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 --2- 172.21.15.12:0/3951320680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc0fd3f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 --2- 172.21.15.12:0/3951320680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 -- 172.21.15.12:0/3951320680 >> 172.21.15.12:0/3951320680 conn(0x7ff1fc0f7430 msgr2=0x7ff1fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 -- 172.21.15.12:0/3951320680 shutdown_connections 2024-03-18T14:48:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.596+0000 7ff202eff700 1 -- 172.21.15.12:0/3951320680 wait complete. 2024-03-18T14:48:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 Processor -- start 2024-03-18T14:48:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 -- start start 2024-03-18T14:48:28.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc10cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc10d410 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff202eff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc111db0 con 0x7ff1fc0fcf80 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff200c9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff200c9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51636/0 (socket says 172.21.15.12:51636) 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.597+0000 7ff200c9b700 1 -- 172.21.15.12:0/3140598643 learned_addr learned my addr 172.21.15.12:0/3140598643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff1fbfff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc10cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff200c9b700 1 -- 172.21.15.12:0/3140598643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fcf80 msgr2=0x7ff1fc10cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff200c9b700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc10cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff200c9b700 1 -- 172.21.15.12:0/3140598643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e8005040 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff200c9b700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b790 tx=0x7ff1f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1f000f770 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1f000fda0 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc111fb0 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1f000d310 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.598+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc1124f0 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.599+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc04c710 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.600+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff1f0010420 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.600+0000 7ff1f97fa700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 0x7ff1ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:28.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.600+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff1f0086e90 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.600+0000 7ff1fbfff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 0x7ff1ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:28.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.601+0000 7ff1fbfff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 0x7ff1ec06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b3b0 tx=0x7ff1e8009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:28.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.605+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7ff1f0056f90 con 0x7ff1fc0fbcd0 2024-03-18T14:48:28.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.805+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1fc101260 con 0x7ff1ec0692a0 2024-03-18T14:48:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.807+0000 7ff1f97fa700 1 -- 172.21.15.12:0/3140598643 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7ff1fc101260 con 0x7ff1ec0692a0 2024-03-18T14:48:28.808 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-03-18T14:48:28.808 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-03-18T14:48:28.808 INFO:teuthology.orchestra.run.smithi012.stdout:smithi100 172.21.15.100 2024-03-18T14:48:28.808 INFO:teuthology.orchestra.run.smithi012.stdout:2 hosts in cluster 2024-03-18T14:48:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 msgr2=0x7ff1ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 0x7ff1ec06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b3b0 tx=0x7ff1e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 msgr2=0x7ff1fc10c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b790 tx=0x7ff1f0009250 comp rx=0 tx=0).stop 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 shutdown_connections 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff1ec0692a0 0x7ff1ec06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1fc0fbcd0 0x7ff1fc10c840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 --2- 172.21.15.12:0/3140598643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1fc0fcf80 0x7ff1fc10cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 >> 172.21.15.12:0/3140598643 conn(0x7ff1fc0f7430 msgr2=0x7ff1fc0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 shutdown_connections 2024-03-18T14:48:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:28.809+0000 7ff202eff700 1 -- 172.21.15.12:0/3140598643 wait complete. 2024-03-18T14:48:29.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-18T14:48:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:30.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:30 smithi012 ceph-mon[29182]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:30.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:30 smithi012 ceph-mon[29182]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:30 smithi100 ceph-mon[37807]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:30 smithi100 ceph-mon[37807]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.448+0000 7fb8972b1700 1 -- 172.21.15.12:0/3304035007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fdfe0 msgr2=0x7fb8900fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.448+0000 7fb8972b1700 1 --2- 172.21.15.12:0/3304035007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fdfe0 0x7fb8900fe450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb880004940 tx=0x7fb880009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- 172.21.15.12:0/3304035007 shutdown_connections 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 --2- 172.21.15.12:0/3304035007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fdfe0 0x7fb8900fe450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 --2- 172.21.15.12:0/3304035007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fce70 0x7fb8900fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- 172.21.15.12:0/3304035007 >> 172.21.15.12:0/3304035007 conn(0x7fb8900f8550 msgr2=0x7fb8900fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- 172.21.15.12:0/3304035007 shutdown_connections 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- 172.21.15.12:0/3304035007 wait complete. 2024-03-18T14:48:31.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 Processor -- start 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- start start 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fdfe0 0x7fb89010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89010e4e0 con 0x7fb8900fce70 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb8972b1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890112e80 con 0x7fb8900fdfe0 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb89504d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb89504d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51654/0 (socket says 172.21.15.12:51654) 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.449+0000 7fb89504d700 1 -- 172.21.15.12:0/283570023 learned_addr learned my addr 172.21.15.12:0/283570023 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb89484c700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fdfe0 0x7fb89010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb89504d700 1 -- 172.21.15.12:0/283570023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fdfe0 msgr2=0x7fb89010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb89504d700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fdfe0 0x7fb89010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb89504d700 1 -- 172.21.15.12:0/283570023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880005040 con 0x7fb8900fce70 2024-03-18T14:48:31.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb89504d700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b790 tx=0x7fb88c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c00f770 con 0x7fb8900fce70 2024-03-18T14:48:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb88c00fda0 con 0x7fb8900fce70 2024-03-18T14:48:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8901130e0 con 0x7fb8900fce70 2024-03-18T14:48:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c00d310 con 0x7fb8900fce70 2024-03-18T14:48:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.450+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb890113650 con 0x7fb8900fce70 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.451+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89004c710 con 0x7fb8900fce70 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.451+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb88c010420 con 0x7fb8900fce70 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.452+0000 7fb885ffb700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 0x7fb87c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.452+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb88c086e70 con 0x7fb8900fce70 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.452+0000 7fb89484c700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 0x7fb87c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.452+0000 7fb89484c700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 0x7fb87c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb88000b3b0 tx=0x7fb880009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.456+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb88c056f70 con 0x7fb8900fce70 2024-03-18T14:48:31.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.656+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb890102280 con 0x7fb87c0692a0 2024-03-18T14:48:31.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.658+0000 7fb885ffb700 1 -- 172.21.15.12:0/283570023 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fb890102280 con 0x7fb87c0692a0 2024-03-18T14:48:31.659 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-18T14:48:31.659 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T14:48:31.659 INFO:teuthology.orchestra.run.smithi012.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 74s ago Has a FileSystem, LVM detected 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 msgr2=0x7fb87c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 0x7fb87c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb88000b3b0 tx=0x7fb880009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 msgr2=0x7fb89010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b790 tx=0x7fb88c009250 comp rx=0 tx=0).stop 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 shutdown_connections 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb87c0692a0 0x7fb87c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8900fce70 0x7fb89010d910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 --2- 172.21.15.12:0/283570023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8900fdfe0 0x7fb89010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:31.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 >> 172.21.15.12:0/283570023 conn(0x7fb8900f8550 msgr2=0x7fb8900f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:31.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 shutdown_connections 2024-03-18T14:48:31.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:31.660+0000 7fb8972b1700 1 -- 172.21.15.12:0/283570023 wait complete. 2024-03-18T14:48:32.318 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:32 smithi012 ceph-mon[29182]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:32.343 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:32.353 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:32.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-18T14:48:32.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:32 smithi100 ceph-mon[37807]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:32.613 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:33.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:33 smithi012 ceph-mon[29182]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:33.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:33 smithi100 ceph-mon[37807]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.201+0000 7fc2ad837700 1 -- 172.21.15.12:0/2761314179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fbcd0 msgr2=0x7fc2a80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.201+0000 7fc2ad837700 1 --2- 172.21.15.12:0/2761314179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a80fc0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009ce0 tx=0x7fc2a0004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.201+0000 7fc2ad837700 1 -- 172.21.15.12:0/2761314179 shutdown_connections 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.201+0000 7fc2ad837700 1 --2- 172.21.15.12:0/2761314179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fcf80 0x7fc2a80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.201+0000 7fc2ad837700 1 --2- 172.21.15.12:0/2761314179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a80fc0e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.202+0000 7fc2ad837700 1 -- 172.21.15.12:0/2761314179 >> 172.21.15.12:0/2761314179 conn(0x7fc2a80f7450 msgr2=0x7fc2a80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.202+0000 7fc2ad837700 1 -- 172.21.15.12:0/2761314179 shutdown_connections 2024-03-18T14:48:34.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.202+0000 7fc2ad837700 1 -- 172.21.15.12:0/2761314179 wait complete. 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.202+0000 7fc2ad837700 1 Processor -- start 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.202+0000 7fc2ad837700 1 -- start start 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2ad837700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a810fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2ad837700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2ad837700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8110940 con 0x7fc2a80fcf80 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2ad837700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8115250 con 0x7fc2a80fbcd0 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51680/0 (socket says 172.21.15.12:51680) 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 -- 172.21.15.12:0/1274209663 learned_addr learned my addr 172.21.15.12:0/1274209663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a6ffd700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a810fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 -- 172.21.15.12:0/1274209663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fbcd0 msgr2=0x7fc2a810fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a810fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 -- 172.21.15.12:0/1274209663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009940 con 0x7fc2a80fcf80 2024-03-18T14:48:34.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2a67fc700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b790 tx=0x7fc290009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29000f6f0 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc29000fd20 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.203+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a81154b0 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.204+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2900103a0 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.204+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81159f0 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.205+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a8101910 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.205+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc290010610 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.205+0000 7fc29f7fe700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 0x7fc29406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.205+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc290086d90 con 0x7fc2a80fcf80 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.205+0000 7fc2a6ffd700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 0x7fc29406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:34.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.206+0000 7fc2a6ffd700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 0x7fc29406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009470 tx=0x7fc2a0004d10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:34.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.210+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc2900533d0 con 0x7fc2a80fcf80 2024-03-18T14:48:34.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:34 smithi012 ceph-mon[29182]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:34.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:34.431+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2a802b3b0 con 0x7fc2940692a0 2024-03-18T14:48:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:34 smithi100 ceph-mon[37807]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:35.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-18T14:48:35.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-18T14:48:36.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.109+0000 7fc29f7fe700 1 -- 172.21.15.12:0/1274209663 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc2a802b3b0 con 0x7fc2940692a0 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 msgr2=0x7fc29406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 0x7fc29406b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009470 tx=0x7fc2a0004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 msgr2=0x7fc2a8110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b790 tx=0x7fc290009250 comp rx=0 tx=0).stop 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 shutdown_connections 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc2940692a0 0x7fc29406b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80fbcd0 0x7fc2a810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 --2- 172.21.15.12:0/1274209663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2a80fcf80 0x7fc2a8110260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.111+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 >> 172.21.15.12:0/1274209663 conn(0x7fc2a80f7450 msgr2=0x7fc2a80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.112+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 shutdown_connections 2024-03-18T14:48:36.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:36.112+0000 7fc2ad837700 1 -- 172.21.15.12:0/1274209663 wait complete. 2024-03-18T14:48:36.360 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:36.360 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:36.361 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-18T14:48:36.361 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T14:48:36.361 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-18T14:48:36.361 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:36.448 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:36 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[29178]: 2024-03-18T14:48:36.088+0000 7fcebac38700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T14:48:36.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T14:48:37.040 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T14:48:37.363 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: fsmap cephfs:0 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: Saving service mds.cephfs spec with placement count:4 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:37.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:37 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: fsmap cephfs:0 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: Saving service mds.cephfs spec with placement count:4 2024-03-18T14:48:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:37.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:37 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:38.368 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:38 smithi012 ceph-mon[29182]: pgmap v174: 65 pgs: 3 creating+peering, 4 active+clean, 58 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:38.368 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:38 smithi012 ceph-mon[29182]: Deploying daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T14:48:38.368 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:38 smithi012 ceph-mon[29182]: osdmap e44: 6 total, 6 up, 6 in 2024-03-18T14:48:38.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:38 smithi100 ceph-mon[37807]: pgmap v174: 65 pgs: 3 creating+peering, 4 active+clean, 58 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:38.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:38 smithi100 ceph-mon[37807]: Deploying daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T14:48:38.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:38 smithi100 ceph-mon[37807]: osdmap e44: 6 total, 6 up, 6 in 2024-03-18T14:48:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:39 smithi012 ceph-mon[29182]: osdmap e45: 6 total, 6 up, 6 in 2024-03-18T14:48:39.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:39 smithi100 ceph-mon[37807]: osdmap e45: 6 total, 6 up, 6 in 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- 172.21.15.12:0/1623939843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fbce0 msgr2=0x7fb0200fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 --2- 172.21.15.12:0/1623939843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fbce0 0x7fb0200fc0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb014004940 tx=0x7fb014009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- 172.21.15.12:0/1623939843 shutdown_connections 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 --2- 172.21.15.12:0/1623939843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fcf90 0x7fb0200fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 --2- 172.21.15.12:0/1623939843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fbce0 0x7fb0200fc0f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- 172.21.15.12:0/1623939843 >> 172.21.15.12:0/1623939843 conn(0x7fb0200f7440 msgr2=0x7fb0200f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- 172.21.15.12:0/1623939843 shutdown_connections 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- 172.21.15.12:0/1623939843 wait complete. 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 Processor -- start 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.786+0000 7fb02540a700 1 -- start start 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb02540a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fbce0 0x7fb020112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb02540a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb02540a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020066ad0 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb02540a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020066c10 con 0x7fb0200fbce0 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55916/0 (socket says 172.21.15.12:55916) 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 -- 172.21.15.12:0/2112497126 learned_addr learned my addr 172.21.15.12:0/2112497126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01effd700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fbce0 0x7fb020112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 -- 172.21.15.12:0/2112497126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fbce0 msgr2=0x7fb020112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fbce0 0x7fb020112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 -- 172.21.15.12:0/2112497126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014005040 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.787+0000 7fb01e7fc700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b790 tx=0x7fb010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.788+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01000f6f0 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.788+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0201155d0 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.788+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01000fd20 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.788+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0100103a0 con 0x7fb0200fcf90 2024-03-18T14:48:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.788+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb020115ab0 con 0x7fb0200fcf90 2024-03-18T14:48:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.789+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02005f5f0 con 0x7fb0200fcf90 2024-03-18T14:48:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.789+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb010011400 con 0x7fb0200fcf90 2024-03-18T14:48:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.790+0000 7fb0077fe700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 0x7fb00806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:39.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.790+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb0100879c0 con 0x7fb0200fcf90 2024-03-18T14:48:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.790+0000 7fb01effd700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 0x7fb00806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.790+0000 7fb01effd700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 0x7fb00806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb014004890 tx=0x7fb014009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:39.794+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fb010053d90 con 0x7fb0200fcf90 2024-03-18T14:48:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.038+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb020104850 con 0x7fb0200fcf90 2024-03-18T14:48:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.038+0000 7fb0077fe700 1 -- 172.21.15.12:0/2112497126 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fb0100573b0 con 0x7fb0200fcf90 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:e2 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 2 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:36.088888+0000 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:48:40.040 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:in 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:up {} 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:48:40.041 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 msgr2=0x7fb00806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 0x7fb00806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb014004890 tx=0x7fb014009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 msgr2=0x7fb020115090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b790 tx=0x7fb010009250 comp rx=0 tx=0).stop 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 shutdown_connections 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fb0080692a0 0x7fb00806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0200fbce0 0x7fb020112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 --2- 172.21.15.12:0/2112497126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0200fcf90 0x7fb020115090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 >> 172.21.15.12:0/2112497126 conn(0x7fb0200f7440 msgr2=0x7fb0200f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 shutdown_connections 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:40.040+0000 7fb02540a700 1 -- 172.21.15.12:0/2112497126 wait complete. 2024-03-18T14:48:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-03-18T14:48:40.293 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:40 smithi012 ceph-mon[29182]: pgmap v177: 65 pgs: 4 creating+activating, 17 creating+peering, 32 active+clean, 12 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:40.293 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:40 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2112497126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:48:40.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:40 smithi100 ceph-mon[37807]: pgmap v177: 65 pgs: 4 creating+activating, 17 creating+peering, 32 active+clean, 12 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:40.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:40 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2112497126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:48:40.914 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:40.925 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:40.925 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-18T14:48:41.201 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:42.465 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: pgmap v178: 65 pgs: 4 creating+activating, 17 creating+peering, 44 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:42.465 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:boot 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: daemon mds.cephfs.smithi012.xtadxo assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: Cluster is now healthy 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: fsmap cephfs:0 1 up:standby 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:creating} 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: daemon mds.cephfs.smithi012.xtadxo is now active in filesystem cephfs as rank 0 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:42.466 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:42 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: pgmap v178: 65 pgs: 4 creating+activating, 17 creating+peering, 44 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:boot 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: daemon mds.cephfs.smithi012.xtadxo assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: Cluster is now healthy 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: fsmap cephfs:0 1 up:standby 2024-03-18T14:48:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:creating} 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: daemon mds.cephfs.smithi012.xtadxo is now active in filesystem cephfs as rank 0 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:42.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:42 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 -- 172.21.15.12:0/2199089835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 msgr2=0x7f703c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 --2- 172.21.15.12:0/2199089835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c0fd280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f702c004940 tx=0x7f702c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 -- 172.21.15.12:0/2199089835 shutdown_connections 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 --2- 172.21.15.12:0/2199089835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 0x7f703c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 --2- 172.21.15.12:0/2199089835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c0fd280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 -- 172.21.15.12:0/2199089835 >> 172.21.15.12:0/2199089835 conn(0x7f703c0f8550 msgr2=0x7f703c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 -- 172.21.15.12:0/2199089835 shutdown_connections 2024-03-18T14:48:43.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 -- 172.21.15.12:0/2199089835 wait complete. 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.099+0000 7f7044743700 1 Processor -- start 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7044743700 1 -- start start 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7044743700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7044743700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 0x7f703c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7044743700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c10e510 con 0x7f703c0fce70 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7044743700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c112eb0 con 0x7f703c0fdfe0 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55954/0 (socket says 172.21.15.12:55954) 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 -- 172.21.15.12:0/2242362687 learned_addr learned my addr 172.21.15.12:0/2242362687 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f7041cde700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 0x7f703c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 -- 172.21.15.12:0/2242362687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 msgr2=0x7f703c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 0x7f703c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 -- 172.21.15.12:0/2242362687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c005040 con 0x7f703c0fce70 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.100+0000 7f70424df700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f702c009150 tx=0x7f702c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:43.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.101+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f702c00e040 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.101+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f702c00bbf0 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.101+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c1130b0 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.101+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f702c01c630 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.101+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c113620 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.102+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c04c710 con 0x7f703c0fce70 2024-03-18T14:48:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.102+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f702c00bd60 con 0x7f703c0fce70 2024-03-18T14:48:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.102+0000 7f7032ffd700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 0x7f702806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.103+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f702c0884c0 con 0x7f703c0fce70 2024-03-18T14:48:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.103+0000 7f7041cde700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 0x7f702806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.103+0000 7f7041cde700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 0x7f702806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7038006fd0 tx=0x7f7038008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.107+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f702c058320 con 0x7f703c0fce70 2024-03-18T14:48:43.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:43.344+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f703c1139d0 con 0x7f703c0fce70 2024-03-18T14:48:43.345 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:43 smithi012 ceph-mon[29182]: Deploying daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T14:48:43.345 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:43 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:active 2024-03-18T14:48:43.345 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:43 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2024-03-18T14:48:43.448 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:43 smithi100 ceph-mon[37807]: Deploying daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T14:48:43.448 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:43 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:active 2024-03-18T14:48:43.448 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:43 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2024-03-18T14:48:44.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.132+0000 7f7032ffd700 1 -- 172.21.15.12:0/2242362687 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f702c057eb0 con 0x7f703c0fce70 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.134+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 msgr2=0x7f702806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.134+0000 7f7044743700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 0x7f702806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7038006fd0 tx=0x7f7038008040 comp rx=0 tx=0).stop 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.134+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 msgr2=0x7f703c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.134+0000 7f7044743700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f702c009150 tx=0x7f702c009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 shutdown_connections 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f70280692a0 0x7f702806b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f703c0fce70 0x7f703c10d940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 --2- 172.21.15.12:0/2242362687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c0fdfe0 0x7f703c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 >> 172.21.15.12:0/2242362687 conn(0x7f703c0f8550 msgr2=0x7f703c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 shutdown_connections 2024-03-18T14:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:44.135+0000 7f7044743700 1 -- 172.21.15.12:0/2242362687 wait complete. 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2242362687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:44 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2242362687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:44.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:44 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: Deploying daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:boot 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2242362687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:48:45.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:45 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: Deploying daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:boot 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2242362687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:48:45.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:45 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby 2024-03-18T14:48:45.637 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:45.647 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:45.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-18T14:48:46.260 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:46.382 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:46 smithi012 ceph-mon[29182]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-18T14:48:46.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:46 smithi100 ceph-mon[37807]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-18T14:48:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] up:boot 2024-03-18T14:48:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T14:48:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:47 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] up:boot 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T14:48:47.358 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:47 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:47.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- 172.21.15.12:0/610280672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 msgr2=0x7f52900fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:47.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 --2- 172.21.15.12:0/610280672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f52900fd280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5288009ce0 tx=0x7f5288004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- 172.21.15.12:0/610280672 shutdown_connections 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 --2- 172.21.15.12:0/610280672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 0x7f52900fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 --2- 172.21.15.12:0/610280672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f52900fd280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- 172.21.15.12:0/610280672 >> 172.21.15.12:0/610280672 conn(0x7f52900f8550 msgr2=0x7f52900fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- 172.21.15.12:0/610280672 shutdown_connections 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- 172.21.15.12:0/610280672 wait complete. 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 Processor -- start 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.860+0000 7f5296e5f700 1 -- start start 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 0x7f529010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529010e5d0 con 0x7f52900fce70 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529010e740 con 0x7f52900fdfe0 2024-03-18T14:48:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55988/0 (socket says 172.21.15.12:55988) 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 -- 172.21.15.12:0/1733110709 learned_addr learned my addr 172.21.15.12:0/1733110709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f528ffff700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 0x7f529010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 -- 172.21.15.12:0/1733110709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 msgr2=0x7f529010dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 0x7f529010dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 -- 172.21.15.12:0/1733110709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009940 con 0x7f52900fce70 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5294bfb700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5288009380 tx=0x7f5288004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528800e040 con 0x7f52900fce70 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52880128e0 con 0x7f52900fce70 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528801ca10 con 0x7f52900fce70 2024-03-18T14:48:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5290113140 con 0x7f52900fce70 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.861+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5290113700 con 0x7f52900fce70 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.863+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529004c710 con 0x7f52900fce70 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.863+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5288012a50 con 0x7f52900fce70 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.863+0000 7f528d7fa700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 0x7f527c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.863+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5288088db0 con 0x7f52900fce70 2024-03-18T14:48:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.863+0000 7f528ffff700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 0x7f527c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.864+0000 7f528ffff700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 0x7f527c06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5280004990 tx=0x7f5280009ee0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:47.868+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f5288058af0 con 0x7f52900fce70 2024-03-18T14:48:48.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.107+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f52901058d0 con 0x7f52900fce70 2024-03-18T14:48:48.457 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:48 smithi100 ceph-mon[37807]: Deploying daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T14:48:48.457 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:48 smithi100 ceph-mon[37807]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-18T14:48:48.457 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:48 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1733110709' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T14:48:48.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:48 smithi012 ceph-mon[29182]: Deploying daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T14:48:48.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:48 smithi012 ceph-mon[29182]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-18T14:48:48.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:48 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1733110709' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T14:48:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.875+0000 7f528d7fa700 1 -- 172.21.15.12:0/1733110709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5288012d00 con 0x7f52900fce70 2024-03-18T14:48:48.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 msgr2=0x7f527c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 0x7f527c06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5280004990 tx=0x7f5280009ee0 comp rx=0 tx=0).stop 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 msgr2=0x7f529010da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5288009380 tx=0x7f5288004d10 comp rx=0 tx=0).stop 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 shutdown_connections 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f527c0692a0 0x7f527c06b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52900fce70 0x7f529010da90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 --2- 172.21.15.12:0/1733110709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52900fdfe0 0x7f529010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.877+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 >> 172.21.15.12:0/1733110709 conn(0x7f52900f8550 msgr2=0x7f52900f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.878+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 shutdown_connections 2024-03-18T14:48:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:48.878+0000 7f5296e5f700 1 -- 172.21.15.12:0/1733110709 wait complete. 2024-03-18T14:48:49.541 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:49.554 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:49.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-18T14:48:49.768 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:49.769 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1733110709' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T14:48:49.769 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.100:6826/1000007069,v1:172.21.15.100:6827/1000007069] up:boot 2024-03-18T14:48:49.769 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T14:48:49.769 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:48:49.769 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:49 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1733110709' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.100:6826/1000007069,v1:172.21.15.100:6827/1000007069] up:boot 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T14:48:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:48:49.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:49 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:50.471 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:50 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:active 2024-03-18T14:48:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:50 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:51.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:50 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] up:active 2024-03-18T14:48:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:50 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:48:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] up:standby 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: Dropping low affinity standby-replay daemon mds.cephfs.smithi100.rutxmj in favor of higher affinity standby. 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 1 up:standby 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:52.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] up:standby 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: Dropping low affinity standby-replay daemon mds.cephfs.smithi100.rutxmj in favor of higher affinity standby. 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 1 up:standby 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:52.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 -- 172.21.15.12:0/1788724106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fce70 msgr2=0x7f111c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 --2- 172.21.15.12:0/1788724106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fce70 0x7f111c0fd280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1110004940 tx=0x7f1110009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 -- 172.21.15.12:0/1788724106 shutdown_connections 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 --2- 172.21.15.12:0/1788724106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fdfe0 0x7f111c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 --2- 172.21.15.12:0/1788724106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fce70 0x7f111c0fd280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 -- 172.21.15.12:0/1788724106 >> 172.21.15.12:0/1788724106 conn(0x7f111c0f8550 msgr2=0x7f111c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 -- 172.21.15.12:0/1788724106 shutdown_connections 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.823+0000 7f1122ee9700 1 -- 172.21.15.12:0/1788724106 wait complete. 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 Processor -- start 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 -- start start 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fce70 0x7f111c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c10e510 con 0x7f111c0fdfe0 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f1122ee9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c112eb0 con 0x7f111c0fce70 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f111bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f111bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38754/0 (socket says 172.21.15.12:38754) 2024-03-18T14:48:52.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.824+0000 7f111bfff700 1 -- 172.21.15.12:0/4125191806 learned_addr learned my addr 172.21.15.12:0/4125191806 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f1120c85700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fce70 0x7f111c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f111bfff700 1 -- 172.21.15.12:0/4125191806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fce70 msgr2=0x7f111c10d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f111bfff700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fce70 0x7f111c10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f111bfff700 1 -- 172.21.15.12:0/4125191806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110005040 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f111bfff700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f110800b790 tx=0x7f1108009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110800f770 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f110800fda0 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c113110 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110800d310 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.825+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c113680 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.826+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c04c710 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.826+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1108010420 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.827+0000 7f11197fa700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 0x7f110c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.827+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1108087250 con 0x7f111c0fdfe0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.827+0000 7f1120c85700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 0x7f110c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:52.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.827+0000 7f1120c85700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 0x7f110c06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1110000c00 tx=0x7f1110009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:52.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:52.831+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f1108056f70 con 0x7f111c0fdfe0 2024-03-18T14:48:53.046 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:52 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:boot 2024-03-18T14:48:53.052 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:52 smithi012 ceph-mon[29182]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:standby 2024-03-18T14:48:53.053 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:52 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:53.053 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:52 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:48:53.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.081+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f111c105950 con 0x7f111c0fdfe0 2024-03-18T14:48:53.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:52 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:boot 2024-03-18T14:48:53.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:52 smithi100 ceph-mon[37807]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:standby 2024-03-18T14:48:53.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:52 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:53.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:52 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:48:53.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.930+0000 7f11197fa700 1 -- 172.21.15.12:0/4125191806 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v15) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1108056b00 con 0x7f111c0fdfe0 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 msgr2=0x7f110c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 0x7f110c06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1110000c00 tx=0x7f1110009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 msgr2=0x7f111c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f110800b790 tx=0x7f1108009250 comp rx=0 tx=0).stop 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 shutdown_connections 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f110c0692a0 0x7f110c06b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f111c0fce70 0x7f111c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 --2- 172.21.15.12:0/4125191806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f111c0fdfe0 0x7f111c10de80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 >> 172.21.15.12:0/4125191806 conn(0x7f111c0f8550 msgr2=0x7f111c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 shutdown_connections 2024-03-18T14:48:53.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:53.932+0000 7f1122ee9700 1 -- 172.21.15.12:0/4125191806 wait complete. 2024-03-18T14:48:53.934 INFO:teuthology.orchestra.run.smithi012.stderr:inline data disabled 2024-03-18T14:48:53.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:53 smithi012 ceph-mon[29182]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-18T14:48:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:53 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4125191806' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-18T14:48:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:53 smithi100 ceph-mon[37807]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-18T14:48:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:53 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4125191806' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-18T14:48:55.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:54 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4125191806' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-18T14:48:55.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:54 smithi012 ceph-mon[29182]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:55.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:54 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4125191806' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-18T14:48:55.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:54 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:48:55.639 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T14:48:55.649 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:48:55.649 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T14:48:55.931 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:56.188 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-18T14:48:56.188 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.188 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.188 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:56.189 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:56.189 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.189 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:55 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:55 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:48:57.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/1632213222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fcf80 msgr2=0x7fdc740fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:57.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/1632213222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fcf80 0x7fdc740fd3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc64004940 tx=0x7fdc64009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:57.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/1632213222 shutdown_connections 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/1632213222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fcf80 0x7fdc740fd3f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/1632213222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fbcd0 0x7fdc740fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/1632213222 >> 172.21.15.12:0/1632213222 conn(0x7fdc740f7430 msgr2=0x7fdc740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/1632213222 shutdown_connections 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/1632213222 wait complete. 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 Processor -- start 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- start start 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fcf80 0x7fdc7410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410e520 con 0x7fdc740fbcd0 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7c3dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410ed20 con 0x7fdc740fcf80 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.629+0000 7fdc7a179700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:57.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38778/0 (socket says 172.21.15.12:38778) 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 -- 172.21.15.12:0/3056266537 learned_addr learned my addr 172.21.15.12:0/3056266537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc79978700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fcf80 0x7fdc7410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 -- 172.21.15.12:0/3056266537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fcf80 msgr2=0x7fdc7410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fcf80 0x7fdc7410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 -- 172.21.15.12:0/3056266537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc64005040 con 0x7fdc740fbcd0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7a179700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000eb40 tx=0x7fdc7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc70007cc0 con 0x7fdc740fbcd0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc7000f040 con 0x7fdc740fbcd0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7410efa0 con 0x7fdc740fbcd0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7000ca10 con 0x7fdc740fbcd0 2024-03-18T14:48:57.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.630+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7410f4e0 con 0x7fdc740fbcd0 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.631+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc74101b10 con 0x7fdc740fbcd0 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.631+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdc7000cc80 con 0x7fdc740fbcd0 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.632+0000 7fdc6affd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.632+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdc70086d80 con 0x7fdc740fbcd0 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.632+0000 7fdc79978700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:48:57.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.632+0000 7fdc79978700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 0x7fdc6006b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009b30 tx=0x7fdc64009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:48:57.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.636+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fdc7004ddb0 con 0x7fdc740fbcd0 2024-03-18T14:48:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.875+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc741048c0 con 0x7fdc740fbcd0 2024-03-18T14:48:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.875+0000 7fdc6affd700 1 -- 172.21.15.12:0/3056266537 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fdc70056b70 con 0x7fdc740fbcd0 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:48:57.877 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:48:57.878 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:48:57.879 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 msgr2=0x7fdc6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 0x7fdc6006b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009b30 tx=0x7fdc64009e30 comp rx=0 tx=0).stop 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 msgr2=0x7fdc7410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000eb40 tx=0x7fdc7000b040 comp rx=0 tx=0).stop 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 shutdown_connections 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc740fbcd0 0x7fdc7410d9e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 --2- 172.21.15.12:0/3056266537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc740fcf80 0x7fdc7410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:48:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.878+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 >> 172.21.15.12:0/3056266537 conn(0x7fdc740f7430 msgr2=0x7fdc74065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:48:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.879+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 shutdown_connections 2024-03-18T14:48:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:48:57.879+0000 7fdc7c3dd700 1 -- 172.21.15.12:0/3056266537 wait complete. 2024-03-18T14:48:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:48:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:57 smithi012 ceph-mon[29182]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:48:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:57 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:57 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3056266537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:48:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:57 smithi100 ceph-mon[37807]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:48:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:57 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:48:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:57 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3056266537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:48:58.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-18T14:48:58.873 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:48:59.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:48:59 smithi012 ceph-mon[29182]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:49:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:48:59 smithi100 ceph-mon[37807]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2443319024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 msgr2=0x7f1a340fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2443319024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a340fd250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1a24004a90 tx=0x7f1a24009dd0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2443319024 shutdown_connections 2024-03-18T14:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2443319024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 0x7f1a340fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2443319024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a340fd250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2443319024 >> 172.21.15.12:0/2443319024 conn(0x7f1a340f8570 msgr2=0x7f1a340fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2443319024 shutdown_connections 2024-03-18T14:49:01.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.048+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2443319024 wait complete. 2024-03-18T14:49:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 Processor -- start 2024-03-18T14:49:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 -- start start 2024-03-18T14:49:01.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 0x7f1a34070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340708c0 con 0x7f1a340fce40 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a38bfe700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34070a30 con 0x7f1a340fe040 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39212/0 (socket says 172.21.15.12:39212) 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 -- 172.21.15.12:0/2240073648 learned_addr learned my addr 172.21.15.12:0/2240073648 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a31d9b700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 0x7f1a34070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 -- 172.21.15.12:0/2240073648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 msgr2=0x7f1a34070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 0x7f1a34070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.049+0000 7f1a3259c700 1 -- 172.21.15.12:0/2240073648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24004650 con 0x7f1a340fce40 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a3259c700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400ba80 tx=0x7f1a24009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a24019070 con 0x7f1a340fce40 2024-03-18T14:49:01.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a34070cb0 con 0x7f1a340fce40 2024-03-18T14:49:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a2400bd00 con 0x7f1a340fce40 2024-03-18T14:49:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a34071190 con 0x7f1a340fce40 2024-03-18T14:49:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.050+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2400e040 con 0x7f1a340fce40 2024-03-18T14:49:01.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.051+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a140052e0 con 0x7f1a340fce40 2024-03-18T14:49:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.051+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1a24027440 con 0x7f1a340fce40 2024-03-18T14:49:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.052+0000 7f1a22ffd700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 0x7f1a1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.052+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1a24087ec0 con 0x7f1a340fce40 2024-03-18T14:49:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.052+0000 7f1a31d9b700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 0x7f1a1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:01.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.052+0000 7f1a31d9b700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 0x7f1a1c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009b80 tx=0x7f1a2800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:01.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.057+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f1a240541f0 con 0x7f1a340fce40 2024-03-18T14:49:01.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.300+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a14005b10 con 0x7f1a340fce40 2024-03-18T14:49:01.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.300+0000 7f1a22ffd700 1 -- 172.21.15.12:0/2240073648 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4697 (secure 0 0 0) 0x7f1a24057810 con 0x7f1a340fce40 2024-03-18T14:49:01.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.302+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 msgr2=0x7f1a1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:01.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.302+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 0x7f1a1c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009b80 tx=0x7f1a2800b040 comp rx=0 tx=0).stop 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.302+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 msgr2=0x7f1a34071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.302+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400ba80 tx=0x7f1a24009dd0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 shutdown_connections 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f1a1c0692a0 0x7f1a1c06b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a340fce40 0x7f1a34071e40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 --2- 172.21.15.12:0/2240073648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a340fe040 0x7f1a34070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:01.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 >> 172.21.15.12:0/2240073648 conn(0x7f1a340f8570 msgr2=0x7f1a340f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:01.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 shutdown_connections 2024-03-18T14:49:01.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:01.303+0000 7f1a38bfe700 1 -- 172.21.15.12:0/2240073648 wait complete. 2024-03-18T14:49:01.305 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:49:01.315 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:49:02.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-18T14:49:02.223 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:01 smithi012 ceph-mon[29182]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:49:02.223 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:01 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2240073648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T14:49:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:01 smithi100 ceph-mon[37807]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-18T14:49:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:01 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2240073648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T14:49:02.532 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:04.221 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:03 smithi012 ceph-mon[29182]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-03-18T14:49:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:03 smithi100 ceph-mon[37807]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 -- 172.21.15.12:0/2659101394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 msgr2=0x7fe2a00fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/2659101394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a00fc0f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe29c004940 tx=0x7fe29c009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 -- 172.21.15.12:0/2659101394 shutdown_connections 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/2659101394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 0x7fe2a00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/2659101394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a00fc0f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 -- 172.21.15.12:0/2659101394 >> 172.21.15.12:0/2659101394 conn(0x7fe2a00f7440 msgr2=0x7fe2a00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 -- 172.21.15.12:0/2659101394 shutdown_connections 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.388+0000 7fe2a7f95700 1 -- 172.21.15.12:0/2659101394 wait complete. 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 Processor -- start 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 -- start start 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 0x7fe2a010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a010e590 con 0x7fe2a00fbce0 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a7f95700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a010ede0 con 0x7fe2a00fcf90 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39232/0 (socket says 172.21.15.12:39232) 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 -- 172.21.15.12:0/1160093666 learned_addr learned my addr 172.21.15.12:0/1160093666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5530700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 0x7fe2a010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 -- 172.21.15.12:0/1160093666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 msgr2=0x7fe2a010df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 0x7fe2a010df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.389+0000 7fe2a5d31700 1 -- 172.21.15.12:0/1160093666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c005040 con 0x7fe2a00fbce0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2a5d31700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00b3b0 tx=0x7fe29c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c00e040 con 0x7fe2a00fbce0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a010f060 con 0x7fe2a00fbce0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a010f5a0 con 0x7fe2a00fbce0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe29c00bbf0 con 0x7fe2a00fbce0 2024-03-18T14:49:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.390+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c01c630 con 0x7fe2a00fbce0 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.391+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a0101b20 con 0x7fe2a00fbce0 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.392+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe29c00bd60 con 0x7fe2a00fbce0 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.392+0000 7fe2967fc700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 0x7fe28c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.392+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe29c087de0 con 0x7fe2a00fbce0 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.393+0000 7fe2a5530700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 0x7fe28c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.393+0000 7fe2a5530700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 0x7fe28c06b7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe290004f60 tx=0x7fe29000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.397+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe29c057ba0 con 0x7fe2a00fbce0 2024-03-18T14:49:05.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.728+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fe2a005f5f0 con 0x7fe2a00fbce0 2024-03-18T14:49:05.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.729+0000 7fe2967fc700 1 -- 172.21.15.12:0/1160093666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+99 (secure 0 0 0) 0x7fe29c057730 con 0x7fe2a00fbce0 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.730+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 msgr2=0x7fe28c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.730+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 0x7fe28c06b7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe290004f60 tx=0x7fe29000b040 comp rx=0 tx=0).stop 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.730+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 msgr2=0x7fe2a010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.730+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00b3b0 tx=0x7fe29c009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 shutdown_connections 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe28c069310 0x7fe28c06b7c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2a00fbce0 0x7fe2a010da50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 --2- 172.21.15.12:0/1160093666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2a00fcf90 0x7fe2a010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 >> 172.21.15.12:0/1160093666 conn(0x7fe2a00f7440 msgr2=0x7fe2a0066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 shutdown_connections 2024-03-18T14:49:05.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:05.731+0000 7fe2a7f95700 1 -- 172.21.15.12:0/1160093666 wait complete. 2024-03-18T14:49:05.743 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1160093666' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-18T14:49:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:05 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1160093666' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-18T14:49:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:05 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:06.406 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-18T14:49:06.419 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T14:49:06.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.553+0000 7f03a5a6e640 1 -- 172.21.15.12:0/3594180086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 msgr2=0x7f03a00fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.553+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/3594180086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a00fee50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f03900048d0 tx=0x7f0390009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- 172.21.15.12:0/3594180086 shutdown_connections 2024-03-18T14:49:06.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/3594180086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a0101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/3594180086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a00fee50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- 172.21.15.12:0/3594180086 >> 172.21.15.12:0/3594180086 conn(0x7f03a00f6840 msgr2=0x7f03a00f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- 172.21.15.12:0/3594180086 shutdown_connections 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- 172.21.15.12:0/3594180086 wait complete. 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 Processor -- start 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- start start 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a010d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a010da40 con 0x7f03a0064f90 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f03a5a6e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0112360 con 0x7f03a00ff390 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f039f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f039f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39256/0 (socket says 172.21.15.12:39256) 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.554+0000 7f039f7fe640 1 -- 172.21.15.12:0/4003252391 learned_addr learned my addr 172.21.15.12:0/4003252391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:06.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039effd640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a010d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039f7fe640 1 -- 172.21.15.12:0/4003252391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 msgr2=0x7f03a010d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039f7fe640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a010d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039f7fe640 1 -- 172.21.15.12:0/4003252391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390005040 con 0x7f03a0064f90 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039effd640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f039f7fe640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f039002bda0 tx=0x7f0390009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0390009590 con 0x7f03a0064f90 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a0112560 con 0x7f03a0064f90 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03900096f0 con 0x7f03a0064f90 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039003b640 con 0x7f03a0064f90 2024-03-18T14:49:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.555+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0112a80 con 0x7f03a0064f90 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.556+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a004bcc0 con 0x7f03a0064f90 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.556+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f039002f070 con 0x7f03a0064f90 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.557+0000 7f0387fff640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 0x7f038806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.557+0000 7f039effd640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 0x7f038806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.557+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f03900a98a0 con 0x7f03a0064f90 2024-03-18T14:49:06.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.557+0000 7f039effd640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 0x7f038806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0394006fd0 tx=0x7f0394008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.561+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0390075b50 con 0x7f03a0064f90 2024-03-18T14:49:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.735+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a0112e30 con 0x7f03a0064f90 2024-03-18T14:49:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.736+0000 7f0387fff640 1 -- 172.21.15.12:0/4003252391 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14265 (secure 0 0 0) 0x7f0390079170 con 0x7f03a0064f90 2024-03-18T14:49:06.737 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:49:06.737 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T14:48:38.095054+0000","last_up_change":"2024-03-18T14:47:15.850925+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-18T14:48:34.682425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-18T14:48:35.325444+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6803","nonce":1389034439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6805","nonce":1389034439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6809","nonce":1389034439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1389034439},{"type":"v1","addr":"172.21.15.12:6807","nonce":1389034439}]},"public_addr":"172.21.15.12:6803/1389034439","cluster_addr":"172.21.15.12:6805/1389034439","heartbeat_back_addr":"172.21.15.12:6809/1389034439","heartbeat_front_addr":"172.21.15.12:6807/1389034439","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6811","nonce":2569125971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6813","nonce":2569125971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6817","nonce":2569125971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2569125971},{"type":"v1","addr":"172.21.15.12:6815","nonce":2569125971}]},"public_addr":"172.21.15.12:6811/2569125971","cluster_addr":"172.21.15.12:6813/2569125971","heartbeat_back_addr":"172.21.15.12:6817/2569125971","heartbeat_front_addr":"172.21.15.12:6815/2569125971","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6819","nonce":892771200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6821","nonce":892771200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6825","nonce":892771200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":892771200},{"type":"v1","addr":"172.21.15.12:6823","nonce":892771200}]},"public_addr":"172.21.15.12:6819/892771200","cluster_addr":"172.21.15.12:6821/892771200","heartbeat_back_addr":"172.21.15.12:6825/892771200","heartbeat_front_addr":"172.21.15.12:6823/892771200","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6801","nonce":2368724172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6803","nonce":2368724172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6807","nonce":2368724172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2368724172},{"type":"v1","addr":"172.21.15.100:6805","nonce":2368724172}]},"public_addr":"172.21.15.100:6801/2368724172","cluster_addr":"172.21.15.100:6803/2368724172","heartbeat_back_addr":"172.21.15.100:6807/2368724172","heartbeat_front_addr":"172.21.15.100:6805/2368724172","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6809","nonce":1429143082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6811","nonce":1429143082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6815","nonce":1429143082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1429143082},{"type":"v1","addr":"172.21.15.100:6813","nonce":1429143082}]},"public_addr":"172.21.15.100:6809/1429143082","cluster_addr":"172.21.15.100:6811/1429143082","heartbeat_back_addr":"172.21.15.100:6815/1429143082","heartbeat_front_addr":"172.21.15.100:6813/1429143082","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6817","nonce":3471230267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6819","nonce":3471230267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6823","nonce":3471230267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3471230267},{"type":"v1","addr":"172.21.15.100:6821","nonce":3471230267}]},"public_addr":"172.21.15.100:6817/3471230267","cluster_addr":"172.21.15.100:6819/3471230267","heartbeat_back_addr":"172.21.15.100:6823/3471230267","heartbeat_front_addr":"172.21.15.100:6821/3471230267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 msgr2=0x7f038806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 0x7f038806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0394006fd0 tx=0x7f0394008040 comp rx=0 tx=0).stop 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 msgr2=0x7f03a010ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f039002bda0 tx=0x7f0390009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 shutdown_connections 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0388069210 0x7f038806b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03a00ff390 0x7f03a010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 --2- 172.21.15.12:0/4003252391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a0064f90 0x7f03a010ce20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.737+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 >> 172.21.15.12:0/4003252391 conn(0x7f03a00f6840 msgr2=0x7f03a00f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.738+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 shutdown_connections 2024-03-18T14:49:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.738+0000 7f03a5a6e640 1 -- 172.21.15.12:0/4003252391 wait complete. 2024-03-18T14:49:06.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T14:46:03.095708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-18T14:48:34.682425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-18T14:48:35.325444+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-18T14:49:06.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T14:49:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- 172.21.15.12:0/1294924330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fc370 msgr2=0x7f3d580fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 --2- 172.21.15.12:0/1294924330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fc370 0x7f3d580fc7e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d480048d0 tx=0x7f3d48009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- 172.21.15.12:0/1294924330 shutdown_connections 2024-03-18T14:49:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 --2- 172.21.15.12:0/1294924330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fc370 0x7f3d580fc7e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 --2- 172.21.15.12:0/1294924330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fb0a0 0x7f3d580fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- 172.21.15.12:0/1294924330 >> 172.21.15.12:0/1294924330 conn(0x7f3d580f6880 msgr2=0x7f3d580f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- 172.21.15.12:0/1294924330 shutdown_connections 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- 172.21.15.12:0/1294924330 wait complete. 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 Processor -- start 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- start start 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fc370 0x7f3d58113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5810d920 con 0x7f3d580fb0a0 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d6035d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58114480 con 0x7f3d580fc370 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d5e0f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d5e0f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39278/0 (socket says 172.21.15.12:39278) 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.850+0000 7f3d5e0f9640 1 -- 172.21.15.12:0/544940514 learned_addr learned my addr 172.21.15.12:0/544940514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d5d8f8640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fc370 0x7f3d58113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d5e0f9640 1 -- 172.21.15.12:0/544940514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fc370 msgr2=0x7f3d58113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d5e0f9640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fc370 0x7f3d58113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d5e0f9640 1 -- 172.21.15.12:0/544940514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48005040 con 0x7f3d580fb0a0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d5e0f9640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d54002a50 tx=0x7f3d54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d54007c30 con 0x7f3d580fb0a0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d5400f040 con 0x7f3d580fb0a0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d581145c0 con 0x7f3d580fb0a0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d5400db50 con 0x7f3d580fb0a0 2024-03-18T14:49:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.851+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d58114ae0 con 0x7f3d580fb0a0 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.852+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d581055a0 con 0x7f3d580fb0a0 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.852+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3d5401c400 con 0x7f3d580fb0a0 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.853+0000 7f3d4effd640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 0x7f3d4406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.853+0000 7f3d5d8f8640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 0x7f3d4406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.853+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3d54086ac0 con 0x7f3d580fb0a0 2024-03-18T14:49:06.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.853+0000 7f3d5d8f8640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 0x7f3d4406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d58065cf0 tx=0x7f3d48009da0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:06.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:06.857+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3d5404cdb0 con 0x7f3d580fb0a0 2024-03-18T14:49:07.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.031+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d58114d00 con 0x7f3d580fb0a0 2024-03-18T14:49:07.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.032+0000 7f3d4effd640 1 -- 172.21.15.12:0/544940514 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d58114d00 con 0x7f3d580fb0a0 2024-03-18T14:49:07.033 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 msgr2=0x7f3d4406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 0x7f3d4406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d58065cf0 tx=0x7f3d48009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 msgr2=0x7f3d58065f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d54002a50 tx=0x7f3d54004600 comp rx=0 tx=0).stop 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 shutdown_connections 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3d44069210 0x7f3d4406b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d580fc370 0x7f3d58113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 --2- 172.21.15.12:0/544940514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d580fb0a0 0x7f3d58065f40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 >> 172.21.15.12:0/544940514 conn(0x7f3d580f6880 msgr2=0x7f3d580ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 shutdown_connections 2024-03-18T14:49:07.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.034+0000 7f3d6035d640 1 -- 172.21.15.12:0/544940514 wait complete. 2024-03-18T14:49:07.044 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:06 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4003252391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:49:07.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 -- 172.21.15.12:0/1650464910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 msgr2=0x7fccc40fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1650464910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc40fb480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccc00048d0 tx=0x7fccc0009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 -- 172.21.15.12:0/1650464910 shutdown_connections 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1650464910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc40fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1650464910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc40fb480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 -- 172.21.15.12:0/1650464910 >> 172.21.15.12:0/1650464910 conn(0x7fccc40f6870 msgr2=0x7fccc40f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 -- 172.21.15.12:0/1650464910 shutdown_connections 2024-03-18T14:49:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.145+0000 7fcccc71d640 1 -- 172.21.15.12:0/1650464910 wait complete. 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 Processor -- start 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 -- start start 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc410d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410d970 con 0x7fccc40fb070 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fcccc71d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41122e0 con 0x7fccc40fb9c0 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39296/0 (socket says 172.21.15.12:39296) 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 -- 172.21.15.12:0/1676085167 learned_addr learned my addr 172.21.15.12:0/1676085167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccc9cb8640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc410d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 -- 172.21.15.12:0/1676085167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 msgr2=0x7fccc410d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc410d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 -- 172.21.15.12:0/1676085167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0005040 con 0x7fccc40fb070 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccc9cb8640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:49:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccca4b9640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.146+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc00094c0 con 0x7fccc40fb070 2024-03-18T14:49:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.147+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccc0009620 con 0x7fccc40fb070 2024-03-18T14:49:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.147+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc4112480 con 0x7fccc40fb070 2024-03-18T14:49:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.147+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc00435f0 con 0x7fccc40fb070 2024-03-18T14:49:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.147+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41129a0 con 0x7fccc40fb070 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.148+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc404bcc0 con 0x7fccc40fb070 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.148+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fccc002f070 con 0x7fccc40fb070 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.148+0000 7fccbaffd640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 0x7fccb006b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.148+0000 7fccc9cb8640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 0x7fccb006b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.148+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fccc00a9170 con 0x7fccc40fb070 2024-03-18T14:49:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.149+0000 7fccc9cb8640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 0x7fccb006b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccc410e250 tx=0x7fccb4008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.153+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fccc0075420 con 0x7fccc40fb070 2024-03-18T14:49:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:06 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4003252391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T14:49:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.322+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fccc4104d80 con 0x7fccc40fb070 2024-03-18T14:49:07.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.323+0000 7fccbaffd640 1 -- 172.21.15.12:0/1676085167 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fccc0078a40 con 0x7fccc40fb070 2024-03-18T14:49:07.324 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T14:49:07.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.324+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 msgr2=0x7fccb006b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.324+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 0x7fccb006b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccc410e250 tx=0x7fccb4008040 comp rx=0 tx=0).stop 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.324+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 msgr2=0x7fccc410cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.324+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.324+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 shutdown_connections 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fccb0069220 0x7fccb006b6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc40fb9c0 0x7fccc410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 --2- 172.21.15.12:0/1676085167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fccc40fb070 0x7fccc410cd50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 >> 172.21.15.12:0/1676085167 conn(0x7fccc40f6870 msgr2=0x7fccc40f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 shutdown_connections 2024-03-18T14:49:07.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.325+0000 7fcccc71d640 1 -- 172.21.15.12:0/1676085167 wait complete. 2024-03-18T14:49:07.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-18T14:49:07.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 -- 172.21.15.12:0/274125203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 msgr2=0x7f2f7c0fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/274125203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fdc10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0048d0 tx=0x7f2f6c009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 -- 172.21.15.12:0/274125203 shutdown_connections 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/274125203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 0x7f2f7c0fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/274125203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fdc10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 -- 172.21.15.12:0/274125203 >> 172.21.15.12:0/274125203 conn(0x7f2f7c0f6000 msgr2=0x7f2f7c0f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 -- 172.21.15.12:0/274125203 shutdown_connections 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 -- 172.21.15.12:0/274125203 wait complete. 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.436+0000 7f2f81a6e640 1 Processor -- start 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f81a6e640 1 -- start start 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f81a6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f81a6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 0x7f2f7c0fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f81a6e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0fd460 con 0x7f2f7c0fd800 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f81a6e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0fba20 con 0x7f2f7c0fe150 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39318/0 (socket says 172.21.15.12:39318) 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 -- 172.21.15.12:0/1768202059 learned_addr learned my addr 172.21.15.12:0/1768202059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7affd640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 0x7f2f7c0fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 -- 172.21.15.12:0/1768202059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 msgr2=0x7f2f7c0fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 0x7f2f7c0fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 -- 172.21.15.12:0/1768202059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c005040 con 0x7f2f7c0fd800 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.437+0000 7f2f7b7fe640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c007da0 tx=0x7f2f6c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.438+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c0094c0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.438+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f6c009620 con 0x7f2f7c0fd800 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.438+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c0fbca0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.438+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c03b620 con 0x7f2f7c0fd800 2024-03-18T14:49:07.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.438+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c0fc1c0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.439+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c105570 con 0x7f2f7c0fd800 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.439+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2f6c02f070 con 0x7f2f7c0fd800 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.439+0000 7f2f63fff640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 0x7f2f6406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.439+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2f6c0a9700 con 0x7f2f7c0fd800 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.440+0000 7f2f7affd640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 0x7f2f6406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.440+0000 7f2f7affd640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 0x7f2f6406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0fcbc0 tx=0x7f2f7000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.444+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f2f6c0759b0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.615+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2f7c04bcc0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.615+0000 7f2f63fff640 1 -- 172.21.15.12:0/1768202059 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2f6c078fd0 con 0x7f2f7c0fd800 2024-03-18T14:49:07.616 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 msgr2=0x7f2f6406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 0x7f2f6406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0fcbc0 tx=0x7f2f7000b040 comp rx=0 tx=0).stop 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 msgr2=0x7f2f7c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c007da0 tx=0x7f2f6c009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 shutdown_connections 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f2f64069210 0x7f2f6406b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7c0fe150 0x7f2f7c0fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 --2- 172.21.15.12:0/1768202059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f7c0fd800 0x7f2f7c0fce10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 >> 172.21.15.12:0/1768202059 conn(0x7f2f7c0f6000 msgr2=0x7f2f7c0f75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 shutdown_connections 2024-03-18T14:49:07.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.617+0000 7f2f81a6e640 1 -- 172.21.15.12:0/1768202059 wait complete. 2024-03-18T14:49:07.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 -- 172.21.15.12:0/3147408128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 msgr2=0x7f41140fc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 --2- 172.21.15.12:0/3147408128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f41140fc7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f41040048d0 tx=0x7f4104009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 -- 172.21.15.12:0/3147408128 shutdown_connections 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 --2- 172.21.15.12:0/3147408128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f41140fc7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 --2- 172.21.15.12:0/3147408128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f41140fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 -- 172.21.15.12:0/3147408128 >> 172.21.15.12:0/3147408128 conn(0x7f41140f6850 msgr2=0x7f41140f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 -- 172.21.15.12:0/3147408128 shutdown_connections 2024-03-18T14:49:07.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.726+0000 7f411b99d640 1 -- 172.21.15.12:0/3147408128 wait complete. 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 Processor -- start 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 -- start start 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f4114065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114112060 con 0x7f41140fc380 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41141121a0 con 0x7f41140fb0b0 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39326/0 (socket says 172.21.15.12:39326) 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 -- 172.21.15.12:0/746184274 learned_addr learned my addr 172.21.15.12:0/746184274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4119739640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f4114065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 -- 172.21.15.12:0/746184274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 msgr2=0x7f4114065fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f4114065fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 -- 172.21.15.12:0/746184274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104005040 con 0x7f41140fc380 2024-03-18T14:49:07.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4119739640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f4114065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4118f38640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4104007d20 tx=0x7f4104009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4104009590 con 0x7f41140fc380 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41040096f0 con 0x7f41140fc380 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41141123a0 con 0x7f41140fc380 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.727+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410403b6d0 con 0x7f41140fc380 2024-03-18T14:49:07.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.728+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4114112890 con 0x7f41140fc380 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.729+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4114105590 con 0x7f41140fc380 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.729+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f410402f070 con 0x7f41140fc380 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.729+0000 7f4109ffb640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 0x7f410006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.729+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f41040a9810 con 0x7f41140fc380 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.730+0000 7f4119739640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 0x7f410006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.730+0000 7f4119739640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 0x7f410006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4110004c60 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:07.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.734+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f4104075b40 con 0x7f41140fc380 2024-03-18T14:49:07.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.941+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4114112a70 con 0x7f41140fc380 2024-03-18T14:49:07.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.941+0000 7f4109ffb640 1 -- 172.21.15.12:0/746184274 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4697 (secure 0 0 0) 0x7f4104079160 con 0x7f41140fc380 2024-03-18T14:49:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:49:07.943 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T14:48:53.920975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.12:6827/1453814985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1453814985},{"type":"v1","addr":"172.21.15.12:6827","nonce":1453814985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi012.oheuui","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.12:6829/1193273652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1193273652},{"type":"v1","addr":"172.21.15.12:6829","nonce":1193273652}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-18T14:49:07.943 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:49:07.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 msgr2=0x7f410006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 0x7f410006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4110004c60 tx=0x7f4110009040 comp rx=0 tx=0).stop 2024-03-18T14:49:07.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 msgr2=0x7f4114111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4104007d20 tx=0x7f4104009da0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 shutdown_connections 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f4100069210 0x7f410006b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41140fc380 0x7f4114111b20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 --2- 172.21.15.12:0/746184274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41140fb0b0 0x7f4114065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 >> 172.21.15.12:0/746184274 conn(0x7f41140f6850 msgr2=0x7f41140ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 shutdown_connections 2024-03-18T14:49:07.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:07.943+0000 7f411b99d640 1 -- 172.21.15.12:0/746184274 wait complete. 2024-03-18T14:49:07.955 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-03-18T14:49:07.955 INFO:teuthology.run_tasks:Running task kclient... 2024-03-18T14:49:07.980 INFO:tasks.kclient:Mounting kernel clients... 2024-03-18T14:49:07.980 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-18T14:49:07.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:07.980 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-03-18T14:49:07.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:07.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-03-18T14:49:08.064 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2024-03-18T14:49:08.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:49:08.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:08.065 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-18T14:49:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:08.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-03-18T14:49:08.160 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-03-18T14:49:08.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T14:49:08.162 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-18T14:49:08.162 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T14:49:08.162 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:08.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T14:49:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:07 smithi012 ceph-mon[29182]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/544940514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1676085167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1768202059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/746184274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T14:49:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:07 smithi100 ceph-mon[37807]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/544940514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1676085167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1768202059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T14:49:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/746184274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T14:49:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 -- 172.21.15.12:0/2323899880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c1059d0 msgr2=0x7f828c0ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 --2- 172.21.15.12:0/2323899880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c1059d0 0x7f828c0ff040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f827c004940 tx=0x7f827c009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 -- 172.21.15.12:0/2323899880 shutdown_connections 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 --2- 172.21.15.12:0/2323899880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c1059d0 0x7f828c0ff040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 --2- 172.21.15.12:0/2323899880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c105030 0x7f828c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 -- 172.21.15.12:0/2323899880 >> 172.21.15.12:0/2323899880 conn(0x7f828c0faba0 msgr2=0x7f828c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:10.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 -- 172.21.15.12:0/2323899880 shutdown_connections 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.052+0000 7f8293882700 1 -- 172.21.15.12:0/2323899880 wait complete. 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 Processor -- start 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 -- start start 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c1059d0 0x7f828c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c116730 con 0x7f828c105030 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8293882700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c10fd10 con 0x7f828c1059d0 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60534/0 (socket says 172.21.15.12:60534) 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 -- 172.21.15.12:0/616675271 learned_addr learned my addr 172.21.15.12:0/616675271 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f8290e1d700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c1059d0 0x7f828c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 -- 172.21.15.12:0/616675271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c1059d0 msgr2=0x7f828c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c1059d0 0x7f828c1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.053+0000 7f829161e700 1 -- 172.21.15.12:0/616675271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c005040 con 0x7f828c105030 2024-03-18T14:49:10.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f829161e700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f828800c8c0 tx=0x7f82880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828800a760 con 0x7f828c105030 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f828800ad90 con 0x7f828c105030 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c10fff0 con 0x7f828c105030 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82880113a0 con 0x7f828c105030 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.054+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c110580 con 0x7f828c105030 2024-03-18T14:49:10.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.055+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c100780 con 0x7f828c105030 2024-03-18T14:49:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.055+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8288012400 con 0x7f828c105030 2024-03-18T14:49:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.056+0000 7f8281ffb700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 0x7f827806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.056+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f82880871a0 con 0x7f828c105030 2024-03-18T14:49:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.056+0000 7f8290e1d700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 0x7f827806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:10.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.056+0000 7f8290e1d700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 0x7f827806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827c004890 tx=0x7f827c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:10.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.060+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f8288056f60 con 0x7f828c105030 2024-03-18T14:49:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:09 smithi012 ceph-mon[29182]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:09 smithi100 ceph-mon[37807]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.300+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f828c04c710 con 0x7f828c105030 2024-03-18T14:49:10.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.300+0000 7f8281ffb700 1 -- 172.21.15.12:0/616675271 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8288056af0 con 0x7f828c105030 2024-03-18T14:49:10.301 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-18T14:49:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 msgr2=0x7f827806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 0x7f827806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827c004890 tx=0x7f827c009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 msgr2=0x7f828c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f828800c8c0 tx=0x7f82880044c0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 shutdown_connections 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f82780692a0 0x7f827806b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f828c105030 0x7f828c115ba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 --2- 172.21.15.12:0/616675271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c1059d0 0x7f828c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 >> 172.21.15.12:0/616675271 conn(0x7f828c0faba0 msgr2=0x7f828c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 shutdown_connections 2024-03-18T14:49:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:10.302+0000 7f8293882700 1 -- 172.21.15.12:0/616675271 wait complete. 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T14:49:10.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T14:49:10.905 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T14:49:10.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:10.905 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-03-18T14:49:10.920 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-03-18T14:49:10.921 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:10.922 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-03-18T14:49:10.922 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T14:49:10.923 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-03-18T14:49:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:11.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:11.130 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:10 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/616675271' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T14:49:11.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:11.135 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T14:49:11.162 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-03-18T14:49:11.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:11.164 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-03-18T14:49:11.233 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-03-18T14:49:11.233 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T14:49:11.233 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-18T14:49:11.233 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-18T14:49:11.233 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-18T14:49:11.234 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-03-18T14:49:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:10 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/616675271' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T14:49:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:11.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:11.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:11.516 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-03-18T14:49:11.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:11.531 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-03-18T14:49:11.585 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:11.585 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-03-18T14:49:11.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T14:49:11.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T14:49:11.586 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-03-18T14:49:11.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:11.707 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T14:49:11.707 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:11.707 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-03-18T14:49:11.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T14:49:11.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T14:49:11.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T14:49:11.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T14:49:11.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T14:49:11.708 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-03-18T14:49:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:11.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:11.850 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:11.850 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-03-18T14:49:11.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-03-18T14:49:11.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T14:49:11.850 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-03-18T14:49:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:11.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T14:49:11.931 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:11.945 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T14:49:11.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T14:49:11.945 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:12.000 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-18T14:49:12.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:12.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-18T14:49:12.091 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-18T14:49:12.091 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-18T14:49:12.091 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2024-03-18T14:49:12.197 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-18T14:49:12.197 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-18T14:49:12.197 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2024-03-18T14:49:12.197 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.100:3300" 2024-03-18T14:49:12.197 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-18T14:49:12.203 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-18T14:49:12.203 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-18T14:49:12.203 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2024-03-18T14:49:12.203 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.100:3300" 2024-03-18T14:49:12.203 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.100:6789" 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.100:6789:/ 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=74f0a062-e535-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout: You just mounted a file system that supports labels which does not 2024-03-18T14:49:12.204 INFO:teuthology.orchestra.run.smithi012.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-18T14:49:12.205 INFO:teuthology.orchestra.run.smithi012.stdout: applications will generate AVC messages and not be allowed access to 2024-03-18T14:49:12.205 INFO:teuthology.orchestra.run.smithi012.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-18T14:49:12.206 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T14:49:12.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:12.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:12.230 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:11 smithi012 ceph-mon[29182]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:12.233 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:12.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:11 smithi100 ceph-mon[37807]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:12.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:12.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:12.337 INFO:teuthology.orchestra.run.smithi012.stdout:client24303 2024-03-18T14:49:12.337 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:49:12.337 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T14:49:12.353 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:12.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:12.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:12.453 INFO:teuthology.orchestra.run.smithi012.stdout:client24303 2024-03-18T14:49:12.453 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:12.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:12.512 INFO:teuthology.orchestra.run.smithi012.stdout:client24303 2024-03-18T14:49:12.513 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/74f0a062-e535-11ee-95c9-87774f69a715.client24303/status') 2024-03-18T14:49:12.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:12.577 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.24303 (3)172.21.15.12:0/3414348782 2024-03-18T14:49:12.577 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-03-18T14:49:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-03-18T14:49:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-03-18T14:49:12.577 INFO:teuthology.orchestra.run.smithi012.stderr:71 bytes copied, 2.0436e-05 s, 3.5 MB/s 2024-03-18T14:49:12.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:12.584 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-18T14:49:12.584 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T14:49:12.584 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:12.584 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T14:49:12.848 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:13 smithi012 ceph-mon[29182]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:13 smithi100 ceph-mon[37807]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 -- 172.21.15.12:0/1905708277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c102e00 msgr2=0x7f3b0c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 --2- 172.21.15.12:0/1905708277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c102e00 0x7f3b0c1031d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3afc004940 tx=0x7f3afc009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 -- 172.21.15.12:0/1905708277 shutdown_connections 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 --2- 172.21.15.12:0/1905708277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 --2- 172.21.15.12:0/1905708277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c102e00 0x7f3b0c1031d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 -- 172.21.15.12:0/1905708277 >> 172.21.15.12:0/1905708277 conn(0x7f3b0c0f89b0 msgr2=0x7f3b0c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 -- 172.21.15.12:0/1905708277 shutdown_connections 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 -- 172.21.15.12:0/1905708277 wait complete. 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.590+0000 7f3b12260700 1 Processor -- start 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b12260700 1 -- start start 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b12260700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c102e00 0x7f3b0c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b12260700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b12260700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c114680 con 0x7f3b0c1037a0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b12260700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10da70 con 0x7f3b0c102e00 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60546/0 (socket says 172.21.15.12:60546) 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 -- 172.21.15.12:0/2450527928 learned_addr learned my addr 172.21.15.12:0/2450527928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0b7fe700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c102e00 0x7f3b0c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 -- 172.21.15.12:0/2450527928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c102e00 msgr2=0x7f3b0c113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c102e00 0x7f3b0c113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 -- 172.21.15.12:0/2450527928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc005040 con 0x7f3b0c1037a0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.591+0000 7f3b0affd700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b00002a50 tx=0x7f3b000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.592+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b00007cc0 con 0x7f3b0c1037a0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.592+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c10dd50 con 0x7f3b0c1037a0 2024-03-18T14:49:14.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.592+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b0000f040 con 0x7f3b0c1037a0 2024-03-18T14:49:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.592+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0000db30 con 0x7f3b0c1037a0 2024-03-18T14:49:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.592+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c10e310 con 0x7f3b0c1037a0 2024-03-18T14:49:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.593+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c100110 con 0x7f3b0c1037a0 2024-03-18T14:49:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.593+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3b0000dc90 con 0x7f3b0c1037a0 2024-03-18T14:49:14.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.594+0000 7f3af3fff700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 0x7f3af406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.594+0000 7f3b0b7fe700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 0x7f3af406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.594+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3b00086fc0 con 0x7f3b0c1037a0 2024-03-18T14:49:14.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.594+0000 7f3b0b7fe700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 0x7f3af406b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.598+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3b00056c60 con 0x7f3b0c1037a0 2024-03-18T14:49:14.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.834+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3b0c10f140 con 0x7f3b0c1037a0 2024-03-18T14:49:14.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.834+0000 7f3af3fff700 1 -- 172.21.15.12:0/2450527928 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f3b000567f0 con 0x7f3b0c1037a0 2024-03-18T14:49:14.836 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-18T14:49:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 msgr2=0x7f3af406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 0x7f3af406b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 msgr2=0x7f3b0c114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b00002a50 tx=0x7f3b000046b0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 shutdown_connections 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3af40692b0 0x7f3af406b760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b0c102e00 0x7f3b0c113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 --2- 172.21.15.12:0/2450527928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0c1037a0 0x7f3b0c114030 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 >> 172.21.15.12:0/2450527928 conn(0x7f3b0c0f89b0 msgr2=0x7f3b0c0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 shutdown_connections 2024-03-18T14:49:14.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:14.836+0000 7f3b12260700 1 -- 172.21.15.12:0/2450527928 wait complete. 2024-03-18T14:49:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:14 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2450527928' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T14:49:15.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:14 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2450527928' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T14:49:15.486 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-18T14:49:15.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:15.486 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-03-18T14:49:15.505 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42213sec preferred_lft 42213sec 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-03-18T14:49:15.506 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-03-18T14:49:15.507 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T14:49:15.507 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-18T14:49:15.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:15.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:15.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:15.724 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T14:49:15.751 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-03-18T14:49:15.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:15.753 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-03-18T14:49:15.821 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-03-18T14:49:15.822 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T14:49:15.822 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-18T14:49:15.822 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-18T14:49:15.822 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-18T14:49:15.822 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-18T14:49:15.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:16.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:16.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:16.121 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-18T14:49:16.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:16.136 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-03-18T14:49:16.191 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:16.191 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-18T14:49:16.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-18T14:49:16.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-18T14:49:16.192 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-18T14:49:16.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:15 smithi012 ceph-mon[29182]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:16.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:16.290 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:15 smithi100 ceph-mon[37807]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:16.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:16.314 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-18T14:49:16.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-18T14:49:16.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-18T14:49:16.316 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-18T14:49:16.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:16.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:16.457 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T14:49:16.458 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-03-18T14:49:16.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-03-18T14:49:16.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T14:49:16.458 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-03-18T14:49:16.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:16.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:16.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T14:49:16.538 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:16.552 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-18T14:49:16.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T14:49:16.553 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:16.606 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-18T14:49:16.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:16.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-18T14:49:16.704 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-18T14:49:16.704 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-18T14:49:16.704 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-03-18T14:49:16.810 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-18T14:49:16.810 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-18T14:49:16.810 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-03-18T14:49:16.810 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.100:3300" 2024-03-18T14:49:16.810 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-18T14:49:16.815 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-18T14:49:16.815 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-18T14:49:16.815 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-03-18T14:49:16.815 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.100:3300" 2024-03-18T14:49:16.815 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-18T14:49:16.816 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.100:6789" 2024-03-18T14:49:16.816 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.100:6789:/ 2024-03-18T14:49:16.816 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=74f0a062-e535-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T14:49:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-18T14:49:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-03-18T14:49:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-18T14:49:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-03-18T14:49:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-18T14:49:16.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T14:49:16.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T14:49:16.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:16.846 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:16.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:16.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:16.942 INFO:teuthology.orchestra.run.smithi100.stdout:client14544 2024-03-18T14:49:16.943 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:49:16.943 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T14:49:16.959 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:17.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:17.054 INFO:teuthology.orchestra.run.smithi100.stdout:client14544 2024-03-18T14:49:17.054 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T14:49:17.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T14:49:17.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:17.109 INFO:teuthology.orchestra.run.smithi100.stdout:client14544 2024-03-18T14:49:17.109 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/74f0a062-e535-11ee-95c9-87774f69a715.client14544/status') 2024-03-18T14:49:17.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T14:49:17.169 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.14544 (3)172.21.15.100:0/2705169264 2024-03-18T14:49:17.169 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-03-18T14:49:17.169 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-03-18T14:49:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-03-18T14:49:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:72 bytes copied, 2.0524e-05 s, 3.5 MB/s 2024-03-18T14:49:17.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-18T14:49:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T14:49:17.176 INFO:teuthology.run_tasks:Running task print... 2024-03-18T14:49:17.189 INFO:teuthology.task.print:**** done client 2024-03-18T14:49:17.189 INFO:teuthology.run_tasks:Running task parallel... 2024-03-18T14:49:17.564 INFO:teuthology.task.parallel:starting parallel... 2024-03-18T14:49:17.564 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-18T14:49:17.565 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-18T14:49:17.565 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:49:17.565 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-18T14:49:17.566 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-18T14:49:17.566 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-18T14:49:17.569 INFO:tasks.workunit:Pulling workunits from ref 4e19f704afd9957ea657ab858f9ecf76401fef6f 2024-03-18T14:49:17.570 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-18T14:49:17.570 INFO:tasks.workunit:timeout=3h 2024-03-18T14:49:17.570 INFO:tasks.workunit:cleanup=True 2024-03-18T14:49:17.570 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:17.589 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-03-18 14:48:41.125970156 +0000 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-03-18 14:49:12.231319977 +0000 2024-03-18T14:49:17.590 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-03-18 14:48:41.125970156 +0000 2024-03-18T14:49:17.590 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-18T14:49:17.590 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-18T14:49:17.664 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:17.681 INFO:teuthology.orchestra.run.smithi100.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:17.681 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-18 14:49:17.658275789 +0000 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-18 14:49:17.658275789 +0000 2024-03-18T14:49:17.682 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-18 14:48:41.125970156 +0000 2024-03-18T14:49:17.682 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-18T14:49:17.682 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-18T14:49:17.754 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4e19f704afd9957ea657ab858f9ecf76401fef6f 2024-03-18T14:49:17.755 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 4e19f704afd9957ea657ab858f9ecf76401fef6f 2024-03-18T14:49:17.769 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-18T14:49:17.811 INFO:tasks.workunit.client.1.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-18T14:49:17.850 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:17 smithi100 ceph-mon[37807]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:17 smithi012 ceph-mon[29182]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- 172.21.15.12:0/3382034989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fdfe0 msgr2=0x7f0ff00fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/3382034989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff00fe450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4004940 tx=0x7f0fe4009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- 172.21.15.12:0/3382034989 shutdown_connections 2024-03-18T14:49:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/3382034989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff00fe450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/3382034989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fce70 0x7f0ff00fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- 172.21.15.12:0/3382034989 >> 172.21.15.12:0/3382034989 conn(0x7f0ff00f8550 msgr2=0x7f0ff00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- 172.21.15.12:0/3382034989 shutdown_connections 2024-03-18T14:49:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- 172.21.15.12:0/3382034989 wait complete. 2024-03-18T14:49:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 Processor -- start 2024-03-18T14:49:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- start start 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff010e400 con 0x7f0ff00fce70 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.508+0000 7f0ff7a51700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff010e570 con 0x7f0ff00fdfe0 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59480/0 (socket says 172.21.15.12:59480) 2024-03-18T14:49:19.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 -- 172.21.15.12:0/2634022093 learned_addr learned my addr 172.21.15.12:0/2634022093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff4fec700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 -- 172.21.15.12:0/2634022093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fdfe0 msgr2=0x7f0ff010de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff010de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 -- 172.21.15.12:0/2634022093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4005040 con 0x7f0ff00fce70 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff57ed700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0001db0 tx=0x7f0fe0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe000ed50 con 0x7f0ff00fce70 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff0112fd0 con 0x7f0ff00fce70 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fe0007df0 con 0x7f0ff00fce70 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.509+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe0014640 con 0x7f0ff00fce70 2024-03-18T14:49:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.510+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff0113500 con 0x7f0ff00fce70 2024-03-18T14:49:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.510+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff004c710 con 0x7f0ff00fce70 2024-03-18T14:49:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.511+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0fe0007920 con 0x7f0ff00fce70 2024-03-18T14:49:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.511+0000 7f0fedffb700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 0x7f0fd806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.511+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0fe0086f40 con 0x7f0ff00fce70 2024-03-18T14:49:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.511+0000 7f0ff4fec700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 0x7f0fd806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.512+0000 7f0ff4fec700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 0x7f0fd806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b3b0 tx=0x7f0fe4009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.516+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f0fe0056d30 con 0x7f0ff00fce70 2024-03-18T14:49:19.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.718+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f0ff01058d0 con 0x7f0ff00fce70 2024-03-18T14:49:19.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.718+0000 7f0fedffb700 1 -- 172.21.15.12:0/2634022093 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f0fe00568c0 con 0x7f0ff00fce70 2024-03-18T14:49:19.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 msgr2=0x7f0fd806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 0x7f0fd806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b3b0 tx=0x7f0fe4009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 msgr2=0x7f0ff010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0001db0 tx=0x7f0fe0009ee0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 shutdown_connections 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f0fd80692a0 0x7f0fd806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ff00fce70 0x7f0ff010d8c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 --2- 172.21.15.12:0/2634022093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ff00fdfe0 0x7f0ff010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 >> 172.21.15.12:0/2634022093 conn(0x7f0ff00f8550 msgr2=0x7f0ff00f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 shutdown_connections 2024-03-18T14:49:19.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:19.720+0000 7f0ff7a51700 1 -- 172.21.15.12:0/2634022093 wait complete. 2024-03-18T14:49:20.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:19 smithi012 ceph-mon[29182]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-18T14:49:20.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:19 smithi100 ceph-mon[37807]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-18T14:49:20.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-18T14:49:20.790 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:21.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:20 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:21.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:20 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:22.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:21 smithi012 ceph-mon[29182]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-03-18T14:49:22.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:21 smithi100 ceph-mon[37807]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-03-18T14:49:22.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 -- 172.21.15.12:0/4020374551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 msgr2=0x7fc6140ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:22.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 --2- 172.21.15.12:0/4020374551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc6140ffa40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc604004940 tx=0x7fc604009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 -- 172.21.15.12:0/4020374551 shutdown_connections 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 --2- 172.21.15.12:0/4020374551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc614102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 --2- 172.21.15.12:0/4020374551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc6140ffa40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.520+0000 7fc61a621700 1 -- 172.21.15.12:0/4020374551 >> 172.21.15.12:0/4020374551 conn(0x7fc6140f7430 msgr2=0x7fc6140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 -- 172.21.15.12:0/4020374551 shutdown_connections 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 -- 172.21.15.12:0/4020374551 wait complete. 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 Processor -- start 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 -- start start 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc61410dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61410e5d0 con 0x7fc614066260 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.521+0000 7fc61a621700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61410ee20 con 0x7fc6140fff80 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59496/0 (socket says 172.21.15.12:59496) 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 -- 172.21.15.12:0/4146030194 learned_addr learned my addr 172.21.15.12:0/4146030194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc61410dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 -- 172.21.15.12:0/4146030194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 msgr2=0x7fc61410dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc61410dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 -- 172.21.15.12:0/4146030194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc604005040 con 0x7fc614066260 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc6137fe700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc61410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc613fff700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc60400b970 tx=0x7fc604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60400e040 con 0x7fc614066260 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc604004290 con 0x7fc614066260 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61410f0a0 con 0x7fc614066260 2024-03-18T14:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.522+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60401c3f0 con 0x7fc614066260 2024-03-18T14:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.523+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61410f5e0 con 0x7fc614066260 2024-03-18T14:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.524+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61405f5f0 con 0x7fc614066260 2024-03-18T14:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.524+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc60400bbf0 con 0x7fc614066260 2024-03-18T14:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.524+0000 7fc610ff9700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 0x7fc5fc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.525+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc604088450 con 0x7fc614066260 2024-03-18T14:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.525+0000 7fc6137fe700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 0x7fc5fc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.525+0000 7fc6137fe700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 0x7fc5fc06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc608010520 tx=0x7fc60800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.529+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fc604058190 con 0x7fc614066260 2024-03-18T14:49:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.739+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fc61410f960 con 0x7fc614066260 2024-03-18T14:49:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.740+0000 7fc610ff9700 1 -- 172.21.15.12:0/4146030194 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc604057d20 con 0x7fc614066260 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 msgr2=0x7fc5fc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 0x7fc5fc06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc608010520 tx=0x7fc60800b040 comp rx=0 tx=0).stop 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 msgr2=0x7fc61410da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc60400b970 tx=0x7fc604009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 shutdown_connections 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fc5fc0692b0 0x7fc5fc06b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc614066260 0x7fc61410da90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 --2- 172.21.15.12:0/4146030194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6140fff80 0x7fc61410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.742+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 >> 172.21.15.12:0/4146030194 conn(0x7fc6140f7430 msgr2=0x7fc6140f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.743+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 shutdown_connections 2024-03-18T14:49:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:22.743+0000 7fc61a621700 1 -- 172.21.15.12:0/4146030194 wait complete. 2024-03-18T14:49:23.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-18T14:49:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:24.214 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:23 smithi012 ceph-mon[29182]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-18T14:49:24.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:23 smithi100 ceph-mon[37807]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/2169781847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fe040 msgr2=0x7ff0c80fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/2169781847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fe040 0x7ff0c80fe4b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc004940 tx=0x7ff0bc009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/2169781847 shutdown_connections 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/2169781847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fe040 0x7ff0c80fe4b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/2169781847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fce40 0x7ff0c80fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/2169781847 >> 172.21.15.12:0/2169781847 conn(0x7ff0c80f8510 msgr2=0x7ff0c80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/2169781847 shutdown_connections 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/2169781847 wait complete. 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.738+0000 7ff0cdc5f700 1 Processor -- start 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0cdc5f700 1 -- start start 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0cdc5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0cdc5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 0x7ff0c810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0cdc5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c810e570 con 0x7ff0c80fce40 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0cdc5f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c810edc0 con 0x7ff0c80fe040 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59512/0 (socket says 172.21.15.12:59512) 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c77fe700 1 -- 172.21.15.12:0/465275121 learned_addr learned my addr 172.21.15.12:0/465275121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c6ffd700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 0x7ff0c810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c6ffd700 1 -- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 msgr2=0x7ff0c810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c6ffd700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c6ffd700 1 -- 172.21.15.12:0/465275121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc005040 con 0x7ff0c80fe040 2024-03-18T14:49:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.739+0000 7ff0c77fe700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0c6ffd700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 0x7ff0c810df70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00b3b0 tx=0x7ff0bc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc00e040 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0bc00bbf0 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c810f040 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc013660 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.740+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c810f580 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.741+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c805f5f0 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.741+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff0bc00bd60 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.742+0000 7ff0affff700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 0x7ff0b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.742+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff0bc088640 con 0x7ff0c80fe040 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.742+0000 7ff0c77fe700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 0x7ff0b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.742+0000 7ff0c77fe700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 0x7ff0b006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800e560 tx=0x7ff0b800b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.746+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7ff0bc058360 con 0x7ff0c80fe040 2024-03-18T14:49:25.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.947+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7ff0c804bed0 con 0x7ff0c80fe040 2024-03-18T14:49:25.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.948+0000 7ff0affff700 1 -- 172.21.15.12:0/465275121 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7ff0bc057ef0 con 0x7ff0c80fe040 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 msgr2=0x7ff0b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 0x7ff0b006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800e560 tx=0x7ff0b800b040 comp rx=0 tx=0).stop 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 msgr2=0x7ff0c810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 0x7ff0c810df70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00b3b0 tx=0x7ff0bc009e30 comp rx=0 tx=0).stop 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 shutdown_connections 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7ff0b00692a0 0x7ff0b006b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0c80fce40 0x7ff0c810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 --2- 172.21.15.12:0/465275121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0c80fe040 0x7ff0c810df70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.950+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 >> 172.21.15.12:0/465275121 conn(0x7ff0c80f8510 msgr2=0x7ff0c80f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.951+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 shutdown_connections 2024-03-18T14:49:25.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:25.951+0000 7ff0cdc5f700 1 -- 172.21.15.12:0/465275121 wait complete. 2024-03-18T14:49:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:25 smithi100 ceph-mon[37807]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-18T14:49:26.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:25 smithi012 ceph-mon[29182]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-18T14:49:26.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-18T14:49:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:49:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:27 smithi012 ceph-mon[29182]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-18T14:49:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:27 smithi100 ceph-mon[37807]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 -- 172.21.15.12:0/1216442053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6928066ed0 msgr2=0x7f69280ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 --2- 172.21.15.12:0/1216442053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6928066ed0 0x7f69280ffab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6920009ce0 tx=0x7f6920004d10 comp rx=0 tx=0).stop 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 -- 172.21.15.12:0/1216442053 shutdown_connections 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 --2- 172.21.15.12:0/1216442053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280ffff0 0x7f69281023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 --2- 172.21.15.12:0/1216442053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6928066ed0 0x7f69280ffab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 -- 172.21.15.12:0/1216442053 >> 172.21.15.12:0/1216442053 conn(0x7f69280f7480 msgr2=0x7f69280f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 -- 172.21.15.12:0/1216442053 shutdown_connections 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.861+0000 7f692fb1f700 1 -- 172.21.15.12:0/1216442053 wait complete. 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 Processor -- start 2024-03-18T14:49:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 -- start start 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 0x7f69280fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280fd900 con 0x7f69280ffff0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692fb1f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280fbef0 con 0x7f6928066ed0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d8bb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 0x7f69280fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d8bb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 0x7f69280fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:55994/0 (socket says 172.21.15.12:55994) 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37144/0 (socket says 172.21.15.12:37144) 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d8bb700 1 -- 172.21.15.12:0/3362494470 learned_addr learned my addr 172.21.15.12:0/3362494470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 -- 172.21.15.12:0/3362494470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 msgr2=0x7f69280fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 0x7f69280fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 -- 172.21.15.12:0/3362494470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009940 con 0x7f69280ffff0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.862+0000 7f692d0ba700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6928067390 tx=0x7f691c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.863+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c0106f0 con 0x7f69280ffff0 2024-03-18T14:49:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.863+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f691c010d20 con 0x7f69280ffff0 2024-03-18T14:49:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.863+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69280fc1a0 con 0x7f69280ffff0 2024-03-18T14:49:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.863+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c0113a0 con 0x7f69280ffff0 2024-03-18T14:49:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.863+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69280fc760 con 0x7f69280ffff0 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.864+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692804c710 con 0x7f69280ffff0 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.864+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f691c010850 con 0x7f69280ffff0 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.864+0000 7f69267fc700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 0x7f691006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.865+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f691c087040 con 0x7f69280ffff0 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.865+0000 7f692d8bb700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 0x7f691006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:49:28.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.865+0000 7f692d8bb700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 0x7f691006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6920009f90 tx=0x7f6920004d10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:49:28.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:28.869+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f691c056e00 con 0x7f69280ffff0 2024-03-18T14:49:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:49:29.089+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f69280f8f70 con 0x7f69100692a0 2024-03-18T14:49:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:29 smithi100 ceph-mon[37807]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-18T14:49:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:29 smithi100 ceph-mon[37807]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:49:30.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:29 smithi012 ceph-mon[29182]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-18T14:49:30.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:29 smithi012 ceph-mon[29182]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:49:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:31 smithi100 ceph-mon[37807]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:31 smithi012 ceph-mon[29182]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:33 smithi100 ceph-mon[37807]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:34.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:33 smithi012 ceph-mon[29182]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:35 smithi100 ceph-mon[37807]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-18T14:49:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:36.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:35 smithi012 ceph-mon[29182]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-18T14:49:36.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:38 smithi100 ceph-mon[37807]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:38.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:38 smithi012 ceph-mon[29182]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:40.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:40 smithi100 ceph-mon[37807]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:40.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:40 smithi012 ceph-mon[29182]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:42 smithi100 ceph-mon[37807]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:42 smithi012 ceph-mon[29182]: pgmap v208: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:44 smithi100 ceph-mon[37807]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:44.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:44 smithi012 ceph-mon[29182]: pgmap v209: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:46 smithi100 ceph-mon[37807]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:46.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:46 smithi012 ceph-mon[29182]: pgmap v210: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:48.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:48 smithi012 ceph-mon[29182]: pgmap v211: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:48.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:48 smithi100 ceph-mon[37807]: pgmap v211: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:50 smithi100 ceph-mon[37807]: pgmap v212: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:50 smithi012 ceph-mon[29182]: pgmap v212: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:49:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:51 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:51 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:49:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:52 smithi100 ceph-mon[37807]: pgmap v213: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:52 smithi012 ceph-mon[29182]: pgmap v213: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:49:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:54 smithi100 ceph-mon[37807]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:54 smithi012 ceph-mon[29182]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-18T14:49:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:56 smithi100 ceph-mon[37807]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-03-18T14:49:56.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:56 smithi012 ceph-mon[29182]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-03-18T14:49:58.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:49:58 smithi100 ceph-mon[37807]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-03-18T14:49:58.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:49:58 smithi012 ceph-mon[29182]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-03-18T14:50:00.292 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:00 smithi012 ceph-mon[29182]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-18T14:50:00.292 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:00 smithi012 ceph-mon[29182]: overall HEALTH_OK 2024-03-18T14:50:00.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:00 smithi100 ceph-mon[37807]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-18T14:50:00.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:00 smithi100 ceph-mon[37807]: overall HEALTH_OK 2024-03-18T14:50:01.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.956+0000 7f69267fc700 1 -- 172.21.15.12:0/3362494470 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f69280f8f70 con 0x7f69100692a0 2024-03-18T14:50:01.958 INFO:teuthology.orchestra.run.smithi012.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 msgr2=0x7f691006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 0x7f691006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6920009f90 tx=0x7f6920004d10 comp rx=0 tx=0).stop 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 msgr2=0x7f69280fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6928067390 tx=0x7f691c0043c0 comp rx=0 tx=0).stop 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 shutdown_connections 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69100692a0 0x7f691006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6928066ed0 0x7f69280fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 --2- 172.21.15.12:0/3362494470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69280ffff0 0x7f69280fb9b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 >> 172.21.15.12:0/3362494470 conn(0x7f69280f7480 msgr2=0x7f69280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 shutdown_connections 2024-03-18T14:50:01.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:01.959+0000 7f692fb1f700 1 -- 172.21.15.12:0/3362494470 wait complete. 2024-03-18T14:50:02.035 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:02 smithi012 ceph-mon[29182]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-18T14:50:02.035 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:02.035 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:02 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:02 smithi100 ceph-mon[37807]: pgmap v218: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-18T14:50:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:02 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:02 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:02.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-18T14:50:03.046 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:50:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:03 smithi012 ceph-mon[29182]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:50:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:50:03.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:03 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:03 smithi100 ceph-mon[37807]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:50:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:50:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:03 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:04 smithi100 ceph-mon[37807]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-18T14:50:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:04 smithi100 ceph-mon[37807]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:50:04.318 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:04 smithi012 ceph-mon[29182]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-18T14:50:04.319 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:04 smithi012 ceph-mon[29182]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- 172.21.15.12:0/41950422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 msgr2=0x7fe1140ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/41950422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe1140ffa60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe110004940 tx=0x7fe110009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- 172.21.15.12:0/41950422 shutdown_connections 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/41950422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 0x7fe114102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/41950422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe1140ffa60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- 172.21.15.12:0/41950422 >> 172.21.15.12:0/41950422 conn(0x7fe1140f7490 msgr2=0x7fe1140f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- 172.21.15.12:0/41950422 shutdown_connections 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- 172.21.15.12:0/41950422 wait complete. 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 Processor -- start 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.001+0000 7fe11bcdf700 1 -- start start 2024-03-18T14:50:05.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe11bcdf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe11bcdf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 0x7fe11410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe11bcdf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11410e460 con 0x7fe114066ef0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe11bcdf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11410e5d0 con 0x7fe1140fffa0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55562/0 (socket says 172.21.15.12:55562) 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 -- 172.21.15.12:0/235240338 learned_addr learned my addr 172.21.15.12:0/235240338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe11927a700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 0x7fe11410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 -- 172.21.15.12:0/235240338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 msgr2=0x7fe11410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 0x7fe11410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 -- 172.21.15.12:0/235240338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110005040 con 0x7fe114066ef0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe119a7b700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.002+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11000e030 con 0x7fe114066ef0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.003+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe110013de0 con 0x7fe114066ef0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.003+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe114112f80 con 0x7fe114066ef0 2024-03-18T14:50:05.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.003+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11001ce50 con 0x7fe114066ef0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.003+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141133e0 con 0x7fe114066ef0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.004+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11404c710 con 0x7fe114066ef0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.004+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe11001c400 con 0x7fe114066ef0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.005+0000 7fe10a7fc700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 0x7fe10006b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.005+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe110088c20 con 0x7fe114066ef0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.005+0000 7fe11927a700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 0x7fe10006b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.005+0000 7fe11927a700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 0x7fe10006b6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe104009b80 tx=0x7fe10400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:05.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.009+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe110058960 con 0x7fe114066ef0 2024-03-18T14:50:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.381+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1140f8f80 con 0x7fe100069240 2024-03-18T14:50:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.382+0000 7fe10a7fc700 1 -- 172.21.15.12:0/235240338 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fe1140f8f80 con 0x7fe100069240 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 msgr2=0x7fe10006b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 0x7fe10006b6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe104009b80 tx=0x7fe10400b040 comp rx=0 tx=0).stop 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 msgr2=0x7fe11410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 shutdown_connections 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe100069240 0x7fe10006b6f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe114066ef0 0x7fe11410d920 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 --2- 172.21.15.12:0/235240338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1140fffa0 0x7fe11410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.385+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 >> 172.21.15.12:0/235240338 conn(0x7fe1140f7490 msgr2=0x7fe1140f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.386+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 shutdown_connections 2024-03-18T14:50:05.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.386+0000 7fe11bcdf700 1 -- 172.21.15.12:0/235240338 wait complete. 2024-03-18T14:50:05.401 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:50:05.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 -- 172.21.15.12:0/1399438124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fe050 msgr2=0x7f78000fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 --2- 172.21.15.12:0/1399438124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fe050 0x7f78000fe4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004940 tx=0x7f77f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 -- 172.21.15.12:0/1399438124 shutdown_connections 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 --2- 172.21.15.12:0/1399438124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fe050 0x7f78000fe4c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 --2- 172.21.15.12:0/1399438124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78000fce50 0x7f78000fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 -- 172.21.15.12:0/1399438124 >> 172.21.15.12:0/1399438124 conn(0x7f78000f8580 msgr2=0x7f78000fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 -- 172.21.15.12:0/1399438124 shutdown_connections 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.539+0000 7f7806951700 1 -- 172.21.15.12:0/1399438124 wait complete. 2024-03-18T14:50:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 Processor -- start 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 -- start start 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78000fe050 0x7f780010bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f77fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780010c240 con 0x7f78000fce50 2024-03-18T14:50:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f7806951700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780010c3b0 con 0x7f78000fe050 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f77fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55586/0 (socket says 172.21.15.12:55586) 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.540+0000 7f77fffff700 1 -- 172.21.15.12:0/3815187156 learned_addr learned my addr 172.21.15.12:0/3815187156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fffff700 1 -- 172.21.15.12:0/3815187156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78000fe050 msgr2=0x7f780010bc40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fffff700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78000fe050 0x7f780010bc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fffff700 1 -- 172.21.15.12:0/3815187156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4005040 con 0x7f78000fce50 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fffff700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77f000eb70 tx=0x7f77f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77f0007cc0 con 0x7f78000fce50 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77f000f040 con 0x7f78000fce50 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77f000cb60 con 0x7f78000fce50 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.541+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7800110db0 con 0x7f78000fce50 2024-03-18T14:50:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.542+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7800111230 con 0x7f78000fce50 2024-03-18T14:50:05.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.543+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78001019e0 con 0x7f78000fce50 2024-03-18T14:50:05.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.543+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f77f000ccc0 con 0x7f78000fce50 2024-03-18T14:50:05.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.544+0000 7f77fcff9700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 0x7f77e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:05.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.544+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f77f0086c50 con 0x7f78000fce50 2024-03-18T14:50:05.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.544+0000 7f77ff7fe700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 0x7f77e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:05.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.545+0000 7f77ff7fe700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 0x7f77e806b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77f4013870 tx=0x7f77f4009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:05.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.549+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f77f004ddb0 con 0x7f78000fce50 2024-03-18T14:50:05.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.983+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f780002b450 con 0x7f77e80692a0 2024-03-18T14:50:05.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.985+0000 7f77fcff9700 1 -- 172.21.15.12:0/3815187156 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f780002b450 con 0x7f77e80692a0 2024-03-18T14:50:05.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 msgr2=0x7f77e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 0x7f77e806b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77f4013870 tx=0x7f77f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 msgr2=0x7f780010b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77f000eb70 tx=0x7f77f000b040 comp rx=0 tx=0).stop 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 shutdown_connections 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f77e80692a0 0x7f77e806b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78000fce50 0x7f780010b700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 --2- 172.21.15.12:0/3815187156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78000fe050 0x7f780010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.988+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 >> 172.21.15.12:0/3815187156 conn(0x7f78000f8580 msgr2=0x7f78000f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.989+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 shutdown_connections 2024-03-18T14:50:05.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:05.989+0000 7f7806951700 1 -- 172.21.15.12:0/3815187156 wait complete. 2024-03-18T14:50:06.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- 172.21.15.12:0/1998491033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fe480 msgr2=0x7f65780fe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 --2- 172.21.15.12:0/1998491033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fe480 0x7f65780fe890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f656c004940 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- 172.21.15.12:0/1998491033 shutdown_connections 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 --2- 172.21.15.12:0/1998491033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fedd0 0x7f65780faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 --2- 172.21.15.12:0/1998491033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fe480 0x7f65780fe890 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- 172.21.15.12:0/1998491033 >> 172.21.15.12:0/1998491033 conn(0x7f65780f6ba0 msgr2=0x7f65780f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- 172.21.15.12:0/1998491033 shutdown_connections 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- 172.21.15.12:0/1998491033 wait complete. 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 Processor -- start 2024-03-18T14:50:06.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.105+0000 7f657edfd700 1 -- start start 2024-03-18T14:50:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f657edfd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fe480 0x7f657810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f657edfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f657edfd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657810e4f0 con 0x7f65780fedd0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f657edfd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657810ed40 con 0x7f65780fe480 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f657cb99700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fe480 0x7f657810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55592/0 (socket says 172.21.15.12:55592) 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 -- 172.21.15.12:0/773561941 learned_addr learned my addr 172.21.15.12:0/773561941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 -- 172.21.15.12:0/773561941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fe480 msgr2=0x7f657810d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fe480 0x7f657810d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.106+0000 7f6577fff700 1 -- 172.21.15.12:0/773561941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c005040 con 0x7f65780fedd0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f6577fff700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6564002a80 tx=0x7f65640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6564007cc0 con 0x7f65780fedd0 2024-03-18T14:50:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657810f020 con 0x7f65780fedd0 2024-03-18T14:50:06.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657810f560 con 0x7f65780fedd0 2024-03-18T14:50:06.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f656400f040 con 0x7f65780fedd0 2024-03-18T14:50:06.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.107+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f656400dac0 con 0x7f65780fedd0 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.108+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657805f5f0 con 0x7f65780fedd0 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.109+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f656400dcc0 con 0x7f65780fedd0 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.109+0000 7f65757fa700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 0x7f656806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.109+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6564086e50 con 0x7f65780fedd0 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.109+0000 7f657cb99700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 0x7f656806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.110+0000 7f657cb99700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 0x7f656806fb60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b3b0 tx=0x7f656c009e30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.114+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f6564056c10 con 0x7f65780fedd0 2024-03-18T14:50:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:06 smithi012 ceph-mon[29182]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:50:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:06 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:06.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:06 smithi100 ceph-mon[37807]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:50:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:06 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:06.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.338+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f657810f890 con 0x7f656806d6b0 2024-03-18T14:50:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.349+0000 7f65757fa700 1 -- 172.21.15.12:0/773561941 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f657810f890 con 0x7f656806d6b0 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (5m) 60s ago 6m 24.7M - 0.25.0 c8568f914cd2 3a6187ef0acb 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (7m) 60s ago 7m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (6m) 68s ago 6m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (5m) 60s ago 6m 80.3M - 9.4.7 954c08fa6188 a14a00878205 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (80s) 60s ago 80s 14.4M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (86s) 60s ago 85s 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (83s) 68s ago 82s 13.5M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:50:06.350 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (78s) 68s ago 77s 10.4M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (8m) 60s ago 8m 480M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (5m) 68s ago 5m 434M - 17.2.7-684-g6716c9f3 19857db3290c 47f9c2dcc3f5 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (8m) 60s ago 8m 44.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (5m) 68s ago 5m 35.9M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (7m) 60s ago 7m 14.4M - 1.5.0 0da6a335fe13 4ebf4cd84beb 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (5m) 68s ago 5m 17.3M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (5m) 60s ago 5m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (4m) 60s ago 4m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (4m) 60s ago 4m 46.7M 4289M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (3m) 68s ago 3m 47.1M 5312M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (3m) 68s ago 3m 40.8M 5312M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (2m) 68s ago 2m 41.0M 5312M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:50:06.351 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (5m) 60s ago 6m 53.0M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:50:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.351+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 msgr2=0x7f656806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.351+0000 7f657edfd700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 0x7f656806fb60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b3b0 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.351+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 msgr2=0x7f657810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.351+0000 7f657edfd700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6564002a80 tx=0x7f65640046b0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 shutdown_connections 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f656806d6b0 0x7f656806fb60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65780fe480 0x7f657810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 --2- 172.21.15.12:0/773561941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65780fedd0 0x7f657810def0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 >> 172.21.15.12:0/773561941 conn(0x7f65780f6ba0 msgr2=0x7f65780f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 shutdown_connections 2024-03-18T14:50:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.352+0000 7f657edfd700 1 -- 172.21.15.12:0/773561941 wait complete. 2024-03-18T14:50:06.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 -- 172.21.15.12:0/209922017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 msgr2=0x7f6a080fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/209922017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a080fd290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f69f8004940 tx=0x7f69f8009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:06.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 -- 172.21.15.12:0/209922017 shutdown_connections 2024-03-18T14:50:06.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/209922017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 0x7f6a080fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/209922017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a080fd290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 -- 172.21.15.12:0/209922017 >> 172.21.15.12:0/209922017 conn(0x7f6a080f8580 msgr2=0x7f6a080fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:06.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 -- 172.21.15.12:0/209922017 shutdown_connections 2024-03-18T14:50:06.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.464+0000 7f6a0da4d700 1 -- 172.21.15.12:0/209922017 wait complete. 2024-03-18T14:50:06.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 Processor -- start 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 -- start start 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 0x7f6a0810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0810e590 con 0x7f6a080fce80 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a0da4d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0810ede0 con 0x7f6a080fdff0 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a06ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a06ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55606/0 (socket says 172.21.15.12:55606) 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.465+0000 7f6a06ffd700 1 -- 172.21.15.12:0/1053702247 learned_addr learned my addr 172.21.15.12:0/1053702247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a067fc700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 0x7f6a0810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a06ffd700 1 -- 172.21.15.12:0/1053702247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 msgr2=0x7f6a0810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a06ffd700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 0x7f6a0810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a06ffd700 1 -- 172.21.15.12:0/1053702247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8005040 con 0x7f6a080fce80 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a06ffd700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69f800b3b0 tx=0x7f69f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f800e040 con 0x7f6a080fce80 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0810f060 con 0x7f6a080fce80 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69f8004290 con 0x7f6a080fce80 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f801c3f0 con 0x7f6a080fce80 2024-03-18T14:50:06.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.466+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0810f5a0 con 0x7f6a080fce80 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.467+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0805f5f0 con 0x7f6a080fce80 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.468+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f69f800bbf0 con 0x7f6a080fce80 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.469+0000 7f69ef7fe700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 0x7f69f006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.469+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f69f80883f0 con 0x7f6a080fce80 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.469+0000 7f6a067fc700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 0x7f69f006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.469+0000 7f6a067fc700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 0x7f69f006b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69fc004f40 tx=0x7f69fc00b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.476+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f69f8058250 con 0x7f6a080fce80 2024-03-18T14:50:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.756+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a0810f8d0 con 0x7f6a080fce80 2024-03-18T14:50:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.756+0000 7f69ef7fe700 1 -- 172.21.15.12:0/1053702247 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f69f8057de0 con 0x7f6a080fce80 2024-03-18T14:50:06.757 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:50:06.757 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:06.758 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:50:06.759 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 14 2024-03-18T14:50:06.759 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:50:06.759 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:50:06.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 msgr2=0x7f69f006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 0x7f69f006b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69fc004f40 tx=0x7f69fc00b040 comp rx=0 tx=0).stop 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 msgr2=0x7f6a0810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69f800b3b0 tx=0x7f69f8009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 shutdown_connections 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f69f00692b0 0x7f69f006b760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a080fce80 0x7f6a0810da50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 --2- 172.21.15.12:0/1053702247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a080fdff0 0x7f6a0810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.758+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 >> 172.21.15.12:0/1053702247 conn(0x7f6a080f8580 msgr2=0x7f6a080f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.759+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 shutdown_connections 2024-03-18T14:50:06.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.759+0000 7f6a0da4d700 1 -- 172.21.15.12:0/1053702247 wait complete. 2024-03-18T14:50:06.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 -- 172.21.15.12:0/4281750798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 msgr2=0x7f28cc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 --2- 172.21.15.12:0/4281750798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc0fd3f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28c0004940 tx=0x7f28c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:06.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 -- 172.21.15.12:0/4281750798 shutdown_connections 2024-03-18T14:50:06.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 --2- 172.21.15.12:0/4281750798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc0fd3f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 --2- 172.21.15.12:0/4281750798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 0x7f28cc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 -- 172.21.15.12:0/4281750798 >> 172.21.15.12:0/4281750798 conn(0x7f28cc0f7430 msgr2=0x7f28cc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 -- 172.21.15.12:0/4281750798 shutdown_connections 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 -- 172.21.15.12:0/4281750798 wait complete. 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.863+0000 7f28d1c91700 1 Processor -- start 2024-03-18T14:50:06.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28d1c91700 1 -- start start 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28d1c91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 0x7f28cc10c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28d1c91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28d1c91700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc10d3c0 con 0x7f28cc0fcf80 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28d1c91700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc10dc10 con 0x7f28cc0fbcd0 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55626/0 (socket says 172.21.15.12:55626) 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 -- 172.21.15.12:0/3035543765 learned_addr learned my addr 172.21.15.12:0/3035543765 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:06.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28cb7fe700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 0x7f28cc10c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 -- 172.21.15.12:0/3035543765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 msgr2=0x7f28cc10c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 0x7f28cc10c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.864+0000 7f28caffd700 1 -- 172.21.15.12:0/3035543765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0005040 con 0x7f28cc0fcf80 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28caffd700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28c0013870 tx=0x7f28c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c0021070 con 0x7f28cc0fcf80 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc10de90 con 0x7f28cc0fcf80 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28c0009430 con 0x7f28cc0fcf80 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c000e070 con 0x7f28cc0fcf80 2024-03-18T14:50:06.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.865+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc10e350 con 0x7f28cc0fcf80 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.866+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc05f5f0 con 0x7f28cc0fcf80 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.866+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f28c0026410 con 0x7f28cc0fcf80 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.867+0000 7f28b3fff700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 0x7f28b406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.867+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f28c0088450 con 0x7f28cc0fcf80 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.867+0000 7f28cb7fe700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 0x7f28b406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:06.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.867+0000 7f28cb7fe700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 0x7f28b406b6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009b80 tx=0x7f28bc00b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:06.871+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f28c0058280 con 0x7f28cc0fcf80 2024-03-18T14:50:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.084+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28cc10e740 con 0x7f28b4069230 2024-03-18T14:50:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.085+0000 7f28b3fff700 1 -- 172.21.15.12:0/3035543765 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f28cc10e740 con 0x7f28b4069230 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "0/2 daemons upgraded", 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 image on host smithi100", 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:50:07.086 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 msgr2=0x7f28b406b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 0x7f28b406b6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009b80 tx=0x7f28bc00b040 comp rx=0 tx=0).stop 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 msgr2=0x7f28cc10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28c0013870 tx=0x7f28c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 shutdown_connections 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f28b4069230 0x7f28b406b6e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28cc0fbcd0 0x7f28cc10c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 --2- 172.21.15.12:0/3035543765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28cc0fcf80 0x7f28cc10cdc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 >> 172.21.15.12:0/3035543765 conn(0x7f28cc0f7430 msgr2=0x7f28cc0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 shutdown_connections 2024-03-18T14:50:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:07.087+0000 7f28d1c91700 1 -- 172.21.15.12:0/3035543765 wait complete. 2024-03-18T14:50:07.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:07 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1053702247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:50:07.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:07.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:07.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:07.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:07.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:07.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:07 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1053702247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:50:07.948 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to '4e19f704afd9957ea657ab858f9ecf76401fef6f'. 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.949 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T14:50:07.950 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-03-18T14:50:07.950 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at 4e19f704afd ignorelist mon down 2024-03-18T14:50:07.954 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-18T14:50:07.974 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T14:50:07.976 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T14:50:07.976 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: Upgrade: Need to upgrade myself (mgr.smithi012.tmscgt) 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi100 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-03-18T14:50:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:08 smithi012 ceph-mon[29182]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: Upgrade: Need to upgrade myself (mgr.smithi012.tmscgt) 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi100 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-03-18T14:50:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:08 smithi100 ceph-mon[37807]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:08.783 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T14:50:08.849 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T14:50:08.891 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T14:50:08.892 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T14:50:08.893 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T14:50:08.931 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T14:50:08.934 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T14:50:08.934 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-18T14:50:08.991 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-18T14:50:08.993 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-18T14:50:08.994 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4e19f704afd9957ea657ab858f9ecf76401fef6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-18T14:50:09.050 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir -p fsstress 2024-03-18T14:50:09.052 INFO:tasks.workunit.client.0.smithi012.stderr:+ pushd fsstress 2024-03-18T14:50:09.052 INFO:tasks.workunit.client.0.smithi012.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-18T14:50:09.052 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-18T14:50:10.390 INFO:tasks.workunit.client.0.smithi012.stderr:+ tar xzf ltp-full.tgz 2024-03-18T14:50:10.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:10 smithi012 ceph-mon[29182]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-18T14:50:10.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:10 smithi100 ceph-mon[37807]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-18T14:50:12.347 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:12 smithi100 ceph-mon[37807]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-03-18T14:50:12.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:12 smithi012 ceph-mon[29182]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-03-18T14:50:14.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:14 smithi012 ceph-mon[29182]: pgmap v224: 65 pgs: 65 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 112 KiB/s rd, 277 KiB/s wr, 3 op/s 2024-03-18T14:50:14.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:14 smithi100 ceph-mon[37807]: pgmap v224: 65 pgs: 65 active+clean; 3.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 112 KiB/s rd, 277 KiB/s wr, 3 op/s 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:Note: switching to '4e19f704afd9957ea657ab858f9ecf76401fef6f'. 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr: git switch -c 2024-03-18T14:50:15.937 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr:Or undo this operation with: 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr: git switch - 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr: 2024-03-18T14:50:15.938 INFO:tasks.workunit.client.1.smithi100.stderr:HEAD is now at 4e19f704afd ignorelist mon down 2024-03-18T14:50:15.942 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-18T14:50:16.005 INFO:tasks.workunit.client.1.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T14:50:16.007 INFO:tasks.workunit.client.1.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-18T14:50:16.007 INFO:tasks.workunit.client.1.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T14:50:16.359 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:16 smithi100 ceph-mon[37807]: pgmap v225: 65 pgs: 65 active+clean; 8.0 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 453 KiB/s rd, 643 KiB/s wr, 9 op/s 2024-03-18T14:50:16.424 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:16 smithi012 ceph-mon[29182]: pgmap v225: 65 pgs: 65 active+clean; 8.0 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 453 KiB/s rd, 643 KiB/s wr, 9 op/s 2024-03-18T14:50:16.971 INFO:tasks.workunit.client.1.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T14:50:17.025 INFO:tasks.workunit.client.1.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T14:50:17.065 INFO:tasks.workunit.client.1.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-18T14:50:17.066 INFO:tasks.workunit.client.1.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-18T14:50:17.066 INFO:tasks.workunit.client.1.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T14:50:17.105 INFO:tasks.workunit.client.1.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-18T14:50:17.108 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T14:50:17.108 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-18T14:50:17.163 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-18T14:50:17.165 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-18T14:50:17.165 DEBUG:teuthology.orchestra.run.smithi100:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4e19f704afd9957ea657ab858f9ecf76401fef6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-18T14:50:17.226 INFO:tasks.workunit.client.1.smithi100.stderr:+ mkdir -p fsstress 2024-03-18T14:50:17.229 INFO:tasks.workunit.client.1.smithi100.stderr:+ pushd fsstress 2024-03-18T14:50:17.229 INFO:tasks.workunit.client.1.smithi100.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-18T14:50:17.229 INFO:tasks.workunit.client.1.smithi100.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-18T14:50:18.393 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:18 smithi012 ceph-mon[29182]: pgmap v226: 65 pgs: 65 active+clean; 9.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 453 KiB/s rd, 810 KiB/s wr, 45 op/s 2024-03-18T14:50:18.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:18 smithi100 ceph-mon[37807]: pgmap v226: 65 pgs: 65 active+clean; 9.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 453 KiB/s rd, 810 KiB/s wr, 45 op/s 2024-03-18T14:50:19.815 INFO:tasks.workunit.client.1.smithi100.stderr:+ tar xzf ltp-full.tgz 2024-03-18T14:50:20.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:20 smithi012 ceph-mon[29182]: pgmap v227: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 336 op/s 2024-03-18T14:50:20.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:20 smithi100 ceph-mon[37807]: pgmap v227: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 336 op/s 2024-03-18T14:50:21.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:21 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:21.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:21 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:22.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:22 smithi012 ceph-mon[29182]: pgmap v228: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 390 op/s 2024-03-18T14:50:22.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:22 smithi100 ceph-mon[37807]: pgmap v228: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 390 op/s 2024-03-18T14:50:24.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:24 smithi012 ceph-mon[29182]: pgmap v229: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 401 op/s 2024-03-18T14:50:24.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:24 smithi100 ceph-mon[37807]: pgmap v229: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 401 op/s 2024-03-18T14:50:26.360 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:26 smithi100 ceph-mon[37807]: pgmap v230: 65 pgs: 65 active+clean; 69 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 434 op/s 2024-03-18T14:50:26.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:26 smithi012 ceph-mon[29182]: pgmap v230: 65 pgs: 65 active+clean; 69 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 434 op/s 2024-03-18T14:50:28.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:28 smithi012 ceph-mon[29182]: pgmap v231: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 475 op/s 2024-03-18T14:50:28.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:28 smithi100 ceph-mon[37807]: pgmap v231: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 475 op/s 2024-03-18T14:50:30.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:30 smithi012 ceph-mon[29182]: pgmap v232: 65 pgs: 65 active+clean; 106 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 799 op/s 2024-03-18T14:50:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:30 smithi100 ceph-mon[37807]: pgmap v232: 65 pgs: 65 active+clean; 106 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 799 op/s 2024-03-18T14:50:32.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:32 smithi012 ceph-mon[29182]: pgmap v233: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-03-18T14:50:32.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:32 smithi100 ceph-mon[37807]: pgmap v233: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-03-18T14:50:34.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:34 smithi100 ceph-mon[37807]: pgmap v234: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 528 op/s 2024-03-18T14:50:34.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:34 smithi012 ceph-mon[29182]: pgmap v234: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 528 op/s 2024-03-18T14:50:34.750 INFO:tasks.workunit.client.0.smithi012.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-18T14:50:34.750 INFO:tasks.workunit.client.0.smithi012.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-18T14:50:34.750 INFO:tasks.workunit.client.0.smithi012.stderr:+ make 2024-03-18T14:50:36.087 INFO:tasks.workunit.client.0.smithi012.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: Upgrade: Updating mgr.smithi100.rwnrcn 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: Deploying daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:50:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:35 smithi012 ceph-mon[29182]: pgmap v235: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 578 op/s 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: Upgrade: Updating mgr.smithi100.rwnrcn 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: Deploying daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:50:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:35 smithi100 ceph-mon[37807]: pgmap v235: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.9 MiB/s wr, 578 op/s 2024-03-18T14:50:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:36 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 -- 172.21.15.12:0/4004004882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fcc40 msgr2=0x7fe3280fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 --2- 172.21.15.12:0/4004004882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fcc40 0x7fe3280fd0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe318004940 tx=0x7fe318009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 -- 172.21.15.12:0/4004004882 shutdown_connections 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 --2- 172.21.15.12:0/4004004882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fcc40 0x7fe3280fd0b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 --2- 172.21.15.12:0/4004004882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 -- 172.21.15.12:0/4004004882 >> 172.21.15.12:0/4004004882 conn(0x7fe3280f6bc0 msgr2=0x7fe3280f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 -- 172.21.15.12:0/4004004882 shutdown_connections 2024-03-18T14:50:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 -- 172.21.15.12:0/4004004882 wait complete. 2024-03-18T14:50:37.208 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:36 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.205+0000 7fe32f951700 1 Processor -- start 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32f951700 1 -- start start 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32f951700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32f951700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 0x7fe328112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32f951700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe328112b90 con 0x7fe3280fc2f0 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32f951700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe328112cd0 con 0x7fe3280fcc40 2024-03-18T14:50:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32d6ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32ceec700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 0x7fe328112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32d6ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38590/0 (socket says 172.21.15.12:38590) 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32d6ed700 1 -- 172.21.15.12:0/1813358479 learned_addr learned my addr 172.21.15.12:0/1813358479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32ceec700 1 -- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 msgr2=0x7fe3280fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32ceec700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32ceec700 1 -- 172.21.15.12:0/1813358479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318005040 con 0x7fe3280fcc40 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.206+0000 7fe32d6ed700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe32ceec700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 0x7fe328112650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe318000c00 tx=0x7fe318009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31800e030 con 0x7fe3280fcc40 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe328112f50 con 0x7fe3280fcc40 2024-03-18T14:50:37.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe31801c400 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31800b650 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.207+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe328113460 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.208+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32804c710 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.209+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe31801ca90 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.209+0000 7fe31dffb700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 0x7fe31406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.209+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe318088cc0 con 0x7fe3280fcc40 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.209+0000 7fe32d6ed700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 0x7fe31406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.210+0000 7fe32d6ed700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 0x7fe31406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe32400f6a0 tx=0x7fe32400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.213+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe318058b20 con 0x7fe3280fcc40 2024-03-18T14:50:37.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.432+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe328113710 con 0x7fe3140692a0 2024-03-18T14:50:37.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.433+0000 7fe31dffb700 1 -- 172.21.15.12:0/1813358479 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe328113710 con 0x7fe3140692a0 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 msgr2=0x7fe31406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 0x7fe31406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe32400f6a0 tx=0x7fe32400b040 comp rx=0 tx=0).stop 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 msgr2=0x7fe328112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 0x7fe328112650 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe318000c00 tx=0x7fe318009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 shutdown_connections 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe3140692a0 0x7fe31406b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3280fc2f0 0x7fe3280fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 --2- 172.21.15.12:0/1813358479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3280fcc40 0x7fe328112650 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 >> 172.21.15.12:0/1813358479 conn(0x7fe3280f6bc0 msgr2=0x7fe3280f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 shutdown_connections 2024-03-18T14:50:37.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.435+0000 7fe32f951700 1 -- 172.21.15.12:0/1813358479 wait complete. 2024-03-18T14:50:37.446 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.548+0000 7fe25eb3b700 1 -- 172.21.15.12:0/1931875683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fcf80 msgr2=0x7fe2580fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.548+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/1931875683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fcf80 0x7fe2580fd3f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe244004940 tx=0x7fe244009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 -- 172.21.15.12:0/1931875683 shutdown_connections 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/1931875683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fcf80 0x7fe2580fd3f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/1931875683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fbcd0 0x7fe2580fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 -- 172.21.15.12:0/1931875683 >> 172.21.15.12:0/1931875683 conn(0x7fe2580f7430 msgr2=0x7fe2580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 -- 172.21.15.12:0/1931875683 shutdown_connections 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 -- 172.21.15.12:0/1931875683 wait complete. 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 Processor -- start 2024-03-18T14:50:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.549+0000 7fe25eb3b700 1 -- start start 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25eb3b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 0x7fe258110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25eb3b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 0x7fe258112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25eb3b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258066370 con 0x7fe2580fbcd0 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25eb3b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580664b0 con 0x7fe2580fcf80 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25c8d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 0x7fe258110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe257fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 0x7fe258112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25c8d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 0x7fe258110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38598/0 (socket says 172.21.15.12:38598) 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe25c8d7700 1 -- 172.21.15.12:0/2300160482 learned_addr learned my addr 172.21.15.12:0/2300160482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe257fff700 1 -- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 msgr2=0x7fe258110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe257fff700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 0x7fe258110680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe257fff700 1 -- 172.21.15.12:0/2300160482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244005040 con 0x7fe2580fcf80 2024-03-18T14:50:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.550+0000 7fe257fff700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 0x7fe258112bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe244000c00 tx=0x7fe244009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.551+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24400e030 con 0x7fe2580fcf80 2024-03-18T14:50:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.551+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe258113170 con 0x7fe2580fcf80 2024-03-18T14:50:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.551+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe244013de0 con 0x7fe2580fcf80 2024-03-18T14:50:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.551+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24401ce50 con 0x7fe2580fcf80 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.551+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258113680 con 0x7fe2580fcf80 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.552+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25805f5f0 con 0x7fe2580fcf80 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.552+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe24401c400 con 0x7fe2580fcf80 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.552+0000 7fe2557fa700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 0x7fe24806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.553+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe244088ac0 con 0x7fe2580fcf80 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.553+0000 7fe25c8d7700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 0x7fe24806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.553+0000 7fe25c8d7700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 0x7fe24806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe24c008fd0 tx=0x7fe24c00b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.561+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fe244054df0 con 0x7fe2580fcf80 2024-03-18T14:50:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.816+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2581012a0 con 0x7fe2480692a0 2024-03-18T14:50:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.817+0000 7fe2557fa700 1 -- 172.21.15.12:0/2300160482 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe2581012a0 con 0x7fe2480692a0 2024-03-18T14:50:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.819+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 msgr2=0x7fe24806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.819+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 0x7fe24806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe24c008fd0 tx=0x7fe24c00b040 comp rx=0 tx=0).stop 2024-03-18T14:50:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.819+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 msgr2=0x7fe258112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.819+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 0x7fe258112bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe244000c00 tx=0x7fe244009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 shutdown_connections 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fe2480692a0 0x7fe24806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2580fbcd0 0x7fe258110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 --2- 172.21.15.12:0/2300160482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2580fcf80 0x7fe258112bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 >> 172.21.15.12:0/2300160482 conn(0x7fe2580f7430 msgr2=0x7fe2580f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 shutdown_connections 2024-03-18T14:50:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.820+0000 7fe25eb3b700 1 -- 172.21.15.12:0/2300160482 wait complete. 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 -- 172.21.15.12:0/3847468761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fcc40 msgr2=0x7fd67c0fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 --2- 172.21.15.12:0/3847468761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fcc40 0x7fd67c0fd0b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd670004940 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 -- 172.21.15.12:0/3847468761 shutdown_connections 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 --2- 172.21.15.12:0/3847468761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fcc40 0x7fd67c0fd0b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 --2- 172.21.15.12:0/3847468761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fc2f0 0x7fd67c0fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 -- 172.21.15.12:0/3847468761 >> 172.21.15.12:0/3847468761 conn(0x7fd67c0f6be0 msgr2=0x7fd67c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 -- 172.21.15.12:0/3847468761 shutdown_connections 2024-03-18T14:50:37.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.957+0000 7fd680d55700 1 -- 172.21.15.12:0/3847468761 wait complete. 2024-03-18T14:50:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 Processor -- start 2024-03-18T14:50:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 -- start start 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fc2f0 0x7fd67c10fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c110650 con 0x7fd67c0fc2f0 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd680d55700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1107c0 con 0x7fd67c0fcc40 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd679d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd679d9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:41338/0 (socket says 172.21.15.12:41338) 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd679d9b700 1 -- 172.21.15.12:0/1028798752 learned_addr learned my addr 172.21.15.12:0/1028798752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.958+0000 7fd67a59c700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fc2f0 0x7fd67c10fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd679d9b700 1 -- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fc2f0 msgr2=0x7fd67c10fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd679d9b700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fc2f0 0x7fd67c10fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd679d9b700 1 -- 172.21.15.12:0/1028798752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670005040 con 0x7fd67c0fcc40 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd679d9b700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd67000b0d0 tx=0x7fd670009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67000e030 con 0x7fd67c0fcc40 2024-03-18T14:50:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd67001c400 con 0x7fd67c0fcc40 2024-03-18T14:50:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67000b630 con 0x7fd67c0fcc40 2024-03-18T14:50:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c115170 con 0x7fd67c0fcc40 2024-03-18T14:50:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.959+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c115670 con 0x7fd67c0fcc40 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.961+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c0052b0 con 0x7fd67c0fcc40 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.961+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd67000b8a0 con 0x7fd67c0fcc40 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.962+0000 7fd66affd700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 0x7fd66406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.962+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd670088a20 con 0x7fd67c0fcc40 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.962+0000 7fd67a59c700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 0x7fd66406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.963+0000 7fd67a59c700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 0x7fd66406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009b80 tx=0x7fd66c00b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:37.966+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fd670058880 con 0x7fd67c0fcc40 2024-03-18T14:50:38.036 INFO:tasks.workunit.client.0.smithi012.stderr:++ readlink -f fsstress 2024-03-18T14:50:38.036 INFO:tasks.workunit.client.0.smithi012.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-18T14:50:38.036 INFO:tasks.workunit.client.0.smithi012.stderr:+ popd 2024-03-18T14:50:38.037 INFO:tasks.workunit.client.0.smithi012.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-18T14:50:38.037 INFO:tasks.workunit.client.0.smithi012.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-18T14:50:38.037 INFO:tasks.workunit.client.0.smithi012.stderr:+ popd 2024-03-18T14:50:38.037 INFO:tasks.workunit.client.0.smithi012.stderr:++ mktemp -d -p . 2024-03-18T14:50:38.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ T=./tmp.lGbcUYkaOr 2024-03-18T14:50:38.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.lGbcUYkaOr -l 1 -n 1000 -p 10 -v 2024-03-18T14:50:38.042 INFO:tasks.workunit.client.0.smithi012.stdout:seed = 1710814184 2024-03-18T14:50:38.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:37 smithi100 ceph-mon[37807]: pgmap v236: 65 pgs: 65 active+clean; 127 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-03-18T14:50:38.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/0: dread - no filename 2024-03-18T14:50:38.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/1: fsync - no filename 2024-03-18T14:50:38.050 INFO:tasks.workunit.client.0.smithi012.stdout:1/0: dwrite - no filename 2024-03-18T14:50:38.050 INFO:tasks.workunit.client.0.smithi012.stdout:1/1: fsync - no filename 2024-03-18T14:50:38.050 INFO:tasks.workunit.client.0.smithi012.stdout:1/2: rename - no filename 2024-03-18T14:50:38.052 INFO:tasks.workunit.client.0.smithi012.stdout:0/2: mknod c0 0 2024-03-18T14:50:38.052 INFO:tasks.workunit.client.0.smithi012.stdout:0/3: chown c0 2645557 1 2024-03-18T14:50:38.054 INFO:tasks.workunit.client.0.smithi012.stdout:1/3: symlink l0 0 2024-03-18T14:50:38.056 INFO:tasks.workunit.client.0.smithi012.stdout:0/4: mkdir d1 0 2024-03-18T14:50:38.058 INFO:tasks.workunit.client.0.smithi012.stdout:3/0: dwrite - no filename 2024-03-18T14:50:38.058 INFO:tasks.workunit.client.0.smithi012.stdout:3/1: rmdir - no directory 2024-03-18T14:50:38.058 INFO:tasks.workunit.client.0.smithi012.stdout:3/2: rename - no filename 2024-03-18T14:50:38.058 INFO:tasks.workunit.client.0.smithi012.stdout:3/3: dwrite - no filename 2024-03-18T14:50:38.059 INFO:tasks.workunit.client.0.smithi012.stdout:2/0: mkdir d0 0 2024-03-18T14:50:38.059 INFO:tasks.workunit.client.0.smithi012.stdout:2/1: dread - no filename 2024-03-18T14:50:38.061 INFO:tasks.workunit.client.0.smithi012.stdout:1/4: mknod c1 0 2024-03-18T14:50:38.062 INFO:tasks.workunit.client.0.smithi012.stdout:0/5: symlink d1/l2 0 2024-03-18T14:50:38.064 INFO:tasks.workunit.client.0.smithi012.stdout:4/0: fsync - no filename 2024-03-18T14:50:38.065 INFO:tasks.workunit.client.0.smithi012.stdout:3/4: creat f0 x:0 0 0 2024-03-18T14:50:38.066 INFO:tasks.workunit.client.0.smithi012.stdout:2/2: symlink d0/l1 0 2024-03-18T14:50:38.068 INFO:tasks.workunit.client.0.smithi012.stdout:1/5: mknod c2 0 2024-03-18T14:50:38.070 INFO:tasks.workunit.client.0.smithi012.stdout:0/6: creat d1/f3 x:0 0 0 2024-03-18T14:50:38.070 INFO:tasks.workunit.client.0.smithi012.stdout:0/7: read - d1/f3 zero size 2024-03-18T14:50:38.070 INFO:tasks.workunit.client.0.smithi012.stdout:0/8: dread - d1/f3 zero size 2024-03-18T14:50:38.070 INFO:tasks.workunit.client.0.smithi012.stdout:0/9: truncate d1/f3 500039 0 2024-03-18T14:50:38.071 INFO:tasks.workunit.client.0.smithi012.stdout:0/10: write d1/f3 [241244,38970] 0 2024-03-18T14:50:38.071 INFO:tasks.workunit.client.0.smithi012.stdout:0/11: write d1/f3 [370337,11987] 0 2024-03-18T14:50:38.073 INFO:tasks.workunit.client.0.smithi012.stdout:4/1: mknod c0 0 2024-03-18T14:50:38.074 INFO:tasks.workunit.client.0.smithi012.stdout:4/2: write - no filename 2024-03-18T14:50:38.074 INFO:tasks.workunit.client.0.smithi012.stdout:4/3: chown c0 5092 1 2024-03-18T14:50:38.075 INFO:tasks.workunit.client.0.smithi012.stdout:3/5: rename f0 to f1 0 2024-03-18T14:50:38.076 INFO:tasks.workunit.client.0.smithi012.stdout:3/6: dread - f1 zero size 2024-03-18T14:50:38.076 INFO:tasks.workunit.client.0.smithi012.stdout:3/7: creat f2 x:0 0 0 2024-03-18T14:50:38.077 INFO:tasks.workunit.client.0.smithi012.stdout:2/3: unlink d0/l1 0 2024-03-18T14:50:38.079 INFO:tasks.workunit.client.0.smithi012.stdout:1/6: unlink l0 0 2024-03-18T14:50:38.081 INFO:tasks.workunit.client.0.smithi012.stdout:0/12: symlink d1/l4 0 2024-03-18T14:50:38.081 INFO:tasks.workunit.client.0.smithi012.stdout:0/13: creat d1/f5 x:0 0 0 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/0: dread - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/1: rmdir - no directory 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/2: read - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/3: chown . 226551 1 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/4: fsync - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/5: dread - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/6: readlink - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/7: rename - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/8: getdents . 0 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/9: write - no filename 2024-03-18T14:50:38.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/10: getdents . 0 2024-03-18T14:50:38.084 INFO:tasks.workunit.client.0.smithi012.stdout:6/11: dwrite - no filename 2024-03-18T14:50:38.084 INFO:tasks.workunit.client.0.smithi012.stdout:6/12: dwrite - no filename 2024-03-18T14:50:38.084 INFO:tasks.workunit.client.0.smithi012.stdout:6/13: dwrite - no filename 2024-03-18T14:50:38.084 INFO:tasks.workunit.client.0.smithi012.stdout:6/14: dread - no filename 2024-03-18T14:50:38.093 INFO:tasks.workunit.client.0.smithi012.stdout:1/7: creat f3 x:0 0 0 2024-03-18T14:50:38.094 INFO:tasks.workunit.client.0.smithi012.stdout:1/8: rmdir - no directory 2024-03-18T14:50:38.097 INFO:tasks.workunit.client.0.smithi012.stdout:0/14: mknod d1/c6 0 2024-03-18T14:50:38.097 INFO:tasks.workunit.client.0.smithi012.stdout:0/15: write d1/f5 [854136,86232] 0 2024-03-18T14:50:38.097 INFO:tasks.workunit.client.0.smithi012.stdout:0/16: read d1/f5 [881379,41977] 0 2024-03-18T14:50:38.097 INFO:tasks.workunit.client.0.smithi012.stdout:0/17: truncate d1/f5 1232254 0 2024-03-18T14:50:38.102 INFO:tasks.workunit.client.0.smithi012.stdout:6/15: symlink l0 0 2024-03-18T14:50:38.103 INFO:tasks.workunit.client.0.smithi012.stdout:3/8: link f2 f3 0 2024-03-18T14:50:38.104 INFO:tasks.workunit.client.0.smithi012.stdout:3/9: write f2 [623958,118525] 0 2024-03-18T14:50:38.106 INFO:tasks.workunit.client.0.smithi012.stdout:0/18: mknod d1/c7 0 2024-03-18T14:50:38.108 INFO:tasks.workunit.client.0.smithi012.stdout:8/0: truncate - no filename 2024-03-18T14:50:38.108 INFO:tasks.workunit.client.0.smithi012.stdout:8/1: write - no filename 2024-03-18T14:50:38.108 INFO:tasks.workunit.client.0.smithi012.stdout:8/2: dwrite - no filename 2024-03-18T14:50:38.108 INFO:tasks.workunit.client.0.smithi012.stdout:8/3: read - no filename 2024-03-18T14:50:38.108 INFO:tasks.workunit.client.0.smithi012.stdout:8/4: rename - no filename 2024-03-18T14:50:38.109 INFO:tasks.workunit.client.0.smithi012.stdout:1/9: dwrite f3 [0,4194304] 0 2024-03-18T14:50:38.109 INFO:tasks.workunit.client.0.smithi012.stdout:1/10: chown f3 3744 1 2024-03-18T14:50:38.110 INFO:tasks.workunit.client.0.smithi012.stdout:7/0: symlink l0 0 2024-03-18T14:50:38.110 INFO:tasks.workunit.client.0.smithi012.stdout:7/1: dwrite - no filename 2024-03-18T14:50:38.112 INFO:tasks.workunit.client.0.smithi012.stdout:6/16: creat f1 x:0 0 0 2024-03-18T14:50:38.115 INFO:tasks.workunit.client.0.smithi012.stdout:0/19: symlink d1/l8 0 2024-03-18T14:50:38.118 INFO:tasks.workunit.client.0.smithi012.stdout:8/5: mkdir d0 0 2024-03-18T14:50:38.118 INFO:tasks.workunit.client.0.smithi012.stdout:8/6: readlink - no filename 2024-03-18T14:50:38.118 INFO:tasks.workunit.client.0.smithi012.stdout:8/7: write - no filename 2024-03-18T14:50:38.118 INFO:tasks.workunit.client.0.smithi012.stdout:8/8: truncate - no filename 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/0: mknod c0 0 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/1: write - no filename 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/2: dwrite - no filename 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/3: dread - no filename 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/4: dwrite - no filename 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/5: chown c0 50853553 1 2024-03-18T14:50:38.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/6: dwrite - no filename 2024-03-18T14:50:38.127 INFO:tasks.workunit.client.0.smithi012.stdout:8/9: mknod d0/c1 0 2024-03-18T14:50:38.129 INFO:tasks.workunit.client.0.smithi012.stdout:9/7: creat f1 x:0 0 0 2024-03-18T14:50:38.131 INFO:tasks.workunit.client.0.smithi012.stdout:8/10: unlink d0/c1 0 2024-03-18T14:50:38.131 INFO:tasks.workunit.client.0.smithi012.stdout:8/11: write - no filename 2024-03-18T14:50:38.132 INFO:tasks.workunit.client.0.smithi012.stdout:9/8: symlink l2 0 2024-03-18T14:50:38.132 INFO:tasks.workunit.client.0.smithi012.stdout:9/9: dread - f1 zero size 2024-03-18T14:50:38.133 INFO:tasks.workunit.client.0.smithi012.stdout:9/10: rmdir - no directory 2024-03-18T14:50:38.133 INFO:tasks.workunit.client.0.smithi012.stdout:9/11: chown c0 0 1 2024-03-18T14:50:38.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/12: symlink d0/l2 0 2024-03-18T14:50:38.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/13: write - no filename 2024-03-18T14:50:38.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/14: write - no filename 2024-03-18T14:50:38.134 INFO:tasks.workunit.client.0.smithi012.stdout:8/15: dread - no filename 2024-03-18T14:50:38.135 INFO:tasks.workunit.client.0.smithi012.stdout:9/12: mkdir d3 0 2024-03-18T14:50:38.136 INFO:tasks.workunit.client.0.smithi012.stdout:8/16: creat d0/f3 x:0 0 0 2024-03-18T14:50:38.136 INFO:tasks.workunit.client.0.smithi012.stdout:8/17: dread - d0/f3 zero size 2024-03-18T14:50:38.138 INFO:tasks.workunit.client.0.smithi012.stdout:9/13: rename f1 to d3/f4 0 2024-03-18T14:50:38.138 INFO:tasks.workunit.client.0.smithi012.stdout:9/14: dread - d3/f4 zero size 2024-03-18T14:50:38.138 INFO:tasks.workunit.client.0.smithi012.stdout:9/15: dread - d3/f4 zero size 2024-03-18T14:50:38.139 INFO:tasks.workunit.client.0.smithi012.stdout:8/18: mknod d0/c4 0 2024-03-18T14:50:38.139 INFO:tasks.workunit.client.0.smithi012.stdout:8/19: creat d0/f5 x:0 0 0 2024-03-18T14:50:38.140 INFO:tasks.workunit.client.0.smithi012.stdout:8/20: creat d0/f6 x:0 0 0 2024-03-18T14:50:38.140 INFO:tasks.workunit.client.0.smithi012.stdout:8/21: write d0/f3 [625497,329] 0 2024-03-18T14:50:38.140 INFO:tasks.workunit.client.0.smithi012.stdout:8/22: chown d0/l2 34 1 2024-03-18T14:50:38.148 INFO:tasks.workunit.client.0.smithi012.stdout:8/23: rename d0/f3 to d0/f7 0 2024-03-18T14:50:38.148 INFO:tasks.workunit.client.0.smithi012.stdout:8/24: creat d0/f8 x:0 0 0 2024-03-18T14:50:38.148 INFO:tasks.workunit.client.0.smithi012.stdout:8/25: creat d0/f9 x:0 0 0 2024-03-18T14:50:38.148 INFO:tasks.workunit.client.0.smithi012.stdout:8/26: chown d0/f6 61091195 1 2024-03-18T14:50:38.156 INFO:tasks.workunit.client.0.smithi012.stdout:8/27: rename d0/f9 to d0/fa 0 2024-03-18T14:50:38.157 INFO:tasks.workunit.client.0.smithi012.stdout:8/28: read d0/f7 [413835,15528] 0 2024-03-18T14:50:38.157 INFO:tasks.workunit.client.0.smithi012.stdout:8/29: creat d0/fb x:0 0 0 2024-03-18T14:50:38.157 INFO:tasks.workunit.client.0.smithi012.stdout:8/30: creat d0/fc x:0 0 0 2024-03-18T14:50:38.157 INFO:tasks.workunit.client.0.smithi012.stdout:8/31: fdatasync d0/fc 0 2024-03-18T14:50:38.157 INFO:tasks.workunit.client.0.smithi012.stdout:8/32: readlink d0/l2 0 2024-03-18T14:50:38.160 INFO:tasks.workunit.client.0.smithi012.stdout:8/33: symlink d0/ld 0 2024-03-18T14:50:38.161 INFO:tasks.workunit.client.0.smithi012.stdout:8/34: creat d0/fe x:0 0 0 2024-03-18T14:50:38.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/35: dwrite d0/f7 [0,4194304] 0 2024-03-18T14:50:38.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/36: write d0/fe [1043058,30669] 0 2024-03-18T14:50:38.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/37: write d0/f6 [163088,27875] 0 2024-03-18T14:50:38.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/38: creat d0/ff x:0 0 0 2024-03-18T14:50:38.193 INFO:tasks.workunit.client.0.smithi012.stdout:8/39: dwrite d0/fa [0,4194304] 0 2024-03-18T14:50:38.193 INFO:tasks.workunit.client.0.smithi012.stdout:8/40: dread - d0/f5 zero size 2024-03-18T14:50:38.193 INFO:tasks.workunit.client.0.smithi012.stdout:8/41: creat d0/f10 x:0 0 0 2024-03-18T14:50:38.193 INFO:tasks.workunit.client.0.smithi012.stdout:8/42: write d0/fb [48271,35496] 0 2024-03-18T14:50:38.199 INFO:tasks.workunit.client.0.smithi012.stdout:8/43: link d0/f5 d0/f11 0 2024-03-18T14:50:38.199 INFO:tasks.workunit.client.0.smithi012.stdout:8/44: dread - d0/ff zero size 2024-03-18T14:50:38.200 INFO:tasks.workunit.client.0.smithi012.stdout:8/45: chown d0/f8 802 1 2024-03-18T14:50:38.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:37 smithi012 ceph-mon[29182]: pgmap v236: 65 pgs: 65 active+clean; 127 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-03-18T14:50:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.249+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd65c000bc0 con 0x7fd6640692a0 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.259+0000 7fd66affd700 1 -- 172.21.15.12:0/1028798752 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fd65c000bc0 con 0x7fd6640692a0 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (6m) 92s ago 7m 24.7M - 0.25.0 c8568f914cd2 3a6187ef0acb 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (7m) 92s ago 7m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (6m) 100s ago 6m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (5m) 92s ago 7m 80.3M - 9.4.7 954c08fa6188 a14a00878205 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (112s) 92s ago 112s 14.4M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (118s) 92s ago 117s 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (115s) 100s ago 114s 13.5M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:50:38.260 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (109s) 100s ago 109s 10.4M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (8m) 92s ago 8m 480M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (6m) 100s ago 6m 434M - 17.2.7-684-g6716c9f3 19857db3290c 47f9c2dcc3f5 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (9m) 92s ago 9m 44.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (6m) 100s ago 6m 35.9M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (7m) 92s ago 7m 14.4M - 1.5.0 0da6a335fe13 4ebf4cd84beb 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (6m) 100s ago 6m 17.3M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (5m) 92s ago 5m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (5m) 92s ago 5m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (4m) 92s ago 4m 46.7M 4289M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (4m) 100s ago 4m 47.1M 5312M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (3m) 100s ago 3m 40.8M 5312M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (3m) 100s ago 3m 41.0M 5312M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:50:38.261 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (5m) 92s ago 6m 53.0M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:50:38.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 msgr2=0x7fd66406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 0x7fd66406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009b80 tx=0x7fd66c00b040 comp rx=0 tx=0).stop 2024-03-18T14:50:38.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 msgr2=0x7fd67c110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd67000b0d0 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 shutdown_connections 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fd6640692a0 0x7fd66406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67c0fc2f0 0x7fd67c10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 --2- 172.21.15.12:0/1028798752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd67c0fcc40 0x7fd67c110050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 >> 172.21.15.12:0/1028798752 conn(0x7fd67c0f6be0 msgr2=0x7fd67c0f74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 shutdown_connections 2024-03-18T14:50:38.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.261+0000 7fd680d55700 1 -- 172.21.15.12:0/1028798752 wait complete. 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 -- 172.21.15.12:0/2845190171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 msgr2=0x7fbe100fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 --2- 172.21.15.12:0/2845190171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe100fc0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbe00004940 tx=0x7fbe00009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 -- 172.21.15.12:0/2845190171 shutdown_connections 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 --2- 172.21.15.12:0/2845190171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe100fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 --2- 172.21.15.12:0/2845190171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe100fc0f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 -- 172.21.15.12:0/2845190171 >> 172.21.15.12:0/2845190171 conn(0x7fbe100f7480 msgr2=0x7fbe100f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 -- 172.21.15.12:0/2845190171 shutdown_connections 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.369+0000 7fbe14f68700 1 -- 172.21.15.12:0/2845190171 wait complete. 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 Processor -- start 2024-03-18T14:50:38.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 -- start start 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe10112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe10066c00 con 0x7fbe100fbce0 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.370+0000 7fbe14f68700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe10066d40 con 0x7fbe100fcf90 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38632/0 (socket says 172.21.15.12:38632) 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 -- 172.21.15.12:0/3383058616 learned_addr learned my addr 172.21.15.12:0/3383058616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0dd9b700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe10112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 -- 172.21.15.12:0/3383058616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 msgr2=0x7fbe10112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe10112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 -- 172.21.15.12:0/3383058616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00005040 con 0x7fbe100fbce0 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0dd9b700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe10112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:50:38.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe0e59c700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe00013870 tx=0x7fbe00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe00021070 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe00009430 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe10113160 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0000e070 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.371+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe10113670 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.373+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1005f5f0 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.373+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbe0001c750 con 0x7fbe100fbce0 2024-03-18T14:50:38.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.373+0000 7fbdfeffd700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 0x7fbdf806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.374+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbe00088360 con 0x7fbe100fbce0 2024-03-18T14:50:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.374+0000 7fbe0dd9b700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 0x7fbdf806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.374+0000 7fbe0dd9b700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 0x7fbdf806b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009b80 tx=0x7fbe0400b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.378+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7fbe00054730 con 0x7fbe100fbce0 2024-03-18T14:50:38.674 INFO:tasks.workunit.client.0.smithi012.stdout:0/20: fsync d1/f3 0 2024-03-18T14:50:38.676 INFO:tasks.workunit.client.0.smithi012.stdout:0/21: dread d1/f3 [0,4194304] 0 2024-03-18T14:50:38.676 INFO:tasks.workunit.client.0.smithi012.stdout:0/22: creat d1/f9 x:0 0 0 2024-03-18T14:50:38.677 INFO:tasks.workunit.client.0.smithi012.stdout:0/23: creat d1/fa x:0 0 0 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/4: getdents . 0 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/5: write - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/6: dwrite - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/7: dread - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/8: write - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/9: truncate - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/10: write - no filename 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/11: chown c0 83 1 2024-03-18T14:50:38.678 INFO:tasks.workunit.client.0.smithi012.stdout:4/12: dread - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/13: dread - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/14: truncate - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/15: dwrite - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/16: write - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/17: dread - no filename 2024-03-18T14:50:38.679 INFO:tasks.workunit.client.0.smithi012.stdout:0/24: mkdir d1/db 0 2024-03-18T14:50:38.680 INFO:tasks.workunit.client.0.smithi012.stdout:3/10: link f1 f4 0 2024-03-18T14:50:38.681 INFO:tasks.workunit.client.0.smithi012.stdout:4/18: creat f1 x:0 0 0 2024-03-18T14:50:38.682 INFO:tasks.workunit.client.0.smithi012.stdout:4/19: dread - f1 zero size 2024-03-18T14:50:38.682 INFO:tasks.workunit.client.0.smithi012.stdout:0/25: mkdir d1/db/dc 0 2024-03-18T14:50:38.684 INFO:tasks.workunit.client.0.smithi012.stdout:2/4: getdents d0 0 2024-03-18T14:50:38.684 INFO:tasks.workunit.client.0.smithi012.stdout:2/5: rename d0 to d0/d2 22 2024-03-18T14:50:38.685 INFO:tasks.workunit.client.0.smithi012.stdout:3/11: symlink l5 0 2024-03-18T14:50:38.685 INFO:tasks.workunit.client.0.smithi012.stdout:3/12: creat f6 x:0 0 0 2024-03-18T14:50:38.685 INFO:tasks.workunit.client.0.smithi012.stdout:3/13: chown f4 0 1 2024-03-18T14:50:38.685 INFO:tasks.workunit.client.0.smithi012.stdout:3/14: write f4 [291995,78607] 0 2024-03-18T14:50:38.686 INFO:tasks.workunit.client.0.smithi012.stdout:4/20: creat f2 x:0 0 0 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.686+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe101139b0 con 0x7fbe100fbce0 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.686+0000 7fbdfeffd700 1 -- 172.21.15.12:0/3383058616 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fbe00004490 con 0x7fbe100fbce0 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:50:38.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 14 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:50:38.688 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:50:38.689 INFO:tasks.workunit.client.0.smithi012.stdout:2/6: symlink d0/l3 0 2024-03-18T14:50:38.689 INFO:tasks.workunit.client.0.smithi012.stdout:2/7: dread - no filename 2024-03-18T14:50:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 msgr2=0x7fbdf806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 0x7fbdf806b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009b80 tx=0x7fbe0400b040 comp rx=0 tx=0).stop 2024-03-18T14:50:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 msgr2=0x7fbe10110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe00013870 tx=0x7fbe00009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 shutdown_connections 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7fbdf80692a0 0x7fbdf806b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe100fbce0 0x7fbe10110670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 --2- 172.21.15.12:0/3383058616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe100fcf90 0x7fbe10112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 >> 172.21.15.12:0/3383058616 conn(0x7fbe100f7480 msgr2=0x7fbe100f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 shutdown_connections 2024-03-18T14:50:38.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.689+0000 7fbe14f68700 1 -- 172.21.15.12:0/3383058616 wait complete. 2024-03-18T14:50:38.694 INFO:tasks.workunit.client.0.smithi012.stdout:3/15: rename l5 to l7 0 2024-03-18T14:50:38.696 INFO:tasks.workunit.client.0.smithi012.stdout:4/21: creat f3 x:0 0 0 2024-03-18T14:50:38.696 INFO:tasks.workunit.client.0.smithi012.stdout:4/22: read - f2 zero size 2024-03-18T14:50:38.702 INFO:tasks.workunit.client.0.smithi012.stdout:0/26: dwrite d1/f3 [0,4194304] 0 2024-03-18T14:50:38.702 INFO:tasks.workunit.client.0.smithi012.stdout:0/27: creat d1/fd x:0 0 0 2024-03-18T14:50:38.702 INFO:tasks.workunit.client.0.smithi012.stdout:0/28: dread - d1/fa zero size 2024-03-18T14:50:38.704 INFO:tasks.workunit.client.0.smithi012.stdout:1/11: getdents . 0 2024-03-18T14:50:38.714 INFO:tasks.workunit.client.0.smithi012.stdout:3/16: dwrite f2 [0,4194304] 0 2024-03-18T14:50:38.714 INFO:tasks.workunit.client.0.smithi012.stdout:3/17: truncate f1 843059 0 2024-03-18T14:50:38.716 INFO:tasks.workunit.client.0.smithi012.stdout:0/29: creat d1/db/dc/fe x:0 0 0 2024-03-18T14:50:38.716 INFO:tasks.workunit.client.0.smithi012.stdout:0/30: chown d1/l2 3057242 1 2024-03-18T14:50:38.716 INFO:tasks.workunit.client.0.smithi012.stdout:0/31: dread - d1/f9 zero size 2024-03-18T14:50:38.719 INFO:tasks.workunit.client.0.smithi012.stdout:1/12: dread f3 [0,4194304] 0 2024-03-18T14:50:38.719 INFO:tasks.workunit.client.0.smithi012.stdout:4/23: mknod c4 0 2024-03-18T14:50:38.720 INFO:tasks.workunit.client.0.smithi012.stdout:4/24: chown f1 7448 1 2024-03-18T14:50:38.723 INFO:tasks.workunit.client.0.smithi012.stdout:0/32: link d1/fa d1/db/ff 0 2024-03-18T14:50:38.723 INFO:tasks.workunit.client.0.smithi012.stdout:0/33: creat d1/f10 x:0 0 0 2024-03-18T14:50:38.723 INFO:tasks.workunit.client.0.smithi012.stdout:1/13: dread f3 [0,4194304] 0 2024-03-18T14:50:38.729 INFO:tasks.workunit.client.0.smithi012.stdout:3/18: dwrite f3 [0,4194304] 0 2024-03-18T14:50:38.729 INFO:tasks.workunit.client.0.smithi012.stdout:3/19: write f4 [928767,15299] 0 2024-03-18T14:50:38.729 INFO:tasks.workunit.client.0.smithi012.stdout:0/34: symlink d1/db/l11 0 2024-03-18T14:50:38.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/14: creat f4 x:0 0 0 2024-03-18T14:50:38.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/15: dread - f4 zero size 2024-03-18T14:50:38.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/16: creat f5 x:0 0 0 2024-03-18T14:50:38.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/17: write f4 [209353,56004] 0 2024-03-18T14:50:38.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/18: rmdir - no directory 2024-03-18T14:50:38.733 INFO:tasks.workunit.client.0.smithi012.stdout:4/25: link f2 f5 0 2024-03-18T14:50:38.733 INFO:tasks.workunit.client.0.smithi012.stdout:4/26: rmdir - no directory 2024-03-18T14:50:38.736 INFO:tasks.workunit.client.0.smithi012.stdout:0/35: mknod d1/db/c12 0 2024-03-18T14:50:38.743 INFO:tasks.workunit.client.0.smithi012.stdout:1/19: mknod c6 0 2024-03-18T14:50:38.747 INFO:tasks.workunit.client.0.smithi012.stdout:4/27: mkdir d6 0 2024-03-18T14:50:38.747 INFO:tasks.workunit.client.0.smithi012.stdout:4/28: write f3 [867360,89018] 0 2024-03-18T14:50:38.753 INFO:tasks.workunit.client.0.smithi012.stdout:0/36: dwrite d1/fd [0,4194304] 0 2024-03-18T14:50:38.753 INFO:tasks.workunit.client.0.smithi012.stdout:0/37: truncate d1/f10 290518 0 2024-03-18T14:50:38.759 INFO:tasks.workunit.client.0.smithi012.stdout:1/20: dread f4 [0,4194304] 0 2024-03-18T14:50:38.762 INFO:tasks.workunit.client.0.smithi012.stdout:4/29: dread f3 [0,4194304] 0 2024-03-18T14:50:38.762 INFO:tasks.workunit.client.0.smithi012.stdout:4/30: chown f5 8330 1 2024-03-18T14:50:38.762 INFO:tasks.workunit.client.0.smithi012.stdout:4/31: write f1 [643760,13678] 0 2024-03-18T14:50:38.762 INFO:tasks.workunit.client.0.smithi012.stdout:1/21: mknod c7 0 2024-03-18T14:50:38.764 INFO:tasks.workunit.client.0.smithi012.stdout:1/22: mkdir d8 0 2024-03-18T14:50:38.766 INFO:tasks.workunit.client.0.smithi012.stdout:9/16: fsync d3/f4 0 2024-03-18T14:50:38.766 INFO:tasks.workunit.client.0.smithi012.stdout:9/17: fsync d3/f4 0 2024-03-18T14:50:38.766 INFO:tasks.workunit.client.0.smithi012.stdout:9/18: write d3/f4 [312748,121708] 0 2024-03-18T14:50:38.767 INFO:tasks.workunit.client.0.smithi012.stdout:4/32: dread f1 [0,4194304] 0 2024-03-18T14:50:38.771 INFO:tasks.workunit.client.0.smithi012.stdout:9/19: rename c0 to d3/c5 0 2024-03-18T14:50:38.771 INFO:tasks.workunit.client.0.smithi012.stdout:9/20: write d3/f4 [1451043,95109] 0 2024-03-18T14:50:38.774 INFO:tasks.workunit.client.0.smithi012.stdout:4/33: symlink d6/l7 0 2024-03-18T14:50:38.777 INFO:tasks.workunit.client.0.smithi012.stdout:4/34: creat d6/f8 x:0 0 0 2024-03-18T14:50:38.777 INFO:tasks.workunit.client.0.smithi012.stdout:4/35: creat d6/f9 x:0 0 0 2024-03-18T14:50:38.778 INFO:tasks.workunit.client.0.smithi012.stdout:4/36: write f1 [1480308,94928] 0 2024-03-18T14:50:38.778 INFO:tasks.workunit.client.0.smithi012.stdout:4/37: dread - f5 zero size 2024-03-18T14:50:38.778 INFO:tasks.workunit.client.0.smithi012.stdout:4/38: write f1 [1854773,41818] 0 2024-03-18T14:50:38.779 INFO:tasks.workunit.client.0.smithi012.stdout:1/23: dwrite f3 [0,4194304] 0 2024-03-18T14:50:38.782 INFO:tasks.workunit.client.0.smithi012.stdout:9/21: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:38.784 INFO:tasks.workunit.client.0.smithi012.stdout:4/39: mkdir d6/da 0 2024-03-18T14:50:38.784 INFO:tasks.workunit.client.0.smithi012.stdout:9/22: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:38.788 INFO:tasks.workunit.client.0.smithi012.stdout:4/40: unlink d6/f8 0 2024-03-18T14:50:38.790 INFO:tasks.workunit.client.0.smithi012.stdout:1/24: dread f3 [0,4194304] 0 2024-03-18T14:50:38.790 INFO:tasks.workunit.client.0.smithi012.stdout:1/25: write f4 [484618,78768] 0 2024-03-18T14:50:38.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/23: creat d3/f6 x:0 0 0 2024-03-18T14:50:38.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/24: chown d3/f4 323 1 2024-03-18T14:50:38.793 INFO:tasks.workunit.client.0.smithi012.stdout:4/41: creat d6/da/fb x:0 0 0 2024-03-18T14:50:38.795 INFO:tasks.workunit.client.0.smithi012.stdout:1/26: creat d8/f9 x:0 0 0 2024-03-18T14:50:38.799 INFO:tasks.workunit.client.0.smithi012.stdout:4/42: mknod d6/cc 0 2024-03-18T14:50:38.801 INFO:tasks.workunit.client.0.smithi012.stdout:1/27: rename d8/f9 to d8/fa 0 2024-03-18T14:50:38.801 INFO:tasks.workunit.client.0.smithi012.stdout:1/28: write f4 [50071,100784] 0 2024-03-18T14:50:38.803 INFO:tasks.workunit.client.0.smithi012.stdout:4/43: rename d6/da/fb to d6/da/fd 0 2024-03-18T14:50:38.807 INFO:tasks.workunit.client.0.smithi012.stdout:1/29: unlink c6 0 2024-03-18T14:50:38.811 INFO:tasks.workunit.client.0.smithi012.stdout:1/30: dread f3 [0,4194304] 0 2024-03-18T14:50:38.812 INFO:tasks.workunit.client.0.smithi012.stdout:1/31: fdatasync f3 0 2024-03-18T14:50:38.812 INFO:tasks.workunit.client.0.smithi012.stdout:1/32: fdatasync f4 0 2024-03-18T14:50:38.814 INFO:tasks.workunit.client.0.smithi012.stdout:1/33: mkdir d8/db 0 2024-03-18T14:50:38.814 INFO:tasks.workunit.client.0.smithi012.stdout:1/34: truncate f4 1175824 0 2024-03-18T14:50:38.814 INFO:tasks.workunit.client.0.smithi012.stdout:1/35: creat d8/fc x:0 0 0 2024-03-18T14:50:38.833 INFO:tasks.workunit.client.0.smithi012.stdout:8/46: rmdir d0 39 2024-03-18T14:50:38.833 INFO:tasks.workunit.client.0.smithi012.stdout:8/47: dread - d0/f8 zero size 2024-03-18T14:50:38.834 INFO:tasks.workunit.client.0.smithi012.stdout:8/48: chown d0/l2 8058583 1 2024-03-18T14:50:38.834 INFO:tasks.workunit.client.0.smithi012.stdout:8/49: chown d0/l2 27 1 2024-03-18T14:50:38.834 INFO:tasks.workunit.client.0.smithi012.stdout:8/50: dread - d0/f5 zero size 2024-03-18T14:50:38.834 INFO:tasks.workunit.client.0.smithi012.stdout:8/51: fdatasync d0/fa 0 2024-03-18T14:50:38.835 INFO:tasks.workunit.client.0.smithi012.stdout:8/52: mkdir d0/d12 0 2024-03-18T14:50:38.835 INFO:tasks.workunit.client.0.smithi012.stdout:8/53: readlink d0/l2 0 2024-03-18T14:50:38.837 INFO:tasks.workunit.client.0.smithi012.stdout:8/54: symlink d0/l13 0 2024-03-18T14:50:38.837 INFO:tasks.workunit.client.0.smithi012.stdout:8/55: truncate d0/f8 765743 0 2024-03-18T14:50:38.837 INFO:tasks.workunit.client.0.smithi012.stdout:8/56: chown d0 153850449 1 2024-03-18T14:50:38.841 INFO:tasks.workunit.client.0.smithi012.stdout:8/57: unlink d0/f8 0 2024-03-18T14:50:38.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.842+0000 7f3c8959a700 1 -- 172.21.15.12:0/1707665687 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c00bcb0 con 0x7f3c8c0fe040 2024-03-18T14:50:38.843 INFO:tasks.workunit.client.0.smithi012.stdout:8/58: read d0/fe [37881,124430] 0 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.842+0000 7f3c90f85700 1 -- 172.21.15.12:0/1707665687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 msgr2=0x7f3c8c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.842+0000 7f3c90f85700 1 --2- 172.21.15.12:0/1707665687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c0fe4b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c004940 tx=0x7f3c7c009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 -- 172.21.15.12:0/1707665687 shutdown_connections 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 --2- 172.21.15.12:0/1707665687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c0fe4b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 --2- 172.21.15.12:0/1707665687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 0x7f3c8c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 -- 172.21.15.12:0/1707665687 >> 172.21.15.12:0/1707665687 conn(0x7f3c8c0f8570 msgr2=0x7f3c8c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 -- 172.21.15.12:0/1707665687 shutdown_connections 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 -- 172.21.15.12:0/1707665687 wait complete. 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 Processor -- start 2024-03-18T14:50:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 -- start start 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 0x7f3c8c10fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.843+0000 7f3c90f85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c90f85700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c110740 con 0x7f3c8c0fe040 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c90f85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c110f40 con 0x7f3c8c0fce40 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38280/0 (socket says 172.21.15.12:38280) 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 -- 172.21.15.12:0/2776606019 learned_addr learned my addr 172.21.15.12:0/2776606019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:50:38.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c8a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 0x7f3c8c10fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 -- 172.21.15.12:0/2776606019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 msgr2=0x7f3c8c10fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 0x7f3c8c10fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 -- 172.21.15.12:0/2776606019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c005040 con 0x7f3c8c0fe040 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c89d9b700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009150 tx=0x7f3c7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c00e040 con 0x7f3c8c0fe040 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c1111c0 con 0x7f3c8c0fe040 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c7c004290 con 0x7f3c8c0fe040 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.844+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c01c630 con 0x7f3c8c0fe040 2024-03-18T14:50:38.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.845+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c111700 con 0x7f3c8c0fe040 2024-03-18T14:50:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.846+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c05f5f0 con 0x7f3c8c0fe040 2024-03-18T14:50:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.846+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c7c00bbf0 con 0x7f3c8c0fe040 2024-03-18T14:50:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.847+0000 7f3c7affd700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 0x7f3c7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:50:38.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.847+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3c7c087d70 con 0x7f3c8c0fe040 2024-03-18T14:50:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.847+0000 7f3c8a59c700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 0x7f3c7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:50:38.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.848+0000 7f3c8a59c700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 0x7f3c7406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0fdbb0 tx=0x7f3c8000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:50:38.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:38.855+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175520 (secure 0 0 0) 0x7f3c7c057bd0 con 0x7f3c8c0fe040 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3383058616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:38 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:39.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3383058616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:50:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:38 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:39.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.157+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 --> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c8c111a80 con 0x7f3c740692a0 2024-03-18T14:50:39.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.158+0000 7f3c7affd700 1 -- 172.21.15.12:0/2776606019 <== mgr.14220 v2:172.21.15.12:6800/965904248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f3c8c111a80 con 0x7f3c740692a0 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "", 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mgr daemons", 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:50:39.159 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:50:39.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 msgr2=0x7f3c7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 0x7f3c7406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0fdbb0 tx=0x7f3c8000b040 comp rx=0 tx=0).stop 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 msgr2=0x7f3c8c110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009150 tx=0x7f3c7c009e30 comp rx=0 tx=0).stop 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 shutdown_connections 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:6800/965904248,v1:172.21.15.12:6801/965904248] conn(0x7f3c740692a0 0x7f3c7406b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c0fce40 0x7f3c8c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 --2- 172.21.15.12:0/2776606019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8c0fe040 0x7f3c8c110140 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.160+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 >> 172.21.15.12:0/2776606019 conn(0x7f3c8c0f8570 msgr2=0x7f3c8c0f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.161+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 shutdown_connections 2024-03-18T14:50:39.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:50:39.161+0000 7f3c90f85700 1 -- 172.21.15.12:0/2776606019 wait complete. 2024-03-18T14:50:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:3/20: fsync f1 0 2024-03-18T14:50:39.272 INFO:tasks.workunit.client.0.smithi012.stdout:0/38: getdents d1/db/dc 0 2024-03-18T14:50:39.273 INFO:tasks.workunit.client.0.smithi012.stdout:0/39: creat d1/db/f13 x:0 0 0 2024-03-18T14:50:39.274 INFO:tasks.workunit.client.0.smithi012.stdout:0/40: truncate d1/f9 508949 0 2024-03-18T14:50:39.276 INFO:tasks.workunit.client.0.smithi012.stdout:0/41: fsync d1/f10 0 2024-03-18T14:50:39.276 INFO:tasks.workunit.client.0.smithi012.stdout:0/42: chown d1/fd 3 1 2024-03-18T14:50:39.277 INFO:tasks.workunit.client.0.smithi012.stdout:0/43: mknod d1/c14 0 2024-03-18T14:50:39.277 INFO:tasks.workunit.client.0.smithi012.stdout:0/44: dread - d1/db/dc/fe zero size 2024-03-18T14:50:39.278 INFO:tasks.workunit.client.0.smithi012.stdout:0/45: mknod d1/c15 0 2024-03-18T14:50:39.280 INFO:tasks.workunit.client.0.smithi012.stdout:0/46: rename d1/db/dc to d1/d16 0 2024-03-18T14:50:39.280 INFO:tasks.workunit.client.0.smithi012.stdout:0/47: chown d1/db/ff 6 1 2024-03-18T14:50:39.280 INFO:tasks.workunit.client.0.smithi012.stdout:1/36: fsync f4 0 2024-03-18T14:50:39.280 INFO:tasks.workunit.client.0.smithi012.stdout:1/37: chown d8/fa 173 1 2024-03-18T14:50:39.280 INFO:tasks.workunit.client.0.smithi012.stdout:1/38: truncate d8/fa 700118 0 2024-03-18T14:50:39.281 INFO:tasks.workunit.client.0.smithi012.stdout:1/39: creat d8/db/fd x:0 0 0 2024-03-18T14:50:39.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/40: symlink d8/db/le 0 2024-03-18T14:50:39.304 INFO:tasks.workunit.client.0.smithi012.stdout:9/25: rmdir d3 39 2024-03-18T14:50:39.304 INFO:tasks.workunit.client.0.smithi012.stdout:9/26: dread - d3/f6 zero size 2024-03-18T14:50:39.304 INFO:tasks.workunit.client.0.smithi012.stdout:9/27: dread - d3/f6 zero size 2024-03-18T14:50:39.304 INFO:tasks.workunit.client.0.smithi012.stdout:0/48: dwrite d1/d16/fe [0,4194304] 0 2024-03-18T14:50:39.305 INFO:tasks.workunit.client.0.smithi012.stdout:9/28: getdents d3 0 2024-03-18T14:50:39.305 INFO:tasks.workunit.client.0.smithi012.stdout:9/29: fdatasync d3/f4 0 2024-03-18T14:50:39.306 INFO:tasks.workunit.client.0.smithi012.stdout:1/41: dwrite d8/fa [0,4194304] 0 2024-03-18T14:50:39.308 INFO:tasks.workunit.client.0.smithi012.stdout:0/49: creat d1/d16/f17 x:0 0 0 2024-03-18T14:50:39.309 INFO:tasks.workunit.client.0.smithi012.stdout:9/30: mknod d3/c7 0 2024-03-18T14:50:39.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/42: link d8/fc d8/db/ff 0 2024-03-18T14:50:39.314 INFO:tasks.workunit.client.0.smithi012.stdout:0/50: unlink d1/f5 0 2024-03-18T14:50:39.316 INFO:tasks.workunit.client.0.smithi012.stdout:9/31: creat d3/f8 x:0 0 0 2024-03-18T14:50:39.316 INFO:tasks.workunit.client.0.smithi012.stdout:9/32: read - d3/f8 zero size 2024-03-18T14:50:39.316 INFO:tasks.workunit.client.0.smithi012.stdout:9/33: readlink l2 0 2024-03-18T14:50:39.316 INFO:tasks.workunit.client.0.smithi012.stdout:9/34: write d3/f8 [935826,119834] 0 2024-03-18T14:50:39.317 INFO:tasks.workunit.client.0.smithi012.stdout:9/35: dread - d3/f6 zero size 2024-03-18T14:50:39.321 INFO:tasks.workunit.client.0.smithi012.stdout:9/36: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:39.321 INFO:tasks.workunit.client.0.smithi012.stdout:9/37: write d3/f4 [2093216,121396] 0 2024-03-18T14:50:39.323 INFO:tasks.workunit.client.0.smithi012.stdout:9/38: mkdir d3/d9 0 2024-03-18T14:50:39.325 INFO:tasks.workunit.client.0.smithi012.stdout:9/39: rename d3/c7 to d3/ca 0 2024-03-18T14:50:39.326 INFO:tasks.workunit.client.0.smithi012.stdout:9/40: creat d3/d9/fb x:0 0 0 2024-03-18T14:50:39.326 INFO:tasks.workunit.client.0.smithi012.stdout:9/41: truncate d3/d9/fb 565516 0 2024-03-18T14:50:39.330 INFO:tasks.workunit.client.0.smithi012.stdout:0/51: dwrite d1/f3 [0,4194304] 0 2024-03-18T14:50:39.347 INFO:tasks.workunit.client.0.smithi012.stdout:9/42: dwrite d3/f8 [0,4194304] 0 2024-03-18T14:50:39.348 INFO:tasks.workunit.client.0.smithi012.stdout:9/43: creat d3/d9/fc x:0 0 0 2024-03-18T14:50:39.348 INFO:tasks.workunit.client.0.smithi012.stdout:9/44: write d3/d9/fc [953149,49787] 0 2024-03-18T14:50:39.444 INFO:tasks.workunit.client.0.smithi012.stdout:0/52: rename d1/db to d1/d16/d18 0 2024-03-18T14:50:39.444 INFO:tasks.workunit.client.0.smithi012.stdout:0/53: write d1/f10 [1214118,89751] 0 2024-03-18T14:50:39.448 INFO:tasks.workunit.client.0.smithi012.stdout:3/21: dwrite f3 [4194304,4194304] 0 2024-03-18T14:50:39.449 INFO:tasks.workunit.client.0.smithi012.stdout:1/43: rmdir d8/db 39 2024-03-18T14:50:39.451 INFO:tasks.workunit.client.0.smithi012.stdout:3/22: mkdir d8 0 2024-03-18T14:50:39.453 INFO:tasks.workunit.client.0.smithi012.stdout:1/44: mkdir d8/db/d10 0 2024-03-18T14:50:39.454 INFO:tasks.workunit.client.0.smithi012.stdout:3/23: dread f4 [0,4194304] 0 2024-03-18T14:50:39.455 INFO:tasks.workunit.client.0.smithi012.stdout:9/45: getdents d3 0 2024-03-18T14:50:39.458 INFO:tasks.workunit.client.0.smithi012.stdout:1/45: link d8/fa d8/db/d10/f11 0 2024-03-18T14:50:39.458 INFO:tasks.workunit.client.0.smithi012.stdout:1/46: chown d8/db/fd 34563833 1 2024-03-18T14:50:39.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/24: rmdir d8 0 2024-03-18T14:50:39.463 INFO:tasks.workunit.client.0.smithi012.stdout:1/47: link c2 d8/db/c12 0 2024-03-18T14:50:39.476 INFO:tasks.workunit.client.0.smithi012.stdout:9/46: dwrite d3/d9/fb [0,4194304] 0 2024-03-18T14:50:39.483 INFO:tasks.workunit.client.0.smithi012.stdout:1/48: dwrite d8/fa [0,4194304] 0 2024-03-18T14:50:39.490 INFO:tasks.workunit.client.0.smithi012.stdout:0/54: truncate d1/f10 20553 0 2024-03-18T14:50:39.493 INFO:tasks.workunit.client.0.smithi012.stdout:0/55: mkdir d1/d16/d19 0 2024-03-18T14:50:39.493 INFO:tasks.workunit.client.0.smithi012.stdout:9/47: dwrite d3/f8 [0,4194304] 0 2024-03-18T14:50:39.493 INFO:tasks.workunit.client.0.smithi012.stdout:1/49: dread d8/db/d10/f11 [0,4194304] 0 2024-03-18T14:50:39.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/50: dread d8/db/d10/f11 [0,4194304] 0 2024-03-18T14:50:39.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/51: fsync f5 0 2024-03-18T14:50:39.500 INFO:tasks.workunit.client.0.smithi012.stdout:1/52: creat d8/db/d10/f13 x:0 0 0 2024-03-18T14:50:39.501 INFO:tasks.workunit.client.0.smithi012.stdout:3/25: getdents . 0 2024-03-18T14:50:39.501 INFO:tasks.workunit.client.0.smithi012.stdout:3/26: fsync f1 0 2024-03-18T14:50:39.505 INFO:tasks.workunit.client.0.smithi012.stdout:1/53: mknod d8/db/c14 0 2024-03-18T14:50:39.505 INFO:tasks.workunit.client.0.smithi012.stdout:1/54: stat f4 0 2024-03-18T14:50:39.505 INFO:tasks.workunit.client.0.smithi012.stdout:1/55: readlink d8/db/le 0 2024-03-18T14:50:39.505 INFO:tasks.workunit.client.0.smithi012.stdout:1/56: chown c2 206 1 2024-03-18T14:50:39.508 INFO:tasks.workunit.client.0.smithi012.stdout:3/27: dread f2 [4194304,4194304] 0 2024-03-18T14:50:39.516 INFO:tasks.workunit.client.0.smithi012.stdout:3/28: mknod c9 0 2024-03-18T14:50:39.517 INFO:tasks.workunit.client.0.smithi012.stdout:3/29: read f4 [748954,80628] 0 2024-03-18T14:50:39.517 INFO:tasks.workunit.client.0.smithi012.stdout:9/48: write d3/d9/fb [5237062,58607] 0 2024-03-18T14:50:39.522 INFO:tasks.workunit.client.0.smithi012.stdout:9/49: rename d3/d9/fb to d3/d9/fd 0 2024-03-18T14:50:39.525 INFO:tasks.workunit.client.0.smithi012.stdout:9/50: write d3/d9/fc [353147,110070] 0 2024-03-18T14:50:39.527 INFO:tasks.workunit.client.0.smithi012.stdout:9/51: link l2 d3/le 0 2024-03-18T14:50:39.527 INFO:tasks.workunit.client.0.smithi012.stdout:9/52: rename d3 to d3/d9/df 22 2024-03-18T14:50:39.527 INFO:tasks.workunit.client.0.smithi012.stdout:0/56: truncate d1/f10 360693 0 2024-03-18T14:50:39.527 INFO:tasks.workunit.client.0.smithi012.stdout:0/57: chown d1/d16/d18/c12 32661 1 2024-03-18T14:50:39.532 INFO:tasks.workunit.client.0.smithi012.stdout:0/58: mknod d1/d16/d18/c1a 0 2024-03-18T14:50:39.534 INFO:tasks.workunit.client.0.smithi012.stdout:3/30: dwrite f4 [0,4194304] 0 2024-03-18T14:50:39.535 INFO:tasks.workunit.client.0.smithi012.stdout:0/59: truncate d1/d16/fe 852357 0 2024-03-18T14:50:39.544 INFO:tasks.workunit.client.0.smithi012.stdout:0/60: unlink d1/d16/d18/c1a 0 2024-03-18T14:50:39.544 INFO:tasks.workunit.client.0.smithi012.stdout:0/61: chown d1/d16 2 1 2024-03-18T14:50:39.547 INFO:tasks.workunit.client.0.smithi012.stdout:0/62: creat d1/d16/d19/f1b x:0 0 0 2024-03-18T14:50:39.547 INFO:tasks.workunit.client.0.smithi012.stdout:0/63: dread - d1/d16/d18/ff zero size 2024-03-18T14:50:39.548 INFO:tasks.workunit.client.0.smithi012.stdout:3/31: dwrite f3 [0,4194304] 0 2024-03-18T14:50:39.549 INFO:tasks.workunit.client.0.smithi012.stdout:0/64: mkdir d1/d1c 0 2024-03-18T14:50:39.551 INFO:tasks.workunit.client.0.smithi012.stdout:3/32: mknod ca 0 2024-03-18T14:50:39.552 INFO:tasks.workunit.client.0.smithi012.stdout:3/33: fsync f4 0 2024-03-18T14:50:39.554 INFO:tasks.workunit.client.0.smithi012.stdout:3/34: creat fb x:0 0 0 2024-03-18T14:50:39.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/53: truncate d3/d9/fd 3457552 0 2024-03-18T14:50:39.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/54: dread - d3/f6 zero size 2024-03-18T14:50:39.558 INFO:tasks.workunit.client.0.smithi012.stdout:3/35: mkdir dc 0 2024-03-18T14:50:39.558 INFO:tasks.workunit.client.0.smithi012.stdout:3/36: dread - fb zero size 2024-03-18T14:50:39.563 INFO:tasks.workunit.client.0.smithi012.stdout:3/37: rename c9 to dc/cd 0 2024-03-18T14:50:39.563 INFO:tasks.workunit.client.0.smithi012.stdout:3/38: write f2 [8754100,115445] 0 2024-03-18T14:50:39.571 INFO:tasks.workunit.client.0.smithi012.stdout:3/39: write f4 [1665392,34001] 0 2024-03-18T14:50:39.574 INFO:tasks.workunit.client.0.smithi012.stdout:3/40: symlink dc/le 0 2024-03-18T14:50:39.578 INFO:tasks.workunit.client.0.smithi012.stdout:3/41: creat dc/ff x:0 0 0 2024-03-18T14:50:39.578 INFO:tasks.workunit.client.0.smithi012.stdout:3/42: fdatasync fb 0 2024-03-18T14:50:39.578 INFO:tasks.workunit.client.0.smithi012.stdout:3/43: dread - f6 zero size 2024-03-18T14:50:39.579 INFO:tasks.workunit.client.0.smithi012.stdout:3/44: mknod dc/c10 0 2024-03-18T14:50:39.582 INFO:tasks.workunit.client.0.smithi012.stdout:3/45: rename f1 to dc/f11 0 2024-03-18T14:50:39.582 INFO:tasks.workunit.client.0.smithi012.stdout:3/46: creat dc/f12 x:0 0 0 2024-03-18T14:50:39.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/55: truncate d3/d9/fd 4402308 0 2024-03-18T14:50:39.591 INFO:tasks.workunit.client.0.smithi012.stdout:9/56: write d3/f4 [518847,98982] 0 2024-03-18T14:50:39.591 INFO:tasks.workunit.client.0.smithi012.stdout:9/57: creat d3/d9/f10 x:0 0 0 2024-03-18T14:50:39.614 INFO:tasks.workunit.client.0.smithi012.stdout:3/47: getdents dc 0 2024-03-18T14:50:39.614 INFO:tasks.workunit.client.0.smithi012.stdout:3/48: write dc/ff [565868,75474] 0 2024-03-18T14:50:39.616 INFO:tasks.workunit.client.0.smithi012.stdout:3/49: mknod dc/c13 0 2024-03-18T14:50:39.621 INFO:tasks.workunit.client.0.smithi012.stdout:3/50: rename f4 to dc/f14 0 2024-03-18T14:50:39.621 INFO:tasks.workunit.client.0.smithi012.stdout:3/51: chown fb 140546 1 2024-03-18T14:50:39.624 INFO:tasks.workunit.client.0.smithi012.stdout:3/52: rmdir dc 39 2024-03-18T14:50:39.624 INFO:tasks.workunit.client.0.smithi012.stdout:3/53: write fb [978173,84004] 0 2024-03-18T14:50:39.626 INFO:tasks.workunit.client.0.smithi012.stdout:9/58: rmdir d3/d9 39 2024-03-18T14:50:39.629 INFO:tasks.workunit.client.0.smithi012.stdout:9/59: mknod d3/c11 0 2024-03-18T14:50:39.630 INFO:tasks.workunit.client.0.smithi012.stdout:3/54: dread dc/f14 [0,4194304] 0 2024-03-18T14:50:39.630 INFO:tasks.workunit.client.0.smithi012.stdout:3/55: truncate f6 810743 0 2024-03-18T14:50:39.631 INFO:tasks.workunit.client.0.smithi012.stdout:3/56: write f6 [229232,16410] 0 2024-03-18T14:50:39.632 INFO:tasks.workunit.client.0.smithi012.stdout:9/60: rename d3/d9/fc to d3/f12 0 2024-03-18T14:50:39.635 INFO:tasks.workunit.client.0.smithi012.stdout:3/57: mknod dc/c15 0 2024-03-18T14:50:39.638 INFO:tasks.workunit.client.0.smithi012.stdout:9/61: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:39.639 INFO:tasks.workunit.client.0.smithi012.stdout:3/58: symlink dc/l16 0 2024-03-18T14:50:39.645 INFO:tasks.workunit.client.0.smithi012.stdout:9/62: dread d3/d9/fd [0,4194304] 0 2024-03-18T14:50:39.648 INFO:tasks.workunit.client.0.smithi012.stdout:9/63: creat d3/d9/f13 x:0 0 0 2024-03-18T14:50:39.651 INFO:tasks.workunit.client.0.smithi012.stdout:9/64: symlink d3/l14 0 2024-03-18T14:50:39.656 INFO:tasks.workunit.client.0.smithi012.stdout:3/59: dwrite dc/f12 [0,4194304] 0 2024-03-18T14:50:39.656 INFO:tasks.workunit.client.0.smithi012.stdout:3/60: stat f3 0 2024-03-18T14:50:39.663 INFO:tasks.workunit.client.0.smithi012.stdout:3/61: link dc/cd dc/c17 0 2024-03-18T14:50:39.667 INFO:tasks.workunit.client.0.smithi012.stdout:3/62: symlink dc/l18 0 2024-03-18T14:50:39.679 INFO:tasks.workunit.client.0.smithi012.stdout:9/65: truncate d3/f4 1841652 0 2024-03-18T14:50:39.681 INFO:tasks.workunit.client.0.smithi012.stdout:9/66: mknod d3/d9/c15 0 2024-03-18T14:50:39.700 INFO:tasks.workunit.client.0.smithi012.stdout:3/63: dwrite dc/f12 [4194304,4194304] 0 2024-03-18T14:50:39.706 INFO:tasks.workunit.client.0.smithi012.stdout:3/64: rename dc/c15 to dc/c19 0 2024-03-18T14:50:39.711 INFO:tasks.workunit.client.0.smithi012.stdout:9/67: write d3/f4 [1789790,123644] 0 2024-03-18T14:50:39.733 INFO:tasks.workunit.client.0.smithi012.stdout:3/65: dwrite dc/f12 [4194304,4194304] 0 2024-03-18T14:50:39.734 INFO:tasks.workunit.client.0.smithi012.stdout:9/68: dwrite d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:39.739 INFO:tasks.workunit.client.0.smithi012.stdout:9/69: rmdir d3 39 2024-03-18T14:50:39.739 INFO:tasks.workunit.client.0.smithi012.stdout:9/70: write d3/f4 [499273,44647] 0 2024-03-18T14:50:39.742 INFO:tasks.workunit.client.0.smithi012.stdout:3/66: mknod dc/c1a 0 2024-03-18T14:50:39.742 INFO:tasks.workunit.client.0.smithi012.stdout:3/67: readlink l7 0 2024-03-18T14:50:39.745 INFO:tasks.workunit.client.0.smithi012.stdout:3/68: link dc/c19 dc/c1b 0 2024-03-18T14:50:39.746 INFO:tasks.workunit.client.0.smithi012.stdout:9/71: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:39.747 INFO:tasks.workunit.client.0.smithi012.stdout:9/72: write d3/f6 [804686,13067] 0 2024-03-18T14:50:39.747 INFO:tasks.workunit.client.0.smithi012.stdout:9/73: write d3/f6 [573478,48876] 0 2024-03-18T14:50:39.749 INFO:tasks.workunit.client.0.smithi012.stdout:3/69: symlink dc/l1c 0 2024-03-18T14:50:39.752 INFO:tasks.workunit.client.0.smithi012.stdout:9/74: creat d3/f16 x:0 0 0 2024-03-18T14:50:39.752 INFO:tasks.workunit.client.0.smithi012.stdout:9/75: chown d3/f16 3777 1 2024-03-18T14:50:39.755 INFO:tasks.workunit.client.0.smithi012.stdout:3/70: dread fb [0,4194304] 0 2024-03-18T14:50:39.769 INFO:tasks.workunit.client.0.smithi012.stdout:9/76: dwrite d3/f16 [0,4194304] 0 2024-03-18T14:50:39.773 INFO:tasks.workunit.client.0.smithi012.stdout:9/77: rename d3/f8 to d3/d9/f17 0 2024-03-18T14:50:39.776 INFO:tasks.workunit.client.0.smithi012.stdout:3/71: dwrite f6 [0,4194304] 0 2024-03-18T14:50:39.781 INFO:tasks.workunit.client.0.smithi012.stdout:9/78: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:39.781 INFO:tasks.workunit.client.0.smithi012.stdout:3/72: fdatasync f2 0 2024-03-18T14:50:39.783 INFO:tasks.workunit.client.0.smithi012.stdout:9/79: unlink d3/c11 0 2024-03-18T14:50:39.783 INFO:tasks.workunit.client.0.smithi012.stdout:9/80: chown d3/d9 0 1 2024-03-18T14:50:39.787 INFO:tasks.workunit.client.0.smithi012.stdout:3/73: dread f2 [0,4194304] 0 2024-03-18T14:50:39.791 INFO:tasks.workunit.client.0.smithi012.stdout:3/74: unlink dc/c1b 0 2024-03-18T14:50:39.808 INFO:tasks.workunit.client.0.smithi012.stdout:9/81: getdents d3/d9 0 2024-03-18T14:50:39.811 INFO:tasks.workunit.client.0.smithi012.stdout:9/82: write d3/d9/f13 [2059765,126033] 0 2024-03-18T14:50:39.814 INFO:tasks.workunit.client.0.smithi012.stdout:9/83: mkdir d3/d18 0 2024-03-18T14:50:39.820 INFO:tasks.workunit.client.0.smithi012.stdout:9/84: rename d3/d18 to d3/d9/d19 0 2024-03-18T14:50:39.820 INFO:tasks.workunit.client.0.smithi012.stdout:9/85: write d3/d9/f10 [301047,119417] 0 2024-03-18T14:50:39.820 INFO:tasks.workunit.client.0.smithi012.stdout:9/86: rename d3/d9 to d3/d9/d19/d1a 22 2024-03-18T14:50:39.820 INFO:tasks.workunit.client.0.smithi012.stdout:9/87: write d3/f6 [162551,82418] 0 2024-03-18T14:50:39.827 INFO:tasks.workunit.client.0.smithi012.stdout:9/88: write d3/d9/f17 [92314,40505] 0 2024-03-18T14:50:39.831 INFO:tasks.workunit.client.0.smithi012.stdout:3/75: truncate dc/ff 510805 0 2024-03-18T14:50:39.835 INFO:tasks.workunit.client.0.smithi012.stdout:9/89: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:39.835 INFO:tasks.workunit.client.0.smithi012.stdout:9/90: fdatasync d3/f16 0 2024-03-18T14:50:39.839 INFO:tasks.workunit.client.0.smithi012.stdout:9/91: write d3/d9/fd [3196226,79737] 0 2024-03-18T14:50:39.843 INFO:tasks.workunit.client.0.smithi012.stdout:9/92: symlink d3/l1b 0 2024-03-18T14:50:39.852 INFO:tasks.workunit.client.0.smithi012.stdout:9/93: getdents d3/d9 0 2024-03-18T14:50:39.855 INFO:tasks.workunit.client.0.smithi012.stdout:9/94: creat d3/d9/f1c x:0 0 0 2024-03-18T14:50:39.858 INFO:tasks.workunit.client.0.smithi012.stdout:9/95: symlink d3/d9/d19/l1d 0 2024-03-18T14:50:39.872 INFO:tasks.workunit.client.0.smithi012.stdout:9/96: dwrite d3/f16 [0,4194304] 0 2024-03-18T14:50:39.892 INFO:tasks.workunit.client.0.smithi012.stdout:9/97: dwrite d3/d9/f10 [0,4194304] 0 2024-03-18T14:50:39.914 INFO:tasks.workunit.client.0.smithi012.stdout:9/98: dwrite d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:39.916 INFO:tasks.workunit.client.0.smithi012.stdout:9/99: mkdir d3/d1e 0 2024-03-18T14:50:39.921 INFO:tasks.workunit.client.0.smithi012.stdout:9/100: rmdir d3/d9 39 2024-03-18T14:50:39.923 INFO:tasks.workunit.client.0.smithi012.stdout:9/101: mknod d3/c1f 0 2024-03-18T14:50:39.925 INFO:tasks.workunit.client.0.smithi012.stdout:9/102: truncate d3/f6 456753 0 2024-03-18T14:50:39.925 INFO:tasks.workunit.client.0.smithi012.stdout:9/103: chown d3/d1e 49090604 1 2024-03-18T14:50:39.925 INFO:tasks.workunit.client.0.smithi012.stdout:9/104: chown d3/c1f 2850 1 2024-03-18T14:50:39.928 INFO:tasks.workunit.client.0.smithi012.stdout:9/105: mkdir d3/d20 0 2024-03-18T14:50:39.930 INFO:tasks.workunit.client.0.smithi012.stdout:9/106: mkdir d3/d1e/d21 0 2024-03-18T14:50:39.932 INFO:tasks.workunit.client.0.smithi012.stdout:9/107: unlink d3/c5 0 2024-03-18T14:50:39.934 INFO:tasks.workunit.client.0.smithi012.stdout:9/108: read d3/d9/f13 [3068687,105112] 0 2024-03-18T14:50:39.936 INFO:tasks.workunit.client.0.smithi012.stdout:9/109: mkdir d3/d22 0 2024-03-18T14:50:39.938 INFO:tasks.workunit.client.0.smithi012.stdout:9/110: rmdir d3/d1e 39 2024-03-18T14:50:39.941 INFO:tasks.workunit.client.0.smithi012.stdout:9/111: mknod d3/d9/c23 0 2024-03-18T14:50:39.941 INFO:tasks.workunit.client.0.smithi012.stdout:9/112: chown d3/d1e/d21 27 1 2024-03-18T14:50:39.944 INFO:tasks.workunit.client.0.smithi012.stdout:9/113: mknod d3/c24 0 2024-03-18T14:50:39.967 INFO:tasks.workunit.client.0.smithi012.stdout:9/114: dwrite d3/d9/fd [0,4194304] 0 2024-03-18T14:50:39.977 INFO:tasks.workunit.client.0.smithi012.stdout:9/115: dread d3/d9/f10 [0,4194304] 0 2024-03-18T14:50:39.981 INFO:tasks.workunit.client.0.smithi012.stdout:9/116: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:39.981 INFO:tasks.workunit.client.0.smithi012.stdout:9/117: chown d3/d9/f1c 15830 1 2024-03-18T14:50:39.986 INFO:tasks.workunit.client.0.smithi012.stdout:9/118: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:40.001 INFO:tasks.workunit.client.0.smithi012.stdout:9/119: dwrite d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:40.004 INFO:tasks.workunit.client.0.smithi012.stdout:9/120: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:40.008 INFO:tasks.workunit.client.0.smithi012.stdout:9/121: getdents d3 0 2024-03-18T14:50:40.010 INFO:tasks.workunit.client.0.smithi012.stdout:9/122: creat d3/f25 x:0 0 0 2024-03-18T14:50:40.014 INFO:tasks.workunit.client.0.smithi012.stdout:9/123: creat d3/d1e/d21/f26 x:0 0 0 2024-03-18T14:50:40.017 INFO:tasks.workunit.client.0.smithi012.stdout:9/124: rmdir d3/d1e 39 2024-03-18T14:50:40.017 INFO:tasks.workunit.client.0.smithi012.stdout:9/125: write d3/f25 [344688,113477] 0 2024-03-18T14:50:40.025 INFO:tasks.workunit.client.0.smithi012.stdout:9/126: dread d3/d9/f10 [0,4194304] 0 2024-03-18T14:50:40.025 INFO:tasks.workunit.client.0.smithi012.stdout:9/127: getdents d3/d9/d19 0 2024-03-18T14:50:40.034 INFO:tasks.workunit.client.0.smithi012.stdout:9/128: dread d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:40.035 INFO:tasks.workunit.client.0.smithi012.stdout:9/129: chown d3/f25 31 1 2024-03-18T14:50:40.039 INFO:tasks.workunit.client.0.smithi012.stdout:9/130: rename d3/d9/c23 to d3/d1e/d21/c27 0 2024-03-18T14:50:40.039 INFO:tasks.workunit.client.0.smithi012.stdout:9/131: chown d3/d1e/d21 506741 1 2024-03-18T14:50:40.039 INFO:tasks.workunit.client.0.smithi012.stdout:9/132: fdatasync d3/d9/f10 0 2024-03-18T14:50:40.043 INFO:tasks.workunit.client.0.smithi012.stdout:9/133: unlink l2 0 2024-03-18T14:50:40.092 INFO:tasks.workunit.client.0.smithi012.stdout:9/134: rmdir d3/d1e/d21 39 2024-03-18T14:50:40.175 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:39 smithi100 ceph-mon[37807]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:40.175 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:39 smithi100 ceph-mon[37807]: pgmap v237: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 8.0 MiB/s wr, 936 op/s 2024-03-18T14:50:40.175 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:39 smithi100 ceph-mon[37807]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:40.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:39 smithi012 ceph-mon[29182]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:39 smithi012 ceph-mon[29182]: pgmap v237: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 8.0 MiB/s wr, 936 op/s 2024-03-18T14:50:40.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:39 smithi012 ceph-mon[29182]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:50:42.212 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:41 smithi100 ceph-mon[37807]: pgmap v238: 65 pgs: 65 active+clean; 180 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 599 op/s 2024-03-18T14:50:42.212 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:41 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:42.212 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:41 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:42.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:41 smithi012 ceph-mon[29182]: pgmap v238: 65 pgs: 65 active+clean; 180 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 599 op/s 2024-03-18T14:50:42.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:41 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:41 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:0/65: sync 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:4/44: sync 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:7/2: sync 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:6/17: sync 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:7/3: dread - no filename 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:7/4: write - no filename 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:3/76: sync 2024-03-18T14:50:43.172 INFO:tasks.workunit.client.0.smithi012.stdout:6/18: stat l0 0 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:1/57: sync 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:5/0: sync 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:0/66: fdatasync d1/f3 0 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:5/1: dread - no filename 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:9/135: sync 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:0/67: chown d1/l4 12 1 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:2/8: sync 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:8/59: sync 2024-03-18T14:50:43.173 INFO:tasks.workunit.client.0.smithi012.stdout:2/9: stat d0 0 2024-03-18T14:50:43.174 INFO:tasks.workunit.client.0.smithi012.stdout:7/5: creat f1 x:0 0 0 2024-03-18T14:50:43.175 INFO:tasks.workunit.client.0.smithi012.stdout:6/19: rename l0 to l2 0 2024-03-18T14:50:43.175 INFO:tasks.workunit.client.0.smithi012.stdout:6/20: chown l2 556 1 2024-03-18T14:50:43.175 INFO:tasks.workunit.client.0.smithi012.stdout:6/21: write f1 [656359,13491] 0 2024-03-18T14:50:43.175 INFO:tasks.workunit.client.0.smithi012.stdout:1/58: dread f4 [0,4194304] 0 2024-03-18T14:50:43.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/2: symlink l0 0 2024-03-18T14:50:43.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/3: chown l0 218598 1 2024-03-18T14:50:43.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/4: dwrite - no filename 2024-03-18T14:50:43.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/5: dwrite - no filename 2024-03-18T14:50:43.178 INFO:tasks.workunit.client.0.smithi012.stdout:0/68: creat d1/d1c/f1d x:0 0 0 2024-03-18T14:50:43.180 INFO:tasks.workunit.client.0.smithi012.stdout:8/60: unlink d0/ld 0 2024-03-18T14:50:43.183 INFO:tasks.workunit.client.0.smithi012.stdout:4/45: rmdir d6 39 2024-03-18T14:50:43.183 INFO:tasks.workunit.client.0.smithi012.stdout:4/46: dread - f2 zero size 2024-03-18T14:50:43.183 INFO:tasks.workunit.client.0.smithi012.stdout:4/47: read - d6/da/fd zero size 2024-03-18T14:50:43.185 INFO:tasks.workunit.client.0.smithi012.stdout:7/6: unlink l0 0 2024-03-18T14:50:43.187 INFO:tasks.workunit.client.0.smithi012.stdout:6/22: symlink l3 0 2024-03-18T14:50:43.187 INFO:tasks.workunit.client.0.smithi012.stdout:6/23: write f1 [1023706,107231] 0 2024-03-18T14:50:43.188 INFO:tasks.workunit.client.0.smithi012.stdout:6/24: read f1 [421802,41600] 0 2024-03-18T14:50:43.188 INFO:tasks.workunit.client.0.smithi012.stdout:6/25: chown l2 182 1 2024-03-18T14:50:43.188 INFO:tasks.workunit.client.0.smithi012.stdout:6/26: creat f4 x:0 0 0 2024-03-18T14:50:43.188 INFO:tasks.workunit.client.0.smithi012.stdout:6/27: write f1 [1187113,12312] 0 2024-03-18T14:50:43.189 INFO:tasks.workunit.client.0.smithi012.stdout:1/59: creat d8/db/f15 x:0 0 0 2024-03-18T14:50:43.190 INFO:tasks.workunit.client.0.smithi012.stdout:9/136: rmdir d3/d9/d19 39 2024-03-18T14:50:43.192 INFO:tasks.workunit.client.0.smithi012.stdout:2/10: mknod d0/c4 0 2024-03-18T14:50:43.192 INFO:tasks.workunit.client.0.smithi012.stdout:2/11: stat d0/c4 0 2024-03-18T14:50:43.193 INFO:tasks.workunit.client.0.smithi012.stdout:0/69: dwrite d1/d1c/f1d [0,4194304] 0 2024-03-18T14:50:43.193 INFO:tasks.workunit.client.0.smithi012.stdout:0/70: read - d1/d16/f17 zero size 2024-03-18T14:50:43.194 INFO:tasks.workunit.client.0.smithi012.stdout:8/61: link d0/fe d0/f14 0 2024-03-18T14:50:43.195 INFO:tasks.workunit.client.0.smithi012.stdout:4/48: rmdir d6 39 2024-03-18T14:50:43.195 INFO:tasks.workunit.client.0.smithi012.stdout:9/137: write d3/d9/f10 [2448099,115163] 0 2024-03-18T14:50:43.195 INFO:tasks.workunit.client.0.smithi012.stdout:9/138: stat d3/d9/f1c 0 2024-03-18T14:50:43.197 INFO:tasks.workunit.client.0.smithi012.stdout:7/7: symlink l2 0 2024-03-18T14:50:43.197 INFO:tasks.workunit.client.0.smithi012.stdout:7/8: creat f3 x:0 0 0 2024-03-18T14:50:43.198 INFO:tasks.workunit.client.0.smithi012.stdout:1/60: mkdir d8/db/d10/d16 0 2024-03-18T14:50:43.199 INFO:tasks.workunit.client.0.smithi012.stdout:2/12: creat d0/f5 x:0 0 0 2024-03-18T14:50:43.203 INFO:tasks.workunit.client.0.smithi012.stdout:8/62: truncate d0/f7 708261 0 2024-03-18T14:50:43.206 INFO:tasks.workunit.client.0.smithi012.stdout:8/63: dread d0/f6 [0,4194304] 0 2024-03-18T14:50:43.207 INFO:tasks.workunit.client.0.smithi012.stdout:8/64: write d0/f11 [664741,105749] 0 2024-03-18T14:50:43.207 INFO:tasks.workunit.client.0.smithi012.stdout:9/139: creat d3/d9/f28 x:0 0 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:7/9: rename l2 to l4 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:7/10: creat f5 x:0 0 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:7/11: write f5 [931476,23798] 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:7/12: creat f6 x:0 0 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:7/13: creat f7 x:0 0 0 2024-03-18T14:50:43.210 INFO:tasks.workunit.client.0.smithi012.stdout:1/61: symlink d8/db/d10/d16/l17 0 2024-03-18T14:50:43.211 INFO:tasks.workunit.client.0.smithi012.stdout:2/13: creat d0/f6 x:0 0 0 2024-03-18T14:50:43.213 INFO:tasks.workunit.client.0.smithi012.stdout:8/65: mkdir d0/d15 0 2024-03-18T14:50:43.213 INFO:tasks.workunit.client.0.smithi012.stdout:8/66: write d0/f5 [970344,21559] 0 2024-03-18T14:50:43.213 INFO:tasks.workunit.client.0.smithi012.stdout:8/67: dread - d0/ff zero size 2024-03-18T14:50:43.213 INFO:tasks.workunit.client.0.smithi012.stdout:8/68: chown d0/f6 7482 1 2024-03-18T14:50:43.215 INFO:tasks.workunit.client.0.smithi012.stdout:0/71: dwrite d1/fa [0,4194304] 0 2024-03-18T14:50:43.215 INFO:tasks.workunit.client.0.smithi012.stdout:0/72: stat d1/d16/f17 0 2024-03-18T14:50:43.216 INFO:tasks.workunit.client.0.smithi012.stdout:7/14: mknod c8 0 2024-03-18T14:50:43.217 INFO:tasks.workunit.client.0.smithi012.stdout:1/62: symlink d8/db/d10/l18 0 2024-03-18T14:50:43.217 INFO:tasks.workunit.client.0.smithi012.stdout:1/63: write f4 [1711954,26511] 0 2024-03-18T14:50:43.223 INFO:tasks.workunit.client.0.smithi012.stdout:7/15: rename c8 to c9 0 2024-03-18T14:50:43.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/14: mknod d0/c7 0 2024-03-18T14:50:43.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/15: stat d0/c4 0 2024-03-18T14:50:43.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/16: chown d0/c7 57584 1 2024-03-18T14:50:43.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/17: write d0/f5 [905211,68153] 0 2024-03-18T14:50:43.226 INFO:tasks.workunit.client.0.smithi012.stdout:7/16: rename f6 to fa 0 2024-03-18T14:50:43.236 INFO:tasks.workunit.client.0.smithi012.stdout:1/64: dwrite f4 [0,4194304] 0 2024-03-18T14:50:43.254 INFO:tasks.workunit.client.0.smithi012.stdout:1/65: dwrite d8/db/f15 [0,4194304] 0 2024-03-18T14:50:43.258 INFO:tasks.workunit.client.0.smithi012.stdout:1/66: rename d8/db/d10/d16/l17 to d8/db/l19 0 2024-03-18T14:50:43.278 INFO:tasks.workunit.client.0.smithi012.stdout:1/67: dwrite d8/db/ff [0,4194304] 0 2024-03-18T14:50:43.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/68: dread d8/fa [0,4194304] 0 2024-03-18T14:50:43.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/69: dread - f5 zero size 2024-03-18T14:50:43.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/70: write d8/db/d10/f11 [5030732,2342] 0 2024-03-18T14:50:43.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/71: chown c1 1639 1 2024-03-18T14:50:43.289 INFO:tasks.workunit.client.0.smithi012.stdout:5/6: getdents . 0 2024-03-18T14:50:43.290 INFO:tasks.workunit.client.0.smithi012.stdout:1/72: read d8/db/ff [859029,90720] 0 2024-03-18T14:50:43.291 INFO:tasks.workunit.client.0.smithi012.stdout:5/7: mknod c1 0 2024-03-18T14:50:43.295 INFO:tasks.workunit.client.0.smithi012.stdout:5/8: creat f2 x:0 0 0 2024-03-18T14:50:43.299 INFO:tasks.workunit.client.0.smithi012.stdout:5/9: mkdir d3 0 2024-03-18T14:50:43.299 INFO:tasks.workunit.client.0.smithi012.stdout:5/10: write f2 [949926,118040] 0 2024-03-18T14:50:43.300 INFO:tasks.workunit.client.0.smithi012.stdout:5/11: chown f2 2681 1 2024-03-18T14:50:43.304 INFO:tasks.workunit.client.0.smithi012.stdout:5/12: rename l0 to d3/l4 0 2024-03-18T14:50:43.309 INFO:tasks.workunit.client.0.smithi012.stdout:5/13: creat d3/f5 x:0 0 0 2024-03-18T14:50:43.317 INFO:tasks.workunit.client.0.smithi012.stdout:8/69: dwrite d0/f7 [0,4194304] 0 2024-03-18T14:50:43.317 INFO:tasks.workunit.client.0.smithi012.stdout:8/70: fsync d0/fe 0 2024-03-18T14:50:43.319 INFO:tasks.workunit.client.0.smithi012.stdout:0/73: truncate d1/fa 229487 0 2024-03-18T14:50:43.320 INFO:tasks.workunit.client.0.smithi012.stdout:5/14: dread f2 [0,4194304] 0 2024-03-18T14:50:43.321 INFO:tasks.workunit.client.0.smithi012.stdout:2/18: rmdir d0 39 2024-03-18T14:50:43.321 INFO:tasks.workunit.client.0.smithi012.stdout:2/19: truncate d0/f6 951872 0 2024-03-18T14:50:43.323 INFO:tasks.workunit.client.0.smithi012.stdout:0/74: mknod d1/d16/d19/c1e 0 2024-03-18T14:50:43.323 INFO:tasks.workunit.client.0.smithi012.stdout:0/75: readlink d1/l8 0 2024-03-18T14:50:43.324 INFO:tasks.workunit.client.0.smithi012.stdout:0/76: fdatasync d1/d1c/f1d 0 2024-03-18T14:50:43.324 INFO:tasks.workunit.client.0.smithi012.stdout:0/77: truncate d1/d16/f17 795894 0 2024-03-18T14:50:43.325 INFO:tasks.workunit.client.0.smithi012.stdout:2/20: creat d0/f8 x:0 0 0 2024-03-18T14:50:43.326 INFO:tasks.workunit.client.0.smithi012.stdout:8/71: dread d0/fa [0,4194304] 0 2024-03-18T14:50:43.326 INFO:tasks.workunit.client.0.smithi012.stdout:0/78: symlink d1/d16/d18/l1f 0 2024-03-18T14:50:43.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/73: rmdir d8/db/d10 39 2024-03-18T14:50:43.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/74: fdatasync d8/db/d10/f13 0 2024-03-18T14:50:43.330 INFO:tasks.workunit.client.0.smithi012.stdout:8/72: creat d0/f16 x:0 0 0 2024-03-18T14:50:43.332 INFO:tasks.workunit.client.0.smithi012.stdout:2/21: mknod d0/c9 0 2024-03-18T14:50:43.336 INFO:tasks.workunit.client.0.smithi012.stdout:8/73: mknod d0/d15/c17 0 2024-03-18T14:50:43.336 INFO:tasks.workunit.client.0.smithi012.stdout:2/22: creat d0/fa x:0 0 0 2024-03-18T14:50:43.336 INFO:tasks.workunit.client.0.smithi012.stdout:2/23: write d0/fa [502173,110402] 0 2024-03-18T14:50:43.338 INFO:tasks.workunit.client.0.smithi012.stdout:8/74: mkdir d0/d18 0 2024-03-18T14:50:43.339 INFO:tasks.workunit.client.0.smithi012.stdout:5/15: dwrite d3/f5 [0,4194304] 0 2024-03-18T14:50:43.343 INFO:tasks.workunit.client.0.smithi012.stdout:1/75: write f4 [2515007,61953] 0 2024-03-18T14:50:43.343 INFO:tasks.workunit.client.0.smithi012.stdout:1/76: creat d8/f1a x:0 0 0 2024-03-18T14:50:43.345 INFO:tasks.workunit.client.0.smithi012.stdout:8/75: write d0/f7 [2779668,5506] 0 2024-03-18T14:50:43.347 INFO:tasks.workunit.client.0.smithi012.stdout:5/16: link d3/l4 d3/l6 0 2024-03-18T14:50:43.348 INFO:tasks.workunit.client.0.smithi012.stdout:2/24: mknod d0/cb 0 2024-03-18T14:50:43.348 INFO:tasks.workunit.client.0.smithi012.stdout:2/25: stat d0/f5 0 2024-03-18T14:50:43.348 INFO:tasks.workunit.client.0.smithi012.stdout:2/26: write d0/f8 [805352,119591] 0 2024-03-18T14:50:43.349 INFO:tasks.workunit.client.0.smithi012.stdout:2/27: write d0/f5 [153053,126018] 0 2024-03-18T14:50:43.355 INFO:tasks.workunit.client.0.smithi012.stdout:8/76: mkdir d0/d18/d19 0 2024-03-18T14:50:43.356 INFO:tasks.workunit.client.0.smithi012.stdout:5/17: rename c1 to d3/c7 0 2024-03-18T14:50:43.359 INFO:tasks.workunit.client.0.smithi012.stdout:8/77: unlink d0/c4 0 2024-03-18T14:50:43.361 INFO:tasks.workunit.client.0.smithi012.stdout:2/28: symlink d0/lc 0 2024-03-18T14:50:43.361 INFO:tasks.workunit.client.0.smithi012.stdout:2/29: chown d0/lc 1102776 1 2024-03-18T14:50:43.364 INFO:tasks.workunit.client.0.smithi012.stdout:8/78: creat d0/d15/f1a x:0 0 0 2024-03-18T14:50:43.365 INFO:tasks.workunit.client.0.smithi012.stdout:5/18: write d3/f5 [2015209,104348] 0 2024-03-18T14:50:43.366 INFO:tasks.workunit.client.0.smithi012.stdout:8/79: read d0/f5 [534749,124376] 0 2024-03-18T14:50:43.372 INFO:tasks.workunit.client.0.smithi012.stdout:8/80: dread d0/fa [0,4194304] 0 2024-03-18T14:50:43.372 INFO:tasks.workunit.client.0.smithi012.stdout:8/81: readlink d0/l2 0 2024-03-18T14:50:43.372 INFO:tasks.workunit.client.0.smithi012.stdout:8/82: read d0/f14 [553279,96978] 0 2024-03-18T14:50:43.373 INFO:tasks.workunit.client.0.smithi012.stdout:5/19: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:43.375 INFO:tasks.workunit.client.0.smithi012.stdout:8/83: mkdir d0/d18/d19/d1b 0 2024-03-18T14:50:43.375 INFO:tasks.workunit.client.0.smithi012.stdout:5/20: write f2 [40533,69647] 0 2024-03-18T14:50:43.375 INFO:tasks.workunit.client.0.smithi012.stdout:5/21: write f2 [452779,102661] 0 2024-03-18T14:50:43.380 INFO:tasks.workunit.client.0.smithi012.stdout:2/30: dwrite d0/f8 [0,4194304] 0 2024-03-18T14:50:43.381 INFO:tasks.workunit.client.0.smithi012.stdout:2/31: dread d0/f6 [0,4194304] 0 2024-03-18T14:50:43.409 INFO:tasks.workunit.client.0.smithi012.stdout:5/22: truncate f2 216251 0 2024-03-18T14:50:43.418 INFO:tasks.workunit.client.0.smithi012.stdout:2/32: truncate d0/f8 3248876 0 2024-03-18T14:50:43.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/33: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:43.441 INFO:tasks.workunit.client.0.smithi012.stdout:5/23: fsync f2 0 2024-03-18T14:50:43.442 INFO:tasks.workunit.client.0.smithi012.stdout:5/24: chown d3/l6 6813541 1 2024-03-18T14:50:43.443 INFO:tasks.workunit.client.0.smithi012.stdout:5/25: mkdir d3/d8 0 2024-03-18T14:50:43.448 INFO:tasks.workunit.client.0.smithi012.stdout:5/26: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:43.449 INFO:tasks.workunit.client.0.smithi012.stdout:5/27: mknod d3/d8/c9 0 2024-03-18T14:50:43.451 INFO:tasks.workunit.client.0.smithi012.stdout:5/28: symlink d3/la 0 2024-03-18T14:50:43.454 INFO:tasks.workunit.client.0.smithi012.stdout:5/29: creat d3/d8/fb x:0 0 0 2024-03-18T14:50:43.459 INFO:tasks.workunit.client.0.smithi012.stdout:5/30: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:43.459 INFO:tasks.workunit.client.0.smithi012.stdout:5/31: creat d3/fc x:0 0 0 2024-03-18T14:50:43.459 INFO:tasks.workunit.client.0.smithi012.stdout:2/34: dread d0/f8 [0,4194304] 0 2024-03-18T14:50:43.460 INFO:tasks.workunit.client.0.smithi012.stdout:2/35: write d0/f6 [557488,125332] 0 2024-03-18T14:50:43.460 INFO:tasks.workunit.client.0.smithi012.stdout:2/36: fsync d0/f8 0 2024-03-18T14:50:43.460 INFO:tasks.workunit.client.0.smithi012.stdout:2/37: read d0/f6 [620711,86712] 0 2024-03-18T14:50:43.464 INFO:tasks.workunit.client.0.smithi012.stdout:5/32: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:43.464 INFO:tasks.workunit.client.0.smithi012.stdout:5/33: truncate d3/d8/fb 759790 0 2024-03-18T14:50:43.465 INFO:tasks.workunit.client.0.smithi012.stdout:2/38: mknod d0/cd 0 2024-03-18T14:50:43.466 INFO:tasks.workunit.client.0.smithi012.stdout:5/34: truncate d3/f5 2397473 0 2024-03-18T14:50:43.469 INFO:tasks.workunit.client.0.smithi012.stdout:5/35: symlink d3/d8/ld 0 2024-03-18T14:50:43.471 INFO:tasks.workunit.client.0.smithi012.stdout:2/39: rename d0/c7 to d0/ce 0 2024-03-18T14:50:43.472 INFO:tasks.workunit.client.0.smithi012.stdout:5/36: mknod d3/d8/ce 0 2024-03-18T14:50:43.472 INFO:tasks.workunit.client.0.smithi012.stdout:5/37: chown d3/d8/ce 293 1 2024-03-18T14:50:43.472 INFO:tasks.workunit.client.0.smithi012.stdout:5/38: creat d3/ff x:0 0 0 2024-03-18T14:50:43.472 INFO:tasks.workunit.client.0.smithi012.stdout:5/39: fdatasync d3/ff 0 2024-03-18T14:50:43.474 INFO:tasks.workunit.client.0.smithi012.stdout:2/40: dread d0/f8 [0,4194304] 0 2024-03-18T14:50:43.478 INFO:tasks.workunit.client.0.smithi012.stdout:2/41: mknod d0/cf 0 2024-03-18T14:50:43.481 INFO:tasks.workunit.client.0.smithi012.stdout:2/42: truncate d0/f5 1485380 0 2024-03-18T14:50:43.489 INFO:tasks.workunit.client.0.smithi012.stdout:5/40: dwrite d3/ff [0,4194304] 0 2024-03-18T14:50:43.491 INFO:tasks.workunit.client.0.smithi012.stdout:5/41: dread f2 [0,4194304] 0 2024-03-18T14:50:43.507 INFO:tasks.workunit.client.0.smithi012.stdout:5/42: unlink d3/f5 0 2024-03-18T14:50:43.508 INFO:tasks.workunit.client.0.smithi012.stdout:5/43: chown d3/d8/c9 2327970 1 2024-03-18T14:50:43.510 INFO:tasks.workunit.client.0.smithi012.stdout:5/44: symlink d3/d8/l10 0 2024-03-18T14:50:43.510 INFO:tasks.workunit.client.0.smithi012.stdout:5/45: write d3/fc [649882,3663] 0 2024-03-18T14:50:43.510 INFO:tasks.workunit.client.0.smithi012.stdout:5/46: truncate d3/fc 1190249 0 2024-03-18T14:50:43.511 INFO:tasks.workunit.client.0.smithi012.stdout:5/47: read d3/fc [1158165,74724] 0 2024-03-18T14:50:43.511 INFO:tasks.workunit.client.0.smithi012.stdout:5/48: write f2 [1232124,41503] 0 2024-03-18T14:50:43.518 INFO:tasks.workunit.client.0.smithi012.stdout:5/49: dread d3/ff [0,4194304] 0 2024-03-18T14:50:43.518 INFO:tasks.workunit.client.0.smithi012.stdout:5/50: write d3/d8/fb [1681944,59191] 0 2024-03-18T14:50:43.520 INFO:tasks.workunit.client.0.smithi012.stdout:5/51: symlink d3/d8/l11 0 2024-03-18T14:50:43.522 INFO:tasks.workunit.client.0.smithi012.stdout:2/43: write d0/f5 [857953,105239] 0 2024-03-18T14:50:43.522 INFO:tasks.workunit.client.0.smithi012.stdout:2/44: write d0/f6 [142290,81584] 0 2024-03-18T14:50:43.546 INFO:tasks.workunit.client.0.smithi012.stdout:2/45: dwrite d0/fa [0,4194304] 0 2024-03-18T14:50:43.548 INFO:tasks.workunit.client.0.smithi012.stdout:5/52: dwrite f2 [0,4194304] 0 2024-03-18T14:50:43.548 INFO:tasks.workunit.client.0.smithi012.stdout:5/53: truncate d3/fc 1306013 0 2024-03-18T14:50:43.556 INFO:tasks.workunit.client.0.smithi012.stdout:2/46: mkdir d0/d10 0 2024-03-18T14:50:43.570 INFO:tasks.workunit.client.0.smithi012.stdout:5/54: dwrite d3/d8/fb [0,4194304] 0 2024-03-18T14:50:43.573 INFO:tasks.workunit.client.0.smithi012.stdout:5/55: symlink d3/d8/l12 0 2024-03-18T14:50:43.573 INFO:tasks.workunit.client.0.smithi012.stdout:5/56: creat d3/d8/f13 x:0 0 0 2024-03-18T14:50:43.573 INFO:tasks.workunit.client.0.smithi012.stdout:5/57: stat d3/d8/c9 0 2024-03-18T14:50:43.573 INFO:tasks.workunit.client.0.smithi012.stdout:5/58: chown d3/d8/l10 108858342 1 2024-03-18T14:50:43.577 INFO:tasks.workunit.client.0.smithi012.stdout:5/59: unlink d3/d8/l11 0 2024-03-18T14:50:43.577 INFO:tasks.workunit.client.0.smithi012.stdout:5/60: creat d3/f14 x:0 0 0 2024-03-18T14:50:43.585 INFO:tasks.workunit.client.0.smithi012.stdout:5/61: write f2 [2237538,14058] 0 2024-03-18T14:50:43.590 INFO:tasks.workunit.client.0.smithi012.stdout:5/62: dread d3/fc [0,4194304] 0 2024-03-18T14:50:43.599 INFO:tasks.workunit.client.0.smithi012.stdout:2/47: rmdir d0 39 2024-03-18T14:50:43.613 INFO:tasks.workunit.client.0.smithi012.stdout:5/63: dwrite d3/f14 [0,4194304] 0 2024-03-18T14:50:43.614 INFO:tasks.workunit.client.0.smithi012.stdout:2/48: dwrite d0/f8 [0,4194304] 0 2024-03-18T14:50:43.618 INFO:tasks.workunit.client.0.smithi012.stdout:5/64: mkdir d3/d15 0 2024-03-18T14:50:43.619 INFO:tasks.workunit.client.0.smithi012.stdout:2/49: getdents d0 0 2024-03-18T14:50:43.621 INFO:tasks.workunit.client.0.smithi012.stdout:5/65: symlink d3/d15/l16 0 2024-03-18T14:50:43.621 INFO:tasks.workunit.client.0.smithi012.stdout:5/66: write d3/fc [2145627,20623] 0 2024-03-18T14:50:43.621 INFO:tasks.workunit.client.0.smithi012.stdout:5/67: read - d3/d8/f13 zero size 2024-03-18T14:50:43.621 INFO:tasks.workunit.client.0.smithi012.stdout:2/50: rmdir d0 39 2024-03-18T14:50:43.623 INFO:tasks.workunit.client.0.smithi012.stdout:5/68: symlink d3/d8/l17 0 2024-03-18T14:50:43.624 INFO:tasks.workunit.client.0.smithi012.stdout:2/51: creat d0/f11 x:0 0 0 2024-03-18T14:50:43.628 INFO:tasks.workunit.client.0.smithi012.stdout:2/52: truncate d0/fa 1976051 0 2024-03-18T14:50:43.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/53: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:43.664 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:43 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:43.665 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:43 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:43.665 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:43 smithi100 ceph-mon[37807]: pgmap v239: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 15 MiB/s wr, 560 op/s 2024-03-18T14:50:43.667 INFO:tasks.workunit.client.0.smithi012.stdout:2/54: dwrite d0/f11 [0,4194304] 0 2024-03-18T14:50:43.671 INFO:tasks.workunit.client.0.smithi012.stdout:5/69: truncate d3/d8/fb 3565641 0 2024-03-18T14:50:43.674 INFO:tasks.workunit.client.0.smithi012.stdout:2/55: symlink d0/d10/l12 0 2024-03-18T14:50:43.675 INFO:tasks.workunit.client.0.smithi012.stdout:2/56: fdatasync d0/fa 0 2024-03-18T14:50:43.692 INFO:tasks.workunit.client.0.smithi012.stdout:2/57: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:43.734 INFO:tasks.workunit.client.0.smithi012.stdout:5/70: dwrite d3/d8/fb [0,4194304] 0 2024-03-18T14:50:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:43 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:43 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:43 smithi012 ceph-mon[29182]: pgmap v239: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 15 MiB/s wr, 560 op/s 2024-03-18T14:50:44.570 INFO:tasks.workunit.client.0.smithi012.stdout:6/28: sync 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:9/140: sync 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:1/77: sync 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:7/17: sync 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:4/49: sync 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:1/78: stat f3 0 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:6/29: dread - f4 zero size 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:4/50: readlink d6/l7 0 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:7/18: chown c9 2915 1 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:7/19: dread - f7 zero size 2024-03-18T14:50:44.571 INFO:tasks.workunit.client.0.smithi012.stdout:3/77: sync 2024-03-18T14:50:44.572 INFO:tasks.workunit.client.0.smithi012.stdout:8/84: sync 2024-03-18T14:50:44.572 INFO:tasks.workunit.client.0.smithi012.stdout:0/79: sync 2024-03-18T14:50:44.572 INFO:tasks.workunit.client.0.smithi012.stdout:5/71: sync 2024-03-18T14:50:44.572 INFO:tasks.workunit.client.0.smithi012.stdout:2/58: sync 2024-03-18T14:50:44.572 INFO:tasks.workunit.client.0.smithi012.stdout:2/59: readlink d0/d10/l12 0 2024-03-18T14:50:44.575 INFO:tasks.workunit.client.0.smithi012.stdout:6/30: rename f1 to f5 0 2024-03-18T14:50:44.576 INFO:tasks.workunit.client.0.smithi012.stdout:6/31: creat f6 x:0 0 0 2024-03-18T14:50:44.576 INFO:tasks.workunit.client.0.smithi012.stdout:6/32: write f4 [702693,109832] 0 2024-03-18T14:50:44.576 INFO:tasks.workunit.client.0.smithi012.stdout:6/33: stat f6 0 2024-03-18T14:50:44.577 INFO:tasks.workunit.client.0.smithi012.stdout:1/79: symlink d8/db/l1b 0 2024-03-18T14:50:44.578 INFO:tasks.workunit.client.0.smithi012.stdout:7/20: symlink lb 0 2024-03-18T14:50:44.580 INFO:tasks.workunit.client.0.smithi012.stdout:8/85: symlink d0/d18/d19/d1b/l1c 0 2024-03-18T14:50:44.580 INFO:tasks.workunit.client.0.smithi012.stdout:8/86: fsync d0/f14 0 2024-03-18T14:50:44.580 INFO:tasks.workunit.client.0.smithi012.stdout:8/87: dread - d0/f16 zero size 2024-03-18T14:50:44.580 INFO:tasks.workunit.client.0.smithi012.stdout:8/88: chown d0/f14 50 1 2024-03-18T14:50:44.584 INFO:tasks.workunit.client.0.smithi012.stdout:9/141: rmdir d3/d22 0 2024-03-18T14:50:44.588 INFO:tasks.workunit.client.0.smithi012.stdout:4/51: unlink c0 0 2024-03-18T14:50:44.592 INFO:tasks.workunit.client.0.smithi012.stdout:7/21: link l4 lc 0 2024-03-18T14:50:44.592 INFO:tasks.workunit.client.0.smithi012.stdout:7/22: fsync f1 0 2024-03-18T14:50:44.593 INFO:tasks.workunit.client.0.smithi012.stdout:8/89: creat d0/d18/f1d x:0 0 0 2024-03-18T14:50:44.594 INFO:tasks.workunit.client.0.smithi012.stdout:9/142: truncate d3/f25 558435 0 2024-03-18T14:50:44.595 INFO:tasks.workunit.client.0.smithi012.stdout:4/52: creat d6/fe x:0 0 0 2024-03-18T14:50:44.596 INFO:tasks.workunit.client.0.smithi012.stdout:4/53: readlink d6/l7 0 2024-03-18T14:50:44.596 INFO:tasks.workunit.client.0.smithi012.stdout:4/54: creat d6/ff x:0 0 0 2024-03-18T14:50:44.596 INFO:tasks.workunit.client.0.smithi012.stdout:8/90: dread d0/fb [0,4194304] 0 2024-03-18T14:50:44.596 INFO:tasks.workunit.client.0.smithi012.stdout:8/91: readlink d0/l13 0 2024-03-18T14:50:44.597 INFO:tasks.workunit.client.0.smithi012.stdout:7/23: mkdir dd 0 2024-03-18T14:50:44.597 INFO:tasks.workunit.client.0.smithi012.stdout:7/24: chown l4 32327082 1 2024-03-18T14:50:44.597 INFO:tasks.workunit.client.0.smithi012.stdout:7/25: write f1 [57965,16346] 0 2024-03-18T14:50:44.598 INFO:tasks.workunit.client.0.smithi012.stdout:9/143: creat d3/d9/f29 x:0 0 0 2024-03-18T14:50:44.598 INFO:tasks.workunit.client.0.smithi012.stdout:9/144: write d3/d9/f28 [846316,92241] 0 2024-03-18T14:50:44.598 INFO:tasks.workunit.client.0.smithi012.stdout:9/145: write d3/d9/f10 [4286152,25912] 0 2024-03-18T14:50:44.603 INFO:tasks.workunit.client.0.smithi012.stdout:4/55: unlink f2 0 2024-03-18T14:50:44.603 INFO:tasks.workunit.client.0.smithi012.stdout:4/56: stat d6 0 2024-03-18T14:50:44.606 INFO:tasks.workunit.client.0.smithi012.stdout:8/92: rename d0/f14 to d0/d12/f1e 0 2024-03-18T14:50:44.606 INFO:tasks.workunit.client.0.smithi012.stdout:8/93: readlink d0/d18/d19/d1b/l1c 0 2024-03-18T14:50:44.606 INFO:tasks.workunit.client.0.smithi012.stdout:8/94: creat d0/f1f x:0 0 0 2024-03-18T14:50:44.608 INFO:tasks.workunit.client.0.smithi012.stdout:7/26: symlink dd/le 0 2024-03-18T14:50:44.608 INFO:tasks.workunit.client.0.smithi012.stdout:7/27: write fa [212119,119271] 0 2024-03-18T14:50:44.609 INFO:tasks.workunit.client.0.smithi012.stdout:4/57: mknod d6/c10 0 2024-03-18T14:50:44.612 INFO:tasks.workunit.client.0.smithi012.stdout:8/95: creat d0/d12/f20 x:0 0 0 2024-03-18T14:50:44.614 INFO:tasks.workunit.client.0.smithi012.stdout:7/28: rename lb to dd/lf 0 2024-03-18T14:50:44.615 INFO:tasks.workunit.client.0.smithi012.stdout:7/29: read f5 [890964,124087] 0 2024-03-18T14:50:44.616 INFO:tasks.workunit.client.0.smithi012.stdout:4/58: rename f3 to d6/f11 0 2024-03-18T14:50:44.618 INFO:tasks.workunit.client.0.smithi012.stdout:7/30: creat dd/f10 x:0 0 0 2024-03-18T14:50:44.618 INFO:tasks.workunit.client.0.smithi012.stdout:7/31: creat dd/f11 x:0 0 0 2024-03-18T14:50:44.619 INFO:tasks.workunit.client.0.smithi012.stdout:8/96: rename d0/d18/d19/d1b/l1c to d0/d15/l21 0 2024-03-18T14:50:44.619 INFO:tasks.workunit.client.0.smithi012.stdout:8/97: fsync d0/d15/f1a 0 2024-03-18T14:50:44.621 INFO:tasks.workunit.client.0.smithi012.stdout:4/59: link f5 d6/f12 0 2024-03-18T14:50:44.623 INFO:tasks.workunit.client.0.smithi012.stdout:8/98: rmdir d0/d18/d19 39 2024-03-18T14:50:44.625 INFO:tasks.workunit.client.0.smithi012.stdout:4/60: rename f5 to d6/da/f13 0 2024-03-18T14:50:44.626 INFO:tasks.workunit.client.0.smithi012.stdout:4/61: write d6/f9 [816347,104853] 0 2024-03-18T14:50:44.648 INFO:tasks.workunit.client.0.smithi012.stdout:2/60: dread d0/f5 [0,4194304] 0 2024-03-18T14:50:44.648 INFO:tasks.workunit.client.0.smithi012.stdout:2/61: truncate d0/fa 249560 0 2024-03-18T14:50:44.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/62: creat d0/d10/f13 x:0 0 0 2024-03-18T14:50:44.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/63: creat d0/f14 x:0 0 0 2024-03-18T14:50:44.650 INFO:tasks.workunit.client.0.smithi012.stdout:0/80: dwrite d1/f9 [0,4194304] 0 2024-03-18T14:50:44.652 INFO:tasks.workunit.client.0.smithi012.stdout:2/64: link d0/l3 d0/d10/l15 0 2024-03-18T14:50:44.652 INFO:tasks.workunit.client.0.smithi012.stdout:2/65: readlink d0/d10/l12 0 2024-03-18T14:50:44.655 INFO:tasks.workunit.client.0.smithi012.stdout:0/81: mkdir d1/d20 0 2024-03-18T14:50:44.655 INFO:tasks.workunit.client.0.smithi012.stdout:0/82: stat d1/l4 0 2024-03-18T14:50:44.657 INFO:tasks.workunit.client.0.smithi012.stdout:0/83: unlink d1/fd 0 2024-03-18T14:50:44.658 INFO:tasks.workunit.client.0.smithi012.stdout:9/146: dwrite d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:44.659 INFO:tasks.workunit.client.0.smithi012.stdout:1/80: dwrite f3 [0,4194304] 0 2024-03-18T14:50:44.659 INFO:tasks.workunit.client.0.smithi012.stdout:1/81: dread - d8/f1a zero size 2024-03-18T14:50:44.659 INFO:tasks.workunit.client.0.smithi012.stdout:3/78: dwrite fb [0,4194304] 0 2024-03-18T14:50:44.659 INFO:tasks.workunit.client.0.smithi012.stdout:5/72: dwrite d3/d8/f13 [0,4194304] 0 2024-03-18T14:50:44.659 INFO:tasks.workunit.client.0.smithi012.stdout:5/73: readlink d3/l6 0 2024-03-18T14:50:44.660 INFO:tasks.workunit.client.0.smithi012.stdout:0/84: dread d1/d16/fe [0,4194304] 0 2024-03-18T14:50:44.662 INFO:tasks.workunit.client.0.smithi012.stdout:1/82: mkdir d8/db/d10/d16/d1c 0 2024-03-18T14:50:44.664 INFO:tasks.workunit.client.0.smithi012.stdout:5/74: dread d3/fc [0,4194304] 0 2024-03-18T14:50:44.669 INFO:tasks.workunit.client.0.smithi012.stdout:5/75: dread d3/fc [0,4194304] 0 2024-03-18T14:50:44.669 INFO:tasks.workunit.client.0.smithi012.stdout:9/147: mknod d3/c2a 0 2024-03-18T14:50:44.670 INFO:tasks.workunit.client.0.smithi012.stdout:4/62: dwrite d6/f9 [0,4194304] 0 2024-03-18T14:50:44.670 INFO:tasks.workunit.client.0.smithi012.stdout:1/83: symlink d8/db/d10/l1d 0 2024-03-18T14:50:44.672 INFO:tasks.workunit.client.0.smithi012.stdout:3/79: rename l7 to dc/l1d 0 2024-03-18T14:50:44.673 INFO:tasks.workunit.client.0.smithi012.stdout:5/76: symlink d3/d8/l18 0 2024-03-18T14:50:44.677 INFO:tasks.workunit.client.0.smithi012.stdout:1/84: mkdir d8/d1e 0 2024-03-18T14:50:44.678 INFO:tasks.workunit.client.0.smithi012.stdout:9/148: getdents d3/d1e/d21 0 2024-03-18T14:50:44.678 INFO:tasks.workunit.client.0.smithi012.stdout:9/149: stat d3/d9/f29 0 2024-03-18T14:50:44.680 INFO:tasks.workunit.client.0.smithi012.stdout:1/85: rename d8/db/le to d8/db/d10/d16/d1c/l1f 0 2024-03-18T14:50:44.681 INFO:tasks.workunit.client.0.smithi012.stdout:0/85: dwrite d1/d16/d18/f13 [0,4194304] 0 2024-03-18T14:50:44.683 INFO:tasks.workunit.client.0.smithi012.stdout:3/80: link dc/l18 dc/l1e 0 2024-03-18T14:50:44.683 INFO:tasks.workunit.client.0.smithi012.stdout:3/81: write f6 [4990453,112246] 0 2024-03-18T14:50:44.684 INFO:tasks.workunit.client.0.smithi012.stdout:9/150: rmdir d3/d9/d19 39 2024-03-18T14:50:44.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/63: write d6/f9 [2713608,97208] 0 2024-03-18T14:50:44.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/64: read - d6/da/f13 zero size 2024-03-18T14:50:44.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/65: fsync f1 0 2024-03-18T14:50:44.691 INFO:tasks.workunit.client.0.smithi012.stdout:9/151: creat d3/d1e/f2b x:0 0 0 2024-03-18T14:50:44.691 INFO:tasks.workunit.client.0.smithi012.stdout:9/152: dread - d3/d1e/d21/f26 zero size 2024-03-18T14:50:44.692 INFO:tasks.workunit.client.0.smithi012.stdout:3/82: truncate dc/f11 3203406 0 2024-03-18T14:50:44.692 INFO:tasks.workunit.client.0.smithi012.stdout:9/153: stat d3/f16 0 2024-03-18T14:50:44.692 INFO:tasks.workunit.client.0.smithi012.stdout:9/154: chown d3/d1e 731670451 1 2024-03-18T14:50:44.692 INFO:tasks.workunit.client.0.smithi012.stdout:9/155: chown d3/d9/f10 7259855 1 2024-03-18T14:50:44.695 INFO:tasks.workunit.client.0.smithi012.stdout:3/83: write f3 [7082959,42419] 0 2024-03-18T14:50:44.696 INFO:tasks.workunit.client.0.smithi012.stdout:3/84: rmdir dc 39 2024-03-18T14:50:44.699 INFO:tasks.workunit.client.0.smithi012.stdout:5/77: dwrite d3/d8/f13 [0,4194304] 0 2024-03-18T14:50:44.699 INFO:tasks.workunit.client.0.smithi012.stdout:5/78: chown d3/d8/c9 499339 1 2024-03-18T14:50:44.699 INFO:tasks.workunit.client.0.smithi012.stdout:5/79: fdatasync d3/fc 0 2024-03-18T14:50:44.699 INFO:tasks.workunit.client.0.smithi012.stdout:5/80: stat d3/d8 0 2024-03-18T14:50:44.700 INFO:tasks.workunit.client.0.smithi012.stdout:1/86: dwrite f5 [0,4194304] 0 2024-03-18T14:50:44.700 INFO:tasks.workunit.client.0.smithi012.stdout:1/87: dread - d8/db/fd zero size 2024-03-18T14:50:44.700 INFO:tasks.workunit.client.0.smithi012.stdout:3/85: fsync f2 0 2024-03-18T14:50:44.700 INFO:tasks.workunit.client.0.smithi012.stdout:3/86: chown dc/c1a 6453 1 2024-03-18T14:50:44.701 INFO:tasks.workunit.client.0.smithi012.stdout:1/88: write d8/fc [4440460,129716] 0 2024-03-18T14:50:44.706 INFO:tasks.workunit.client.0.smithi012.stdout:3/87: dread f3 [4194304,4194304] 0 2024-03-18T14:50:44.706 INFO:tasks.workunit.client.0.smithi012.stdout:5/81: link d3/d8/f13 d3/d15/f19 0 2024-03-18T14:50:44.717 INFO:tasks.workunit.client.0.smithi012.stdout:9/156: dwrite d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:44.717 INFO:tasks.workunit.client.0.smithi012.stdout:9/157: fsync d3/d9/f28 0 2024-03-18T14:50:44.718 INFO:tasks.workunit.client.0.smithi012.stdout:9/158: write d3/d9/f10 [3750239,92645] 0 2024-03-18T14:50:44.718 INFO:tasks.workunit.client.0.smithi012.stdout:1/89: dwrite d8/db/d10/f13 [0,4194304] 0 2024-03-18T14:50:44.721 INFO:tasks.workunit.client.0.smithi012.stdout:1/90: rmdir d8/db/d10/d16 39 2024-03-18T14:50:44.722 INFO:tasks.workunit.client.0.smithi012.stdout:9/159: mkdir d3/d9/d2c 0 2024-03-18T14:50:44.723 INFO:tasks.workunit.client.0.smithi012.stdout:9/160: write d3/f6 [1200928,108587] 0 2024-03-18T14:50:44.724 INFO:tasks.workunit.client.0.smithi012.stdout:1/91: write d8/db/d10/f11 [2277790,69615] 0 2024-03-18T14:50:44.725 INFO:tasks.workunit.client.0.smithi012.stdout:9/161: unlink d3/c2a 0 2024-03-18T14:50:44.725 INFO:tasks.workunit.client.0.smithi012.stdout:9/162: dread - d3/d1e/d21/f26 zero size 2024-03-18T14:50:44.727 INFO:tasks.workunit.client.0.smithi012.stdout:1/92: symlink d8/db/l20 0 2024-03-18T14:50:44.729 INFO:tasks.workunit.client.0.smithi012.stdout:1/93: unlink f3 0 2024-03-18T14:50:44.729 INFO:tasks.workunit.client.0.smithi012.stdout:1/94: chown d8/db 256825 1 2024-03-18T14:50:44.729 INFO:tasks.workunit.client.0.smithi012.stdout:1/95: write f4 [1617606,16695] 0 2024-03-18T14:50:44.825 INFO:tasks.workunit.client.0.smithi012.stdout:6/34: fsync f6 0 2024-03-18T14:50:44.826 INFO:tasks.workunit.client.0.smithi012.stdout:6/35: mknod c7 0 2024-03-18T14:50:44.826 INFO:tasks.workunit.client.0.smithi012.stdout:6/36: dread - f6 zero size 2024-03-18T14:50:44.827 INFO:tasks.workunit.client.0.smithi012.stdout:6/37: rename f6 to f8 0 2024-03-18T14:50:44.828 INFO:tasks.workunit.client.0.smithi012.stdout:6/38: creat f9 x:0 0 0 2024-03-18T14:50:44.828 INFO:tasks.workunit.client.0.smithi012.stdout:6/39: dread - f8 zero size 2024-03-18T14:50:44.828 INFO:tasks.workunit.client.0.smithi012.stdout:6/40: truncate f8 2088 0 2024-03-18T14:50:44.843 INFO:tasks.workunit.client.0.smithi012.stdout:7/32: rmdir dd 39 2024-03-18T14:50:44.846 INFO:tasks.workunit.client.0.smithi012.stdout:7/33: creat dd/f12 x:0 0 0 2024-03-18T14:50:44.848 INFO:tasks.workunit.client.0.smithi012.stdout:7/34: mkdir dd/d13 0 2024-03-18T14:50:44.849 INFO:tasks.workunit.client.0.smithi012.stdout:7/35: chown dd/le 9031470 1 2024-03-18T14:50:44.851 INFO:tasks.workunit.client.0.smithi012.stdout:7/36: symlink dd/l14 0 2024-03-18T14:50:44.853 INFO:tasks.workunit.client.0.smithi012.stdout:7/37: creat dd/d13/f15 x:0 0 0 2024-03-18T14:50:44.854 INFO:tasks.workunit.client.0.smithi012.stdout:7/38: rename lc to dd/l16 0 2024-03-18T14:50:44.855 INFO:tasks.workunit.client.0.smithi012.stdout:7/39: symlink dd/d13/l17 0 2024-03-18T14:50:44.855 INFO:tasks.workunit.client.0.smithi012.stdout:7/40: truncate f5 1196114 0 2024-03-18T14:50:44.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/41: symlink dd/d13/l18 0 2024-03-18T14:50:44.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/42: write dd/d13/f15 [702730,18170] 0 2024-03-18T14:50:44.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/43: write f3 [263790,20599] 0 2024-03-18T14:50:44.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/44: write f3 [381542,33782] 0 2024-03-18T14:50:44.881 INFO:tasks.workunit.client.0.smithi012.stdout:3/88: getdents dc 0 2024-03-18T14:50:44.884 INFO:tasks.workunit.client.0.smithi012.stdout:3/89: creat dc/f1f x:0 0 0 2024-03-18T14:50:44.889 INFO:tasks.workunit.client.0.smithi012.stdout:6/41: dwrite f5 [0,4194304] 0 2024-03-18T14:50:44.889 INFO:tasks.workunit.client.0.smithi012.stdout:6/42: fdatasync f4 0 2024-03-18T14:50:44.893 INFO:tasks.workunit.client.0.smithi012.stdout:2/66: dwrite d0/fa [0,4194304] 0 2024-03-18T14:50:44.898 INFO:tasks.workunit.client.0.smithi012.stdout:2/67: rename d0/f11 to d0/d10/f16 0 2024-03-18T14:50:44.907 INFO:tasks.workunit.client.0.smithi012.stdout:6/43: read f5 [1082497,12541] 0 2024-03-18T14:50:44.910 INFO:tasks.workunit.client.0.smithi012.stdout:6/44: rename c7 to ca 0 2024-03-18T14:50:44.910 INFO:tasks.workunit.client.0.smithi012.stdout:6/45: creat fb x:0 0 0 2024-03-18T14:50:44.911 INFO:tasks.workunit.client.0.smithi012.stdout:7/45: dwrite f5 [0,4194304] 0 2024-03-18T14:50:44.911 INFO:tasks.workunit.client.0.smithi012.stdout:7/46: truncate dd/f11 75345 0 2024-03-18T14:50:44.915 INFO:tasks.workunit.client.0.smithi012.stdout:6/46: dread f5 [0,4194304] 0 2024-03-18T14:50:44.915 INFO:tasks.workunit.client.0.smithi012.stdout:6/47: rmdir - no directory 2024-03-18T14:50:44.915 INFO:tasks.workunit.client.0.smithi012.stdout:6/48: stat f8 0 2024-03-18T14:50:44.920 INFO:tasks.workunit.client.0.smithi012.stdout:6/49: dread f4 [0,4194304] 0 2024-03-18T14:50:44.920 INFO:tasks.workunit.client.0.smithi012.stdout:6/50: write f5 [583153,85996] 0 2024-03-18T14:50:44.920 INFO:tasks.workunit.client.0.smithi012.stdout:0/86: dwrite d1/d16/d18/f13 [4194304,4194304] 0 2024-03-18T14:50:44.920 INFO:tasks.workunit.client.0.smithi012.stdout:0/87: creat d1/d16/d19/f21 x:0 0 0 2024-03-18T14:50:44.920 INFO:tasks.workunit.client.0.smithi012.stdout:2/68: dwrite d0/f5 [4194304,4194304] 0 2024-03-18T14:50:44.929 INFO:tasks.workunit.client.0.smithi012.stdout:7/47: dwrite fa [0,4194304] 0 2024-03-18T14:50:44.929 INFO:tasks.workunit.client.0.smithi012.stdout:7/48: creat dd/d13/f19 x:0 0 0 2024-03-18T14:50:44.932 INFO:tasks.workunit.client.0.smithi012.stdout:6/51: unlink l2 0 2024-03-18T14:50:44.932 INFO:tasks.workunit.client.0.smithi012.stdout:6/52: write f4 [1299534,102022] 0 2024-03-18T14:50:44.937 INFO:tasks.workunit.client.0.smithi012.stdout:2/69: dwrite d0/f6 [0,4194304] 0 2024-03-18T14:50:44.937 INFO:tasks.workunit.client.0.smithi012.stdout:2/70: dread - d0/f14 zero size 2024-03-18T14:50:44.938 INFO:tasks.workunit.client.0.smithi012.stdout:2/71: chown d0/cd 1214602 1 2024-03-18T14:50:44.945 INFO:tasks.workunit.client.0.smithi012.stdout:0/88: mkdir d1/d16/d18/d22 0 2024-03-18T14:50:44.954 INFO:tasks.workunit.client.0.smithi012.stdout:7/49: dwrite fa [0,4194304] 0 2024-03-18T14:50:44.954 INFO:tasks.workunit.client.0.smithi012.stdout:6/53: dwrite f4 [0,4194304] 0 2024-03-18T14:50:44.954 INFO:tasks.workunit.client.0.smithi012.stdout:6/54: write f5 [1475187,62555] 0 2024-03-18T14:50:44.954 INFO:tasks.workunit.client.0.smithi012.stdout:6/55: write fb [100342,121207] 0 2024-03-18T14:50:44.960 INFO:tasks.workunit.client.0.smithi012.stdout:7/50: dread dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:44.960 INFO:tasks.workunit.client.0.smithi012.stdout:7/51: write dd/f11 [380608,24845] 0 2024-03-18T14:50:44.961 INFO:tasks.workunit.client.0.smithi012.stdout:0/89: dwrite d1/d16/d19/f1b [0,4194304] 0 2024-03-18T14:50:44.963 INFO:tasks.workunit.client.0.smithi012.stdout:2/72: symlink d0/l17 0 2024-03-18T14:50:44.969 INFO:tasks.workunit.client.0.smithi012.stdout:0/90: read d1/f3 [1366206,37235] 0 2024-03-18T14:50:44.970 INFO:tasks.workunit.client.0.smithi012.stdout:6/56: symlink lc 0 2024-03-18T14:50:44.970 INFO:tasks.workunit.client.0.smithi012.stdout:6/57: stat f8 0 2024-03-18T14:50:44.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/58: truncate f8 424536 0 2024-03-18T14:50:44.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/59: creat fd x:0 0 0 2024-03-18T14:50:44.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/60: creat fe x:0 0 0 2024-03-18T14:50:44.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/61: chown l3 32041 1 2024-03-18T14:50:44.972 INFO:tasks.workunit.client.0.smithi012.stdout:7/52: mknod dd/d13/c1a 0 2024-03-18T14:50:44.974 INFO:tasks.workunit.client.0.smithi012.stdout:7/53: dread f1 [0,4194304] 0 2024-03-18T14:50:44.975 INFO:tasks.workunit.client.0.smithi012.stdout:7/54: dread f1 [0,4194304] 0 2024-03-18T14:50:44.981 INFO:tasks.workunit.client.0.smithi012.stdout:2/73: mkdir d0/d10/d18 0 2024-03-18T14:50:44.984 INFO:tasks.workunit.client.0.smithi012.stdout:6/62: dwrite f4 [0,4194304] 0 2024-03-18T14:50:44.986 INFO:tasks.workunit.client.0.smithi012.stdout:0/91: mkdir d1/d20/d23 0 2024-03-18T14:50:44.986 INFO:tasks.workunit.client.0.smithi012.stdout:0/92: rename d1 to d1/d1c/d24 22 2024-03-18T14:50:44.987 INFO:tasks.workunit.client.0.smithi012.stdout:6/63: fdatasync f5 0 2024-03-18T14:50:44.988 INFO:tasks.workunit.client.0.smithi012.stdout:6/64: chown fb 1016091 1 2024-03-18T14:50:44.988 INFO:tasks.workunit.client.0.smithi012.stdout:6/65: chown lc 160833162 1 2024-03-18T14:50:44.988 INFO:tasks.workunit.client.0.smithi012.stdout:6/66: fsync f9 0 2024-03-18T14:50:44.994 INFO:tasks.workunit.client.0.smithi012.stdout:7/55: unlink f7 0 2024-03-18T14:50:44.996 INFO:tasks.workunit.client.0.smithi012.stdout:6/67: dread f5 [0,4194304] 0 2024-03-18T14:50:44.996 INFO:tasks.workunit.client.0.smithi012.stdout:6/68: creat ff x:0 0 0 2024-03-18T14:50:44.996 INFO:tasks.workunit.client.0.smithi012.stdout:6/69: readlink l3 0 2024-03-18T14:50:44.996 INFO:tasks.workunit.client.0.smithi012.stdout:6/70: fdatasync f8 0 2024-03-18T14:50:44.998 INFO:tasks.workunit.client.0.smithi012.stdout:2/74: mknod d0/c19 0 2024-03-18T14:50:45.003 INFO:tasks.workunit.client.0.smithi012.stdout:0/93: dwrite d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:45.006 INFO:tasks.workunit.client.0.smithi012.stdout:2/75: symlink d0/d10/l1a 0 2024-03-18T14:50:45.012 INFO:tasks.workunit.client.0.smithi012.stdout:0/94: rmdir d1/d20 39 2024-03-18T14:50:45.028 INFO:tasks.workunit.client.0.smithi012.stdout:6/71: fsync fe 0 2024-03-18T14:50:45.030 INFO:tasks.workunit.client.0.smithi012.stdout:6/72: rename ca to c10 0 2024-03-18T14:50:45.030 INFO:tasks.workunit.client.0.smithi012.stdout:0/95: dwrite d1/d16/d19/f1b [0,4194304] 0 2024-03-18T14:50:45.035 INFO:tasks.workunit.client.0.smithi012.stdout:0/96: mknod d1/d20/d23/c25 0 2024-03-18T14:50:45.035 INFO:tasks.workunit.client.0.smithi012.stdout:0/97: chown d1/d16/d18/c12 8 1 2024-03-18T14:50:45.037 INFO:tasks.workunit.client.0.smithi012.stdout:6/73: read f4 [2626646,45720] 0 2024-03-18T14:50:45.037 INFO:tasks.workunit.client.0.smithi012.stdout:0/98: mkdir d1/d20/d23/d26 0 2024-03-18T14:50:45.041 INFO:tasks.workunit.client.0.smithi012.stdout:6/74: unlink lc 0 2024-03-18T14:50:45.043 INFO:tasks.workunit.client.0.smithi012.stdout:0/99: symlink d1/d20/d23/l27 0 2024-03-18T14:50:45.043 INFO:tasks.workunit.client.0.smithi012.stdout:0/100: chown d1/c6 116004234 1 2024-03-18T14:50:45.051 INFO:tasks.workunit.client.0.smithi012.stdout:2/76: truncate d0/f5 5052245 0 2024-03-18T14:50:45.053 INFO:tasks.workunit.client.0.smithi012.stdout:2/77: creat d0/d10/d18/f1b x:0 0 0 2024-03-18T14:50:45.068 INFO:tasks.workunit.client.0.smithi012.stdout:0/101: dwrite d1/d16/d19/f1b [0,4194304] 0 2024-03-18T14:50:45.068 INFO:tasks.workunit.client.0.smithi012.stdout:0/102: truncate d1/fa 885956 0 2024-03-18T14:50:45.068 INFO:tasks.workunit.client.0.smithi012.stdout:2/78: dwrite d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:45.069 INFO:tasks.workunit.client.0.smithi012.stdout:6/75: dwrite f8 [0,4194304] 0 2024-03-18T14:50:45.069 INFO:tasks.workunit.client.0.smithi012.stdout:6/76: write fe [510835,87434] 0 2024-03-18T14:50:45.069 INFO:tasks.workunit.client.0.smithi012.stdout:6/77: read f5 [542913,4458] 0 2024-03-18T14:50:45.071 INFO:tasks.workunit.client.0.smithi012.stdout:0/103: symlink d1/d1c/l28 0 2024-03-18T14:50:45.073 INFO:tasks.workunit.client.0.smithi012.stdout:6/78: dread f5 [0,4194304] 0 2024-03-18T14:50:45.074 INFO:tasks.workunit.client.0.smithi012.stdout:6/79: read fb [174849,46968] 0 2024-03-18T14:50:45.074 INFO:tasks.workunit.client.0.smithi012.stdout:6/80: dread - fd zero size 2024-03-18T14:50:45.074 INFO:tasks.workunit.client.0.smithi012.stdout:6/81: chown fe 0 1 2024-03-18T14:50:45.079 INFO:tasks.workunit.client.0.smithi012.stdout:6/82: dread fe [0,4194304] 0 2024-03-18T14:50:45.080 INFO:tasks.workunit.client.0.smithi012.stdout:0/104: link d1/d20/d23/c25 d1/d20/d23/d26/c29 0 2024-03-18T14:50:45.080 INFO:tasks.workunit.client.0.smithi012.stdout:0/105: chown d1/fa 13 1 2024-03-18T14:50:45.080 INFO:tasks.workunit.client.0.smithi012.stdout:0/106: creat d1/d16/d19/f2a x:0 0 0 2024-03-18T14:50:45.080 INFO:tasks.workunit.client.0.smithi012.stdout:0/107: stat d1/d1c 0 2024-03-18T14:50:45.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/83: dread f4 [0,4194304] 0 2024-03-18T14:50:45.085 INFO:tasks.workunit.client.0.smithi012.stdout:0/108: link d1/d20/d23/d26/c29 d1/d16/d19/c2b 0 2024-03-18T14:50:45.087 INFO:tasks.workunit.client.0.smithi012.stdout:6/84: link f5 f11 0 2024-03-18T14:50:45.087 INFO:tasks.workunit.client.0.smithi012.stdout:6/85: creat f12 x:0 0 0 2024-03-18T14:50:45.089 INFO:tasks.workunit.client.0.smithi012.stdout:0/109: creat d1/d16/d18/d22/f2c x:0 0 0 2024-03-18T14:50:45.092 INFO:tasks.workunit.client.0.smithi012.stdout:2/79: write d0/f5 [430003,37639] 0 2024-03-18T14:50:45.093 INFO:tasks.workunit.client.0.smithi012.stdout:2/80: truncate d0/d10/f16 4806022 0 2024-03-18T14:50:45.093 INFO:tasks.workunit.client.0.smithi012.stdout:2/81: creat d0/f1c x:0 0 0 2024-03-18T14:50:45.093 INFO:tasks.workunit.client.0.smithi012.stdout:6/86: mknod c13 0 2024-03-18T14:50:45.093 INFO:tasks.workunit.client.0.smithi012.stdout:6/87: chown f11 246159750 1 2024-03-18T14:50:45.098 INFO:tasks.workunit.client.0.smithi012.stdout:2/82: rename d0/d10/l1a to d0/l1d 0 2024-03-18T14:50:45.101 INFO:tasks.workunit.client.0.smithi012.stdout:6/88: truncate f11 1937748 0 2024-03-18T14:50:45.104 INFO:tasks.workunit.client.0.smithi012.stdout:6/89: rename fd to f14 0 2024-03-18T14:50:45.108 INFO:tasks.workunit.client.0.smithi012.stdout:6/90: mknod c15 0 2024-03-18T14:50:45.117 INFO:tasks.workunit.client.0.smithi012.stdout:6/91: dread f8 [0,4194304] 0 2024-03-18T14:50:45.121 INFO:tasks.workunit.client.0.smithi012.stdout:2/83: dwrite d0/f14 [0,4194304] 0 2024-03-18T14:50:45.125 INFO:tasks.workunit.client.0.smithi012.stdout:0/110: rmdir d1/d16/d19 39 2024-03-18T14:50:45.199 INFO:tasks.workunit.client.0.smithi012.stdout:2/84: symlink d0/d10/d18/l1e 0 2024-03-18T14:50:45.202 INFO:tasks.workunit.client.0.smithi012.stdout:0/111: mkdir d1/d2d 0 2024-03-18T14:50:45.203 INFO:tasks.workunit.client.0.smithi012.stdout:6/92: fsync f11 0 2024-03-18T14:50:45.203 INFO:tasks.workunit.client.0.smithi012.stdout:6/93: write fb [1033558,40675] 0 2024-03-18T14:50:45.203 INFO:tasks.workunit.client.0.smithi012.stdout:6/94: dread - f9 zero size 2024-03-18T14:50:45.203 INFO:tasks.workunit.client.0.smithi012.stdout:6/95: readlink l3 0 2024-03-18T14:50:45.213 INFO:tasks.workunit.client.0.smithi012.stdout:0/112: dread d1/f9 [0,4194304] 0 2024-03-18T14:50:45.213 INFO:tasks.workunit.client.0.smithi012.stdout:0/113: stat d1/d16/d19/c1e 0 2024-03-18T14:50:45.215 INFO:tasks.workunit.client.0.smithi012.stdout:0/114: creat d1/d20/f2e x:0 0 0 2024-03-18T14:50:45.217 INFO:tasks.workunit.client.0.smithi012.stdout:0/115: symlink d1/d1c/l2f 0 2024-03-18T14:50:45.220 INFO:tasks.workunit.client.0.smithi012.stdout:0/116: creat d1/d16/d18/f30 x:0 0 0 2024-03-18T14:50:45.222 INFO:tasks.workunit.client.0.smithi012.stdout:2/85: dwrite d0/d10/f16 [0,4194304] 0 2024-03-18T14:50:45.222 INFO:tasks.workunit.client.0.smithi012.stdout:6/96: dwrite fe [0,4194304] 0 2024-03-18T14:50:45.222 INFO:tasks.workunit.client.0.smithi012.stdout:6/97: creat f16 x:0 0 0 2024-03-18T14:50:45.223 INFO:tasks.workunit.client.0.smithi012.stdout:0/117: getdents d1/d16 0 2024-03-18T14:50:45.223 INFO:tasks.workunit.client.0.smithi012.stdout:0/118: chown d1 1816021 1 2024-03-18T14:50:45.227 INFO:tasks.workunit.client.0.smithi012.stdout:6/98: dread fb [0,4194304] 0 2024-03-18T14:50:45.232 INFO:tasks.workunit.client.0.smithi012.stdout:6/99: mkdir d17 0 2024-03-18T14:50:45.235 INFO:tasks.workunit.client.0.smithi012.stdout:6/100: dread f8 [0,4194304] 0 2024-03-18T14:50:45.241 INFO:tasks.workunit.client.0.smithi012.stdout:6/101: mkdir d17/d18 0 2024-03-18T14:50:45.246 INFO:tasks.workunit.client.0.smithi012.stdout:2/86: dwrite d0/f14 [4194304,4194304] 0 2024-03-18T14:50:45.247 INFO:tasks.workunit.client.0.smithi012.stdout:2/87: creat d0/d10/d18/f1f x:0 0 0 2024-03-18T14:50:45.250 INFO:tasks.workunit.client.0.smithi012.stdout:2/88: dread d0/f5 [4194304,4194304] 0 2024-03-18T14:50:45.250 INFO:tasks.workunit.client.0.smithi012.stdout:2/89: stat d0 0 2024-03-18T14:50:45.253 INFO:tasks.workunit.client.0.smithi012.stdout:2/90: truncate d0/d10/f13 2636017 0 2024-03-18T14:50:45.254 INFO:tasks.workunit.client.0.smithi012.stdout:6/102: dwrite fe [0,4194304] 0 2024-03-18T14:50:45.256 INFO:tasks.workunit.client.0.smithi012.stdout:0/119: rename d1/d1c to d1/d16/d31 0 2024-03-18T14:50:45.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/103: mkdir d17/d19 0 2024-03-18T14:50:45.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/104: readlink l3 0 2024-03-18T14:50:45.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/105: stat l3 0 2024-03-18T14:50:45.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/106: chown d17/d18 45184 1 2024-03-18T14:50:45.261 INFO:tasks.workunit.client.0.smithi012.stdout:0/120: mknod d1/d20/d23/d26/c32 0 2024-03-18T14:50:45.261 INFO:tasks.workunit.client.0.smithi012.stdout:0/121: readlink d1/d16/d31/l28 0 2024-03-18T14:50:45.262 INFO:tasks.workunit.client.0.smithi012.stdout:0/122: dread d1/f10 [0,4194304] 0 2024-03-18T14:50:45.262 INFO:tasks.workunit.client.0.smithi012.stdout:6/107: creat d17/f1a x:0 0 0 2024-03-18T14:50:45.262 INFO:tasks.workunit.client.0.smithi012.stdout:6/108: creat d17/f1b x:0 0 0 2024-03-18T14:50:45.262 INFO:tasks.workunit.client.0.smithi012.stdout:6/109: fsync f9 0 2024-03-18T14:50:45.262 INFO:tasks.workunit.client.0.smithi012.stdout:6/110: write d17/f1b [894385,40400] 0 2024-03-18T14:50:45.265 INFO:tasks.workunit.client.0.smithi012.stdout:2/91: dread d0/f8 [0,4194304] 0 2024-03-18T14:50:45.266 INFO:tasks.workunit.client.0.smithi012.stdout:0/123: mkdir d1/d16/d18/d22/d33 0 2024-03-18T14:50:45.267 INFO:tasks.workunit.client.0.smithi012.stdout:2/92: mknod d0/d10/c20 0 2024-03-18T14:50:45.267 INFO:tasks.workunit.client.0.smithi012.stdout:2/93: read - d0/d10/d18/f1f zero size 2024-03-18T14:50:45.268 INFO:tasks.workunit.client.0.smithi012.stdout:0/124: write d1/fa [426556,54810] 0 2024-03-18T14:50:45.271 INFO:tasks.workunit.client.0.smithi012.stdout:0/125: symlink d1/d16/d19/l34 0 2024-03-18T14:50:45.271 INFO:tasks.workunit.client.0.smithi012.stdout:0/126: dread - d1/d16/d19/f2a zero size 2024-03-18T14:50:45.274 INFO:tasks.workunit.client.0.smithi012.stdout:0/127: creat d1/d16/d31/f35 x:0 0 0 2024-03-18T14:50:45.275 INFO:tasks.workunit.client.0.smithi012.stdout:0/128: truncate d1/f3 3494607 0 2024-03-18T14:50:45.280 INFO:tasks.workunit.client.0.smithi012.stdout:0/129: creat d1/d20/d23/d26/f36 x:0 0 0 2024-03-18T14:50:45.281 INFO:tasks.workunit.client.0.smithi012.stdout:2/94: fsync d0/d10/f13 0 2024-03-18T14:50:45.281 INFO:tasks.workunit.client.0.smithi012.stdout:2/95: chown d0/l17 131 1 2024-03-18T14:50:45.281 INFO:tasks.workunit.client.0.smithi012.stdout:2/96: creat d0/f21 x:0 0 0 2024-03-18T14:50:45.284 INFO:tasks.workunit.client.0.smithi012.stdout:0/130: creat d1/d16/d19/f37 x:0 0 0 2024-03-18T14:50:45.284 INFO:tasks.workunit.client.0.smithi012.stdout:0/131: write d1/fa [1294519,97577] 0 2024-03-18T14:50:45.285 INFO:tasks.workunit.client.0.smithi012.stdout:2/97: read d0/f8 [1091060,28094] 0 2024-03-18T14:50:45.285 INFO:tasks.workunit.client.0.smithi012.stdout:2/98: stat d0/f1c 0 2024-03-18T14:50:45.288 INFO:tasks.workunit.client.0.smithi012.stdout:0/132: mknod d1/d2d/c38 0 2024-03-18T14:50:45.290 INFO:tasks.workunit.client.0.smithi012.stdout:2/99: mknod d0/c22 0 2024-03-18T14:50:45.291 INFO:tasks.workunit.client.0.smithi012.stdout:2/100: chown d0/c4 648225109 1 2024-03-18T14:50:45.356 INFO:tasks.workunit.client.1.smithi100.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-18T14:50:45.357 INFO:tasks.workunit.client.1.smithi100.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-18T14:50:45.357 INFO:tasks.workunit.client.1.smithi100.stderr:+ make 2024-03-18T14:50:45.370 INFO:tasks.workunit.client.0.smithi012.stdout:4/66: sync 2024-03-18T14:50:45.370 INFO:tasks.workunit.client.0.smithi012.stdout:9/163: sync 2024-03-18T14:50:45.370 INFO:tasks.workunit.client.0.smithi012.stdout:5/82: sync 2024-03-18T14:50:45.370 INFO:tasks.workunit.client.0.smithi012.stdout:8/99: sync 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:9/164: dread - d3/d9/f29 zero size 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:1/96: sync 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:1/97: fdatasync d8/db/fd 0 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:5/83: creat d3/d8/f1a x:0 0 0 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:9/165: write d3/d1e/d21/f26 [742404,98115] 0 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:9/166: write d3/d1e/d21/f26 [1241353,129019] 0 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:8/100: link d0/d15/c17 d0/d15/c22 0 2024-03-18T14:50:45.371 INFO:tasks.workunit.client.0.smithi012.stdout:8/101: truncate d0/fb 1015458 0 2024-03-18T14:50:45.373 INFO:tasks.workunit.client.0.smithi012.stdout:1/98: truncate d8/fa 5018964 0 2024-03-18T14:50:45.375 INFO:tasks.workunit.client.0.smithi012.stdout:5/84: creat d3/d15/f1b x:0 0 0 2024-03-18T14:50:45.375 INFO:tasks.workunit.client.0.smithi012.stdout:5/85: creat d3/d15/f1c x:0 0 0 2024-03-18T14:50:45.376 INFO:tasks.workunit.client.0.smithi012.stdout:5/86: fdatasync d3/d8/fb 0 2024-03-18T14:50:45.376 INFO:tasks.workunit.client.0.smithi012.stdout:5/87: write d3/d15/f1c [922186,70286] 0 2024-03-18T14:50:45.376 INFO:tasks.workunit.client.0.smithi012.stdout:5/88: creat d3/d15/f1d x:0 0 0 2024-03-18T14:50:45.376 INFO:tasks.workunit.client.0.smithi012.stdout:9/167: chown d3/ca 1736 1 2024-03-18T14:50:45.378 INFO:tasks.workunit.client.0.smithi012.stdout:4/67: mkdir d6/d14 0 2024-03-18T14:50:45.379 INFO:tasks.workunit.client.0.smithi012.stdout:1/99: truncate d8/db/ff 781377 0 2024-03-18T14:50:45.383 INFO:tasks.workunit.client.0.smithi012.stdout:5/89: unlink d3/f14 0 2024-03-18T14:50:45.385 INFO:tasks.workunit.client.0.smithi012.stdout:9/168: symlink d3/d9/d2c/l2d 0 2024-03-18T14:50:45.385 INFO:tasks.workunit.client.0.smithi012.stdout:9/169: stat d3/d9/f28 0 2024-03-18T14:50:45.387 INFO:tasks.workunit.client.0.smithi012.stdout:4/68: creat d6/f15 x:0 0 0 2024-03-18T14:50:45.387 INFO:tasks.workunit.client.0.smithi012.stdout:8/102: dwrite d0/f1f [0,4194304] 0 2024-03-18T14:50:45.387 INFO:tasks.workunit.client.0.smithi012.stdout:8/103: dread - d0/f16 zero size 2024-03-18T14:50:45.388 INFO:tasks.workunit.client.0.smithi012.stdout:9/170: creat d3/f2e x:0 0 0 2024-03-18T14:50:45.393 INFO:tasks.workunit.client.0.smithi012.stdout:9/171: fsync d3/f4 0 2024-03-18T14:50:45.393 INFO:tasks.workunit.client.0.smithi012.stdout:9/172: write d3/d1e/f2b [784213,38360] 0 2024-03-18T14:50:45.394 INFO:tasks.workunit.client.0.smithi012.stdout:1/100: dwrite f5 [0,4194304] 0 2024-03-18T14:50:45.395 INFO:tasks.workunit.client.0.smithi012.stdout:8/104: rmdir d0/d18 39 2024-03-18T14:50:45.399 INFO:tasks.workunit.client.0.smithi012.stdout:4/69: rename d6/f15 to d6/da/f16 0 2024-03-18T14:50:45.399 INFO:tasks.workunit.client.0.smithi012.stdout:4/70: chown c4 77 1 2024-03-18T14:50:45.401 INFO:tasks.workunit.client.0.smithi012.stdout:1/101: rename c7 to d8/db/d10/d16/d1c/c21 0 2024-03-18T14:50:45.402 INFO:tasks.workunit.client.0.smithi012.stdout:4/71: mknod d6/d14/c17 0 2024-03-18T14:50:45.404 INFO:tasks.workunit.client.0.smithi012.stdout:9/173: link d3/l1b d3/l2f 0 2024-03-18T14:50:45.405 INFO:tasks.workunit.client.0.smithi012.stdout:1/102: mknod d8/db/d10/d16/c22 0 2024-03-18T14:50:45.406 INFO:tasks.workunit.client.0.smithi012.stdout:4/72: creat d6/da/f18 x:0 0 0 2024-03-18T14:50:45.407 INFO:tasks.workunit.client.0.smithi012.stdout:9/174: creat d3/d1e/d21/f30 x:0 0 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/103: creat d8/db/d10/d16/d1c/f23 x:0 0 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/104: readlink d8/db/d10/l18 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/105: fdatasync d8/db/d10/d16/d1c/f23 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/106: getdents d8/d1e 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/107: fdatasync d8/db/f15 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/108: write f5 [5224502,52692] 0 2024-03-18T14:50:45.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/109: stat d8/db/d10/l18 0 2024-03-18T14:50:45.411 INFO:tasks.workunit.client.0.smithi012.stdout:1/110: symlink d8/l24 0 2024-03-18T14:50:45.411 INFO:tasks.workunit.client.0.smithi012.stdout:1/111: truncate d8/f1a 427584 0 2024-03-18T14:50:45.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/105: dwrite d0/d15/f1a [0,4194304] 0 2024-03-18T14:50:45.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/106: write d0/d18/f1d [766584,69128] 0 2024-03-18T14:50:45.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/107: write d0/f10 [689181,66698] 0 2024-03-18T14:50:45.413 INFO:tasks.workunit.client.0.smithi012.stdout:9/175: rename d3/d9/f28 to d3/d20/f31 0 2024-03-18T14:50:45.421 INFO:tasks.workunit.client.0.smithi012.stdout:1/112: dread d8/db/f15 [0,4194304] 0 2024-03-18T14:50:45.424 INFO:tasks.workunit.client.0.smithi012.stdout:8/108: write d0/d15/f1a [849898,55155] 0 2024-03-18T14:50:45.426 INFO:tasks.workunit.client.0.smithi012.stdout:8/109: mknod d0/d18/c23 0 2024-03-18T14:50:45.426 INFO:tasks.workunit.client.0.smithi012.stdout:8/110: creat d0/d12/f24 x:0 0 0 2024-03-18T14:50:45.426 INFO:tasks.workunit.client.0.smithi012.stdout:8/111: dread - d0/ff zero size 2024-03-18T14:50:45.427 INFO:tasks.workunit.client.0.smithi012.stdout:8/112: dread - d0/d12/f24 zero size 2024-03-18T14:50:45.427 INFO:tasks.workunit.client.0.smithi012.stdout:8/113: dread - d0/d12/f24 zero size 2024-03-18T14:50:45.429 INFO:tasks.workunit.client.0.smithi012.stdout:8/114: rename d0/f5 to d0/d18/f25 0 2024-03-18T14:50:45.430 INFO:tasks.workunit.client.0.smithi012.stdout:9/176: dwrite d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:45.430 INFO:tasks.workunit.client.0.smithi012.stdout:8/115: symlink d0/d18/d19/l26 0 2024-03-18T14:50:45.432 INFO:tasks.workunit.client.0.smithi012.stdout:1/113: dread d8/db/d10/f13 [0,4194304] 0 2024-03-18T14:50:45.432 INFO:tasks.workunit.client.0.smithi012.stdout:1/114: truncate f5 5401879 0 2024-03-18T14:50:45.434 INFO:tasks.workunit.client.0.smithi012.stdout:9/177: truncate d3/f16 2745469 0 2024-03-18T14:50:45.436 INFO:tasks.workunit.client.0.smithi012.stdout:8/116: rename d0/f11 to d0/d18/d19/d1b/f27 0 2024-03-18T14:50:45.436 INFO:tasks.workunit.client.0.smithi012.stdout:8/117: truncate d0/d12/f24 103845 0 2024-03-18T14:50:45.442 INFO:tasks.workunit.client.0.smithi012.stdout:9/178: read d3/f12 [66017,76986] 0 2024-03-18T14:50:45.442 INFO:tasks.workunit.client.0.smithi012.stdout:9/179: dread - d3/d9/f1c zero size 2024-03-18T14:50:45.446 INFO:tasks.workunit.client.0.smithi012.stdout:9/180: link d3/f6 d3/d9/f32 0 2024-03-18T14:50:45.453 INFO:tasks.workunit.client.0.smithi012.stdout:8/118: dwrite d0/ff [0,4194304] 0 2024-03-18T14:50:45.453 INFO:tasks.workunit.client.0.smithi012.stdout:8/119: creat d0/d15/f28 x:0 0 0 2024-03-18T14:50:45.453 INFO:tasks.workunit.client.0.smithi012.stdout:8/120: write d0/d12/f20 [80932,40627] 0 2024-03-18T14:50:45.522 INFO:tasks.workunit.client.0.smithi012.stdout:4/73: rename d6/d14 to d6/d19 0 2024-03-18T14:50:45.523 INFO:tasks.workunit.client.0.smithi012.stdout:8/121: rename d0/f1f to d0/d18/d19/f29 0 2024-03-18T14:50:45.526 INFO:tasks.workunit.client.0.smithi012.stdout:8/122: mkdir d0/d18/d19/d1b/d2a 0 2024-03-18T14:50:45.528 INFO:tasks.workunit.client.0.smithi012.stdout:8/123: mknod d0/d18/d19/d1b/c2b 0 2024-03-18T14:50:45.529 INFO:tasks.workunit.client.0.smithi012.stdout:8/124: mkdir d0/d12/d2c 0 2024-03-18T14:50:45.530 INFO:tasks.workunit.client.0.smithi012.stdout:8/125: rename d0/fe to d0/d12/f2d 0 2024-03-18T14:50:45.544 INFO:tasks.workunit.client.1.smithi100.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-18T14:50:45.549 INFO:tasks.workunit.client.0.smithi012.stdout:3/90: sync 2024-03-18T14:50:45.549 INFO:tasks.workunit.client.0.smithi012.stdout:7/56: sync 2024-03-18T14:50:45.551 INFO:tasks.workunit.client.0.smithi012.stdout:7/57: link dd/d13/l18 dd/l1b 0 2024-03-18T14:50:45.551 INFO:tasks.workunit.client.0.smithi012.stdout:7/58: creat dd/d13/f1c x:0 0 0 2024-03-18T14:50:45.551 INFO:tasks.workunit.client.0.smithi012.stdout:3/91: symlink dc/l20 0 2024-03-18T14:50:45.583 INFO:tasks.workunit.client.0.smithi012.stdout:4/74: dwrite d6/da/f13 [0,4194304] 0 2024-03-18T14:50:45.594 INFO:tasks.workunit.client.0.smithi012.stdout:4/75: write d6/da/f13 [2127621,14495] 0 2024-03-18T14:50:45.596 INFO:tasks.workunit.client.0.smithi012.stdout:4/76: mkdir d6/d1a 0 2024-03-18T14:50:45.598 INFO:tasks.workunit.client.0.smithi012.stdout:3/92: dwrite dc/f11 [0,4194304] 0 2024-03-18T14:50:45.598 INFO:tasks.workunit.client.0.smithi012.stdout:3/93: stat dc/l20 0 2024-03-18T14:50:45.599 INFO:tasks.workunit.client.0.smithi012.stdout:3/94: mknod dc/c21 0 2024-03-18T14:50:45.602 INFO:tasks.workunit.client.0.smithi012.stdout:8/126: dwrite d0/d15/f28 [0,4194304] 0 2024-03-18T14:50:45.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/59: dwrite dd/d13/f19 [0,4194304] 0 2024-03-18T14:50:45.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/60: write dd/f10 [727004,116502] 0 2024-03-18T14:50:45.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/61: chown dd/d13/c1a 310153434 1 2024-03-18T14:50:45.604 INFO:tasks.workunit.client.0.smithi012.stdout:8/127: rmdir d0/d18/d19 39 2024-03-18T14:50:45.605 INFO:tasks.workunit.client.0.smithi012.stdout:9/181: dwrite d3/f16 [0,4194304] 0 2024-03-18T14:50:45.605 INFO:tasks.workunit.client.0.smithi012.stdout:9/182: fsync d3/f6 0 2024-03-18T14:50:45.606 INFO:tasks.workunit.client.0.smithi012.stdout:7/62: rename dd/d13/f19 to dd/f1d 0 2024-03-18T14:50:45.608 INFO:tasks.workunit.client.0.smithi012.stdout:8/128: unlink d0/fb 0 2024-03-18T14:50:45.609 INFO:tasks.workunit.client.0.smithi012.stdout:3/95: write fb [3146891,8162] 0 2024-03-18T14:50:45.611 INFO:tasks.workunit.client.0.smithi012.stdout:9/183: creat d3/d9/f33 x:0 0 0 2024-03-18T14:50:45.611 INFO:tasks.workunit.client.0.smithi012.stdout:9/184: fsync d3/d9/f17 0 2024-03-18T14:50:45.611 INFO:tasks.workunit.client.0.smithi012.stdout:9/185: read d3/f4 [174661,43541] 0 2024-03-18T14:50:45.612 INFO:tasks.workunit.client.0.smithi012.stdout:4/77: dwrite f1 [0,4194304] 0 2024-03-18T14:50:45.613 INFO:tasks.workunit.client.0.smithi012.stdout:7/63: unlink dd/f11 0 2024-03-18T14:50:45.613 INFO:tasks.workunit.client.0.smithi012.stdout:7/64: chown fa 0 1 2024-03-18T14:50:45.613 INFO:tasks.workunit.client.0.smithi012.stdout:7/65: creat dd/f1e x:0 0 0 2024-03-18T14:50:45.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/129: symlink d0/d18/l2e 0 2024-03-18T14:50:45.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/130: readlink d0/d18/d19/l26 0 2024-03-18T14:50:45.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/131: chown d0/d18/d19/d1b/c2b 2706 1 2024-03-18T14:50:45.616 INFO:tasks.workunit.client.0.smithi012.stdout:3/96: truncate f3 1449938 0 2024-03-18T14:50:45.619 INFO:tasks.workunit.client.0.smithi012.stdout:9/186: dread d3/f16 [0,4194304] 0 2024-03-18T14:50:45.619 INFO:tasks.workunit.client.0.smithi012.stdout:4/78: creat d6/d1a/f1b x:0 0 0 2024-03-18T14:50:45.622 INFO:tasks.workunit.client.0.smithi012.stdout:7/66: symlink dd/l1f 0 2024-03-18T14:50:45.622 INFO:tasks.workunit.client.0.smithi012.stdout:7/67: truncate f3 1263692 0 2024-03-18T14:50:45.622 INFO:tasks.workunit.client.0.smithi012.stdout:7/68: fsync dd/d13/f1c 0 2024-03-18T14:50:45.623 INFO:tasks.workunit.client.0.smithi012.stdout:7/69: dread f1 [0,4194304] 0 2024-03-18T14:50:45.623 INFO:tasks.workunit.client.0.smithi012.stdout:7/70: truncate f1 313435 0 2024-03-18T14:50:45.623 INFO:tasks.workunit.client.0.smithi012.stdout:8/132: link d0/d12/f1e d0/d12/d2c/f2f 0 2024-03-18T14:50:45.626 INFO:tasks.workunit.client.0.smithi012.stdout:3/97: creat dc/f22 x:0 0 0 2024-03-18T14:50:45.626 INFO:tasks.workunit.client.0.smithi012.stdout:3/98: fsync dc/f11 0 2024-03-18T14:50:45.627 INFO:tasks.workunit.client.0.smithi012.stdout:9/187: stat d3/l14 0 2024-03-18T14:50:45.627 INFO:tasks.workunit.client.0.smithi012.stdout:9/188: chown d3/f6 14768 1 2024-03-18T14:50:45.629 INFO:tasks.workunit.client.0.smithi012.stdout:8/133: creat d0/d18/d19/d1b/f30 x:0 0 0 2024-03-18T14:50:45.629 INFO:tasks.workunit.client.0.smithi012.stdout:8/134: chown d0/d18/d19/f29 32700579 1 2024-03-18T14:50:45.629 INFO:tasks.workunit.client.0.smithi012.stdout:8/135: readlink d0/d18/d19/l26 0 2024-03-18T14:50:45.630 INFO:tasks.workunit.client.0.smithi012.stdout:9/189: mknod d3/d9/d19/c34 0 2024-03-18T14:50:45.630 INFO:tasks.workunit.client.0.smithi012.stdout:8/136: dread d0/d12/f20 [0,4194304] 0 2024-03-18T14:50:45.631 INFO:tasks.workunit.client.0.smithi012.stdout:8/137: fsync d0/d18/d19/d1b/f27 0 2024-03-18T14:50:45.633 INFO:tasks.workunit.client.0.smithi012.stdout:8/138: mknod d0/d15/c31 0 2024-03-18T14:50:45.634 INFO:tasks.workunit.client.0.smithi012.stdout:0/133: sync 2024-03-18T14:50:45.634 INFO:tasks.workunit.client.0.smithi012.stdout:6/111: sync 2024-03-18T14:50:45.639 INFO:tasks.workunit.client.0.smithi012.stdout:0/134: symlink d1/d20/d23/d26/l39 0 2024-03-18T14:50:45.640 INFO:tasks.workunit.client.0.smithi012.stdout:0/135: dread - d1/d16/d19/f37 zero size 2024-03-18T14:50:45.640 INFO:tasks.workunit.client.0.smithi012.stdout:0/136: chown d1/d16/f17 13794925 1 2024-03-18T14:50:45.640 INFO:tasks.workunit.client.0.smithi012.stdout:0/137: readlink d1/d16/d18/l11 0 2024-03-18T14:50:45.640 INFO:tasks.workunit.client.0.smithi012.stdout:0/138: symlink d1/d20/d23/l3a 0 2024-03-18T14:50:45.641 INFO:tasks.workunit.client.0.smithi012.stdout:0/139: readlink d1/d16/d31/l2f 0 2024-03-18T14:50:45.641 INFO:tasks.workunit.client.0.smithi012.stdout:0/140: fsync d1/d16/f17 0 2024-03-18T14:50:45.644 INFO:tasks.workunit.client.0.smithi012.stdout:0/141: rename d1/d16/d18/d22 to d1/d20/d23/d3b 0 2024-03-18T14:50:45.645 INFO:tasks.workunit.client.0.smithi012.stdout:3/99: read f6 [3409772,33446] 0 2024-03-18T14:50:45.649 INFO:tasks.workunit.client.0.smithi012.stdout:2/101: sync 2024-03-18T14:50:45.649 INFO:tasks.workunit.client.0.smithi012.stdout:2/102: chown d0/lc 813 1 2024-03-18T14:50:45.649 INFO:tasks.workunit.client.0.smithi012.stdout:2/103: read - d0/f21 zero size 2024-03-18T14:50:45.649 INFO:tasks.workunit.client.0.smithi012.stdout:4/79: dwrite f1 [0,4194304] 0 2024-03-18T14:50:45.649 INFO:tasks.workunit.client.0.smithi012.stdout:6/112: dread d17/f1b [0,4194304] 0 2024-03-18T14:50:45.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/104: symlink d0/d10/l23 0 2024-03-18T14:50:45.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/105: fdatasync d0/f8 0 2024-03-18T14:50:45.650 INFO:tasks.workunit.client.0.smithi012.stdout:6/113: mknod d17/d18/c1c 0 2024-03-18T14:50:45.651 INFO:tasks.workunit.client.0.smithi012.stdout:6/114: readlink l3 0 2024-03-18T14:50:45.651 INFO:tasks.workunit.client.0.smithi012.stdout:7/71: dwrite f3 [0,4194304] 0 2024-03-18T14:50:45.651 INFO:tasks.workunit.client.0.smithi012.stdout:7/72: fsync dd/f12 0 2024-03-18T14:50:45.652 INFO:tasks.workunit.client.0.smithi012.stdout:4/80: creat d6/d19/f1c x:0 0 0 2024-03-18T14:50:45.652 INFO:tasks.workunit.client.0.smithi012.stdout:6/115: mknod d17/c1d 0 2024-03-18T14:50:45.655 INFO:tasks.workunit.client.0.smithi012.stdout:6/116: write d17/f1b [610550,106790] 0 2024-03-18T14:50:45.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/73: mknod dd/c20 0 2024-03-18T14:50:45.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/74: truncate f3 4969144 0 2024-03-18T14:50:45.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/75: fdatasync f1 0 2024-03-18T14:50:45.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/76: write dd/d13/f15 [707212,52451] 0 2024-03-18T14:50:45.658 INFO:tasks.workunit.client.0.smithi012.stdout:4/81: readlink d6/l7 0 2024-03-18T14:50:45.658 INFO:tasks.workunit.client.0.smithi012.stdout:4/82: creat d6/d19/f1d x:0 0 0 2024-03-18T14:50:45.658 INFO:tasks.workunit.client.0.smithi012.stdout:3/100: dread dc/f14 [0,4194304] 0 2024-03-18T14:50:45.659 INFO:tasks.workunit.client.0.smithi012.stdout:7/77: mknod dd/d13/c21 0 2024-03-18T14:50:45.659 INFO:tasks.workunit.client.0.smithi012.stdout:2/106: read d0/fa [867019,48084] 0 2024-03-18T14:50:45.662 INFO:tasks.workunit.client.0.smithi012.stdout:4/83: mknod d6/d19/c1e 0 2024-03-18T14:50:45.662 INFO:tasks.workunit.client.0.smithi012.stdout:4/84: write d6/da/f18 [368529,94753] 0 2024-03-18T14:50:45.663 INFO:tasks.workunit.client.0.smithi012.stdout:4/85: write d6/d19/f1d [538019,10041] 0 2024-03-18T14:50:45.663 INFO:tasks.workunit.client.0.smithi012.stdout:4/86: creat d6/d1a/f1f x:0 0 0 2024-03-18T14:50:45.664 INFO:tasks.workunit.client.0.smithi012.stdout:2/107: dread d0/f8 [0,4194304] 0 2024-03-18T14:50:45.664 INFO:tasks.workunit.client.0.smithi012.stdout:2/108: fsync d0/f5 0 2024-03-18T14:50:45.664 INFO:tasks.workunit.client.0.smithi012.stdout:6/117: dread fe [0,4194304] 0 2024-03-18T14:50:45.666 INFO:tasks.workunit.client.0.smithi012.stdout:9/190: dwrite d3/d9/f33 [0,4194304] 0 2024-03-18T14:50:45.667 INFO:tasks.workunit.client.0.smithi012.stdout:9/191: write d3/d9/f10 [1998434,83040] 0 2024-03-18T14:50:45.667 INFO:tasks.workunit.client.0.smithi012.stdout:7/78: link f1 dd/f22 0 2024-03-18T14:50:45.667 INFO:tasks.workunit.client.0.smithi012.stdout:8/139: dwrite d0/ff [0,4194304] 0 2024-03-18T14:50:45.668 INFO:tasks.workunit.client.0.smithi012.stdout:4/87: dread d6/d19/f1d [0,4194304] 0 2024-03-18T14:50:45.669 INFO:tasks.workunit.client.0.smithi012.stdout:3/101: mknod dc/c23 0 2024-03-18T14:50:45.671 INFO:tasks.workunit.client.0.smithi012.stdout:6/118: creat d17/d18/f1e x:0 0 0 2024-03-18T14:50:45.671 INFO:tasks.workunit.client.0.smithi012.stdout:6/119: creat d17/d18/f1f x:0 0 0 2024-03-18T14:50:45.671 INFO:tasks.workunit.client.0.smithi012.stdout:6/120: chown l3 1 1 2024-03-18T14:50:45.679 INFO:tasks.workunit.client.0.smithi012.stdout:9/192: mkdir d3/d1e/d21/d35 0 2024-03-18T14:50:45.679 INFO:tasks.workunit.client.0.smithi012.stdout:9/193: read d3/f6 [331701,52800] 0 2024-03-18T14:50:45.682 INFO:tasks.workunit.client.0.smithi012.stdout:8/140: mknod d0/c32 0 2024-03-18T14:50:45.683 INFO:tasks.workunit.client.0.smithi012.stdout:2/109: dwrite d0/f21 [0,4194304] 0 2024-03-18T14:50:45.683 INFO:tasks.workunit.client.0.smithi012.stdout:4/88: creat d6/f20 x:0 0 0 2024-03-18T14:50:45.684 INFO:tasks.workunit.client.0.smithi012.stdout:3/102: creat dc/f24 x:0 0 0 2024-03-18T14:50:45.688 INFO:tasks.workunit.client.0.smithi012.stdout:6/121: dwrite d17/d18/f1f [0,4194304] 0 2024-03-18T14:50:45.688 INFO:tasks.workunit.client.0.smithi012.stdout:6/122: fsync f11 0 2024-03-18T14:50:45.688 INFO:tasks.workunit.client.0.smithi012.stdout:9/194: truncate d3/d9/f13 3084918 0 2024-03-18T14:50:45.688 INFO:tasks.workunit.client.0.smithi012.stdout:9/195: creat d3/d1e/d21/f36 x:0 0 0 2024-03-18T14:50:45.688 INFO:tasks.workunit.client.0.smithi012.stdout:9/196: fsync d3/d9/f29 0 2024-03-18T14:50:45.689 INFO:tasks.workunit.client.0.smithi012.stdout:7/79: rmdir dd/d13 39 2024-03-18T14:50:45.689 INFO:tasks.workunit.client.0.smithi012.stdout:4/89: dread d6/da/f18 [0,4194304] 0 2024-03-18T14:50:45.691 INFO:tasks.workunit.client.0.smithi012.stdout:8/141: unlink d0/ff 0 2024-03-18T14:50:45.692 INFO:tasks.workunit.client.0.smithi012.stdout:8/142: chown d0/d18/f25 2 1 2024-03-18T14:50:45.692 INFO:tasks.workunit.client.0.smithi012.stdout:3/103: dread fb [0,4194304] 0 2024-03-18T14:50:45.695 INFO:tasks.workunit.client.0.smithi012.stdout:3/104: dread f3 [0,4194304] 0 2024-03-18T14:50:45.697 INFO:tasks.workunit.client.0.smithi012.stdout:6/123: creat d17/d19/f20 x:0 0 0 2024-03-18T14:50:45.699 INFO:tasks.workunit.client.0.smithi012.stdout:9/197: readlink d3/le 0 2024-03-18T14:50:45.700 INFO:tasks.workunit.client.0.smithi012.stdout:1/115: sync 2024-03-18T14:50:45.700 INFO:tasks.workunit.client.0.smithi012.stdout:5/90: sync 2024-03-18T14:50:45.700 INFO:tasks.workunit.client.0.smithi012.stdout:5/91: dread - d3/d8/f1a zero size 2024-03-18T14:50:45.701 INFO:tasks.workunit.client.0.smithi012.stdout:8/143: mkdir d0/d12/d33 0 2024-03-18T14:50:45.701 INFO:tasks.workunit.client.0.smithi012.stdout:8/144: write d0/fc [349372,6814] 0 2024-03-18T14:50:45.701 INFO:tasks.workunit.client.0.smithi012.stdout:8/145: chown d0/d15/c31 12 1 2024-03-18T14:50:45.702 INFO:tasks.workunit.client.0.smithi012.stdout:2/110: rmdir d0/d10/d18 39 2024-03-18T14:50:45.702 INFO:tasks.workunit.client.0.smithi012.stdout:2/111: truncate d0/f1c 549328 0 2024-03-18T14:50:45.702 INFO:tasks.workunit.client.0.smithi012.stdout:1/116: dread d8/fc [0,4194304] 0 2024-03-18T14:50:45.702 INFO:tasks.workunit.client.0.smithi012.stdout:1/117: write d8/fa [2608667,29115] 0 2024-03-18T14:50:45.702 INFO:tasks.workunit.client.0.smithi012.stdout:1/118: creat d8/db/f25 x:0 0 0 2024-03-18T14:50:45.703 INFO:tasks.workunit.client.0.smithi012.stdout:3/105: stat dc/c10 0 2024-03-18T14:50:45.707 INFO:tasks.workunit.client.0.smithi012.stdout:6/124: link f5 d17/d18/f21 0 2024-03-18T14:50:45.707 INFO:tasks.workunit.client.0.smithi012.stdout:6/125: fsync f8 0 2024-03-18T14:50:45.708 INFO:tasks.workunit.client.0.smithi012.stdout:7/80: dwrite dd/d13/f1c [0,4194304] 0 2024-03-18T14:50:45.708 INFO:tasks.workunit.client.0.smithi012.stdout:7/81: write dd/f1e [145189,113590] 0 2024-03-18T14:50:45.709 INFO:tasks.workunit.client.0.smithi012.stdout:4/90: rename d6/d1a/f1f to d6/f21 0 2024-03-18T14:50:45.713 INFO:tasks.workunit.client.0.smithi012.stdout:1/119: dread d8/fa [0,4194304] 0 2024-03-18T14:50:45.713 INFO:tasks.workunit.client.0.smithi012.stdout:1/120: chown d8/db/c12 26 1 2024-03-18T14:50:45.714 INFO:tasks.workunit.client.0.smithi012.stdout:2/112: truncate d0/f6 1869639 0 2024-03-18T14:50:45.714 INFO:tasks.workunit.client.0.smithi012.stdout:2/113: chown d0/cf 6396 1 2024-03-18T14:50:45.717 INFO:tasks.workunit.client.0.smithi012.stdout:3/106: creat dc/f25 x:0 0 0 2024-03-18T14:50:45.717 INFO:tasks.workunit.client.0.smithi012.stdout:3/107: write dc/f1f [409177,118401] 0 2024-03-18T14:50:45.720 INFO:tasks.workunit.client.0.smithi012.stdout:7/82: mkdir dd/d13/d23 0 2024-03-18T14:50:45.720 INFO:tasks.workunit.client.0.smithi012.stdout:7/83: readlink dd/l1f 0 2024-03-18T14:50:45.721 INFO:tasks.workunit.client.0.smithi012.stdout:1/121: symlink d8/db/l26 0 2024-03-18T14:50:45.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/108: creat dc/f26 x:0 0 0 2024-03-18T14:50:45.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/109: dread - dc/f26 zero size 2024-03-18T14:50:45.723 INFO:tasks.workunit.client.0.smithi012.stdout:4/91: symlink d6/l22 0 2024-03-18T14:50:45.723 INFO:tasks.workunit.client.0.smithi012.stdout:8/146: dwrite d0/f7 [0,4194304] 0 2024-03-18T14:50:45.723 INFO:tasks.workunit.client.0.smithi012.stdout:8/147: fsync d0/f16 0 2024-03-18T14:50:45.724 INFO:tasks.workunit.client.0.smithi012.stdout:6/126: dwrite d17/d18/f1e [0,4194304] 0 2024-03-18T14:50:45.725 INFO:tasks.workunit.client.0.smithi012.stdout:5/92: rename d3/d8 to d3/d1e 0 2024-03-18T14:50:45.725 INFO:tasks.workunit.client.0.smithi012.stdout:5/93: creat d3/f1f x:0 0 0 2024-03-18T14:50:45.725 INFO:tasks.workunit.client.0.smithi012.stdout:1/122: mkdir d8/d1e/d27 0 2024-03-18T14:50:45.727 INFO:tasks.workunit.client.0.smithi012.stdout:1/123: dread d8/f1a [0,4194304] 0 2024-03-18T14:50:45.727 INFO:tasks.workunit.client.0.smithi012.stdout:1/124: write d8/db/d10/f11 [1749024,62120] 0 2024-03-18T14:50:45.727 INFO:tasks.workunit.client.0.smithi012.stdout:1/125: truncate d8/db/f15 4556569 0 2024-03-18T14:50:45.727 INFO:tasks.workunit.client.0.smithi012.stdout:4/92: truncate d6/fe 705783 0 2024-03-18T14:50:45.729 INFO:tasks.workunit.client.0.smithi012.stdout:8/148: symlink d0/l34 0 2024-03-18T14:50:45.729 INFO:tasks.workunit.client.0.smithi012.stdout:8/149: write d0/d18/d19/d1b/f30 [1026134,21205] 0 2024-03-18T14:50:45.729 INFO:tasks.workunit.client.0.smithi012.stdout:8/150: creat d0/f35 x:0 0 0 2024-03-18T14:50:45.729 INFO:tasks.workunit.client.0.smithi012.stdout:8/151: fdatasync d0/d18/f25 0 2024-03-18T14:50:45.730 INFO:tasks.workunit.client.0.smithi012.stdout:8/152: fdatasync d0/d18/d19/d1b/f27 0 2024-03-18T14:50:45.730 INFO:tasks.workunit.client.0.smithi012.stdout:8/153: write d0/fc [1329425,97875] 0 2024-03-18T14:50:45.730 INFO:tasks.workunit.client.0.smithi012.stdout:7/84: write f3 [1991174,105114] 0 2024-03-18T14:50:45.735 INFO:tasks.workunit.client.0.smithi012.stdout:1/126: dread f5 [0,4194304] 0 2024-03-18T14:50:45.735 INFO:tasks.workunit.client.0.smithi012.stdout:5/94: mknod d3/d1e/c20 0 2024-03-18T14:50:45.738 INFO:tasks.workunit.client.0.smithi012.stdout:3/110: rename fb to dc/f27 0 2024-03-18T14:50:45.742 INFO:tasks.workunit.client.0.smithi012.stdout:6/127: dwrite f16 [0,4194304] 0 2024-03-18T14:50:45.745 INFO:tasks.workunit.client.0.smithi012.stdout:1/127: creat d8/d1e/d27/f28 x:0 0 0 2024-03-18T14:50:45.745 INFO:tasks.workunit.client.0.smithi012.stdout:1/128: fsync d8/db/d10/f13 0 2024-03-18T14:50:45.749 INFO:tasks.workunit.client.0.smithi012.stdout:7/85: getdents dd 0 2024-03-18T14:50:45.749 INFO:tasks.workunit.client.0.smithi012.stdout:7/86: fsync dd/f12 0 2024-03-18T14:50:45.749 INFO:tasks.workunit.client.0.smithi012.stdout:7/87: dread - dd/f12 zero size 2024-03-18T14:50:45.754 INFO:tasks.workunit.client.0.smithi012.stdout:1/129: rename d8/db/l19 to d8/db/d10/l29 0 2024-03-18T14:50:45.755 INFO:tasks.workunit.client.0.smithi012.stdout:1/130: creat d8/db/d10/d16/d1c/f2a x:0 0 0 2024-03-18T14:50:45.794 INFO:tasks.workunit.client.0.smithi012.stdout:2/114: dwrite d0/f6 [0,4194304] 0 2024-03-18T14:50:45.794 INFO:tasks.workunit.client.0.smithi012.stdout:2/115: write d0/f21 [4634294,122399] 0 2024-03-18T14:50:45.796 INFO:tasks.workunit.client.0.smithi012.stdout:2/116: creat d0/d10/d18/f24 x:0 0 0 2024-03-18T14:50:45.796 INFO:tasks.workunit.client.0.smithi012.stdout:2/117: readlink d0/d10/l15 0 2024-03-18T14:50:45.797 INFO:tasks.workunit.client.0.smithi012.stdout:2/118: mknod d0/d10/d18/c25 0 2024-03-18T14:50:45.798 INFO:tasks.workunit.client.0.smithi012.stdout:2/119: chown d0/d10/l23 108 1 2024-03-18T14:50:45.798 INFO:tasks.workunit.client.0.smithi012.stdout:2/120: fdatasync d0/fa 0 2024-03-18T14:50:45.801 INFO:tasks.workunit.client.0.smithi012.stdout:3/111: write dc/f11 [256485,9599] 0 2024-03-18T14:50:45.801 INFO:tasks.workunit.client.0.smithi012.stdout:3/112: dread - dc/f26 zero size 2024-03-18T14:50:45.802 INFO:tasks.workunit.client.0.smithi012.stdout:3/113: symlink dc/l28 0 2024-03-18T14:50:45.803 INFO:tasks.workunit.client.0.smithi012.stdout:3/114: chown dc/f14 84571693 1 2024-03-18T14:50:45.803 INFO:tasks.workunit.client.0.smithi012.stdout:3/115: chown dc/l20 62 1 2024-03-18T14:50:45.803 INFO:tasks.workunit.client.0.smithi012.stdout:5/95: dwrite d3/f1f [0,4194304] 0 2024-03-18T14:50:45.807 INFO:tasks.workunit.client.0.smithi012.stdout:7/88: write dd/f10 [605597,842] 0 2024-03-18T14:50:45.811 INFO:tasks.workunit.client.0.smithi012.stdout:5/96: mkdir d3/d15/d21 0 2024-03-18T14:50:45.811 INFO:tasks.workunit.client.0.smithi012.stdout:5/97: creat d3/f22 x:0 0 0 2024-03-18T14:50:45.812 INFO:tasks.workunit.client.0.smithi012.stdout:7/89: mknod dd/d13/d23/c24 0 2024-03-18T14:50:45.812 INFO:tasks.workunit.client.0.smithi012.stdout:7/90: stat dd/d13/d23/c24 0 2024-03-18T14:50:45.848 INFO:tasks.workunit.client.0.smithi012.stdout:6/128: dwrite f5 [0,4194304] 0 2024-03-18T14:50:45.848 INFO:tasks.workunit.client.0.smithi012.stdout:6/129: chown c10 15 1 2024-03-18T14:50:45.848 INFO:tasks.workunit.client.0.smithi012.stdout:5/98: dwrite d3/d1e/fb [4194304,4194304] 0 2024-03-18T14:50:45.848 INFO:tasks.workunit.client.0.smithi012.stdout:5/99: truncate d3/f22 677412 0 2024-03-18T14:50:45.848 INFO:tasks.workunit.client.0.smithi012.stdout:4/93: dwrite d6/f9 [0,4194304] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:1/131: dwrite d8/db/fd [0,4194304] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:1/132: creat d8/db/d10/d16/d1c/f2b x:0 0 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:2/121: dwrite d0/d10/d18/f1b [0,4194304] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:2/122: write d0/d10/d18/f1f [451706,129787] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:3/116: dwrite dc/f1f [0,4194304] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:7/91: dwrite dd/f12 [0,4194304] 0 2024-03-18T14:50:45.849 INFO:tasks.workunit.client.0.smithi012.stdout:7/92: truncate dd/f10 1715364 0 2024-03-18T14:50:45.850 INFO:tasks.workunit.client.0.smithi012.stdout:5/100: creat d3/d15/d21/f23 x:0 0 0 2024-03-18T14:50:45.857 INFO:tasks.workunit.client.0.smithi012.stdout:5/101: dread d3/d1e/fb [0,4194304] 0 2024-03-18T14:50:45.860 INFO:tasks.workunit.client.0.smithi012.stdout:1/133: truncate d8/db/d10/f13 3118446 0 2024-03-18T14:50:45.862 INFO:tasks.workunit.client.0.smithi012.stdout:2/123: stat d0/l3 0 2024-03-18T14:50:45.865 INFO:tasks.workunit.client.0.smithi012.stdout:6/130: dwrite f16 [0,4194304] 0 2024-03-18T14:50:45.865 INFO:tasks.workunit.client.0.smithi012.stdout:6/131: creat d17/f22 x:0 0 0 2024-03-18T14:50:45.865 INFO:tasks.workunit.client.0.smithi012.stdout:6/132: creat d17/d18/f23 x:0 0 0 2024-03-18T14:50:45.867 INFO:tasks.workunit.client.0.smithi012.stdout:7/93: mknod dd/c25 0 2024-03-18T14:50:45.868 INFO:tasks.workunit.client.0.smithi012.stdout:4/94: getdents d6/d1a 0 2024-03-18T14:50:45.868 INFO:tasks.workunit.client.0.smithi012.stdout:4/95: chown c4 2 1 2024-03-18T14:50:45.870 INFO:tasks.workunit.client.0.smithi012.stdout:5/102: truncate d3/ff 1703325 0 2024-03-18T14:50:45.872 INFO:tasks.workunit.client.0.smithi012.stdout:1/134: truncate d8/f1a 292177 0 2024-03-18T14:50:45.872 INFO:tasks.workunit.client.0.smithi012.stdout:1/135: creat d8/db/d10/d16/d1c/f2c x:0 0 0 2024-03-18T14:50:45.873 INFO:tasks.workunit.client.0.smithi012.stdout:1/136: creat d8/db/f2d x:0 0 0 2024-03-18T14:50:45.873 INFO:tasks.workunit.client.0.smithi012.stdout:5/103: write d3/fc [1714938,51317] 0 2024-03-18T14:50:45.873 INFO:tasks.workunit.client.0.smithi012.stdout:5/104: chown d3/d1e/l17 1 1 2024-03-18T14:50:45.873 INFO:tasks.workunit.client.0.smithi012.stdout:2/124: truncate d0/f5 2719469 0 2024-03-18T14:50:45.874 INFO:tasks.workunit.client.0.smithi012.stdout:0/142: sync 2024-03-18T14:50:45.875 INFO:tasks.workunit.client.0.smithi012.stdout:3/117: rmdir dc 39 2024-03-18T14:50:45.880 INFO:tasks.workunit.client.0.smithi012.stdout:6/133: dwrite f9 [0,4194304] 0 2024-03-18T14:50:45.882 INFO:tasks.workunit.client.0.smithi012.stdout:7/94: mkdir dd/d13/d26 0 2024-03-18T14:50:45.882 INFO:tasks.workunit.client.0.smithi012.stdout:7/95: write f3 [5105546,106001] 0 2024-03-18T14:50:45.883 INFO:tasks.workunit.client.0.smithi012.stdout:4/96: truncate d6/da/f13 3131756 0 2024-03-18T14:50:45.884 INFO:tasks.workunit.client.0.smithi012.stdout:6/134: dread f8 [0,4194304] 0 2024-03-18T14:50:45.888 INFO:tasks.workunit.client.0.smithi012.stdout:0/143: mknod d1/d16/d31/c3c 0 2024-03-18T14:50:45.891 INFO:tasks.workunit.client.0.smithi012.stdout:3/118: unlink dc/f25 0 2024-03-18T14:50:45.891 INFO:tasks.workunit.client.0.smithi012.stdout:3/119: stat dc/f22 0 2024-03-18T14:50:45.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/105: dwrite d3/f1f [4194304,4194304] 0 2024-03-18T14:50:45.894 INFO:tasks.workunit.client.0.smithi012.stdout:5/106: write f2 [4580801,120312] 0 2024-03-18T14:50:45.894 INFO:tasks.workunit.client.0.smithi012.stdout:5/107: creat d3/f24 x:0 0 0 2024-03-18T14:50:45.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/198: sync 2024-03-18T14:50:45.895 INFO:tasks.workunit.client.0.smithi012.stdout:4/97: write d6/da/fd [546963,127038] 0 2024-03-18T14:50:45.896 INFO:tasks.workunit.client.0.smithi012.stdout:8/154: sync 2024-03-18T14:50:45.897 INFO:tasks.workunit.client.0.smithi012.stdout:2/125: dread d0/f21 [0,4194304] 0 2024-03-18T14:50:45.898 INFO:tasks.workunit.client.0.smithi012.stdout:6/135: dread f11 [0,4194304] 0 2024-03-18T14:50:45.898 INFO:tasks.workunit.client.0.smithi012.stdout:7/96: dread dd/f10 [0,4194304] 0 2024-03-18T14:50:45.899 INFO:tasks.workunit.client.0.smithi012.stdout:5/108: write d3/fc [1778053,118650] 0 2024-03-18T14:50:45.902 INFO:tasks.workunit.client.0.smithi012.stdout:6/136: dread f8 [0,4194304] 0 2024-03-18T14:50:45.902 INFO:tasks.workunit.client.0.smithi012.stdout:6/137: creat d17/d19/f24 x:0 0 0 2024-03-18T14:50:45.902 INFO:tasks.workunit.client.0.smithi012.stdout:8/155: mknod d0/d18/d19/d1b/c36 0 2024-03-18T14:50:45.903 INFO:tasks.workunit.client.0.smithi012.stdout:5/109: symlink d3/l25 0 2024-03-18T14:50:45.904 INFO:tasks.workunit.client.0.smithi012.stdout:5/110: write d3/f24 [230236,19012] 0 2024-03-18T14:50:45.905 INFO:tasks.workunit.client.0.smithi012.stdout:9/199: getdents d3/d1e/d21 0 2024-03-18T14:50:45.905 INFO:tasks.workunit.client.0.smithi012.stdout:9/200: write d3/d1e/d21/f30 [395696,7068] 0 2024-03-18T14:50:45.907 INFO:tasks.workunit.client.0.smithi012.stdout:4/98: getdents d6 0 2024-03-18T14:50:45.908 INFO:tasks.workunit.client.0.smithi012.stdout:4/99: dread - d6/ff zero size 2024-03-18T14:50:45.911 INFO:tasks.workunit.client.0.smithi012.stdout:2/126: write d0/f6 [425153,129126] 0 2024-03-18T14:50:45.911 INFO:tasks.workunit.client.0.smithi012.stdout:8/156: creat d0/d12/d2c/f37 x:0 0 0 2024-03-18T14:50:45.912 INFO:tasks.workunit.client.0.smithi012.stdout:8/157: read d0/d12/f2d [916140,76755] 0 2024-03-18T14:50:45.914 INFO:tasks.workunit.client.0.smithi012.stdout:8/158: rename d0/d12/f24 to d0/d15/f38 0 2024-03-18T14:50:45.922 INFO:tasks.workunit.client.0.smithi012.stdout:8/159: dread d0/f10 [0,4194304] 0 2024-03-18T14:50:45.922 INFO:tasks.workunit.client.0.smithi012.stdout:9/201: dread d3/d9/f32 [0,4194304] 0 2024-03-18T14:50:45.922 INFO:tasks.workunit.client.0.smithi012.stdout:9/202: readlink d3/d9/d19/l1d 0 2024-03-18T14:50:45.924 INFO:tasks.workunit.client.0.smithi012.stdout:9/203: symlink d3/d1e/l37 0 2024-03-18T14:50:45.925 INFO:tasks.workunit.client.0.smithi012.stdout:9/204: chown d3/c1f 0 1 2024-03-18T14:50:45.926 INFO:tasks.workunit.client.0.smithi012.stdout:9/205: symlink d3/d1e/d21/d35/l38 0 2024-03-18T14:50:45.927 INFO:tasks.workunit.client.0.smithi012.stdout:9/206: chown d3/d9/d2c 10890 1 2024-03-18T14:50:45.927 INFO:tasks.workunit.client.0.smithi012.stdout:9/207: write d3/d9/fd [4419243,125968] 0 2024-03-18T14:50:45.930 INFO:tasks.workunit.client.0.smithi012.stdout:5/111: write d3/d1e/fb [7532167,10090] 0 2024-03-18T14:50:45.936 INFO:tasks.workunit.client.0.smithi012.stdout:8/160: dread d0/d15/f28 [0,4194304] 0 2024-03-18T14:50:45.937 INFO:tasks.workunit.client.0.smithi012.stdout:8/161: write d0/d18/f25 [871490,104113] 0 2024-03-18T14:50:45.938 INFO:tasks.workunit.client.0.smithi012.stdout:8/162: symlink d0/d18/d19/l39 0 2024-03-18T14:50:45.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/163: creat d0/d18/d19/f3a x:0 0 0 2024-03-18T14:50:45.951 INFO:tasks.workunit.client.0.smithi012.stdout:3/120: dwrite dc/f11 [0,4194304] 0 2024-03-18T14:50:45.951 INFO:tasks.workunit.client.0.smithi012.stdout:0/144: dwrite d1/d16/d19/f37 [0,4194304] 0 2024-03-18T14:50:45.951 INFO:tasks.workunit.client.0.smithi012.stdout:0/145: chown d1/d16/d19 1863775108 1 2024-03-18T14:50:45.951 INFO:tasks.workunit.client.0.smithi012.stdout:0/146: fdatasync d1/f10 0 2024-03-18T14:50:45.952 INFO:tasks.workunit.client.0.smithi012.stdout:1/137: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:50:45.954 INFO:tasks.workunit.client.0.smithi012.stdout:0/147: rmdir d1/d20/d23 39 2024-03-18T14:50:45.957 INFO:tasks.workunit.client.0.smithi012.stdout:0/148: unlink d1/d16/d31/l2f 0 2024-03-18T14:50:45.957 INFO:tasks.workunit.client.0.smithi012.stdout:4/100: dread d6/da/fd [0,4194304] 0 2024-03-18T14:50:45.958 INFO:tasks.workunit.client.0.smithi012.stdout:0/149: mknod d1/d16/d19/c3d 0 2024-03-18T14:50:45.959 INFO:tasks.workunit.client.0.smithi012.stdout:4/101: rename d6/d19/c17 to d6/d19/c23 0 2024-03-18T14:50:45.959 INFO:tasks.workunit.client.0.smithi012.stdout:4/102: truncate d6/f20 696331 0 2024-03-18T14:50:45.961 INFO:tasks.workunit.client.0.smithi012.stdout:6/138: dwrite fe [0,4194304] 0 2024-03-18T14:50:45.961 INFO:tasks.workunit.client.0.smithi012.stdout:6/139: write d17/f22 [66452,7225] 0 2024-03-18T14:50:45.961 INFO:tasks.workunit.client.0.smithi012.stdout:2/127: dwrite d0/f21 [0,4194304] 0 2024-03-18T14:50:45.961 INFO:tasks.workunit.client.0.smithi012.stdout:2/128: truncate d0/d10/d18/f1f 1360558 0 2024-03-18T14:50:45.962 INFO:tasks.workunit.client.0.smithi012.stdout:2/129: write d0/fa [544419,83454] 0 2024-03-18T14:50:45.963 INFO:tasks.workunit.client.0.smithi012.stdout:8/164: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:45.963 INFO:tasks.workunit.client.0.smithi012.stdout:9/208: dwrite d3/f16 [0,4194304] 0 2024-03-18T14:50:45.965 INFO:tasks.workunit.client.0.smithi012.stdout:0/150: dread d1/f9 [0,4194304] 0 2024-03-18T14:50:45.967 INFO:tasks.workunit.client.0.smithi012.stdout:7/97: dwrite dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:45.967 INFO:tasks.workunit.client.0.smithi012.stdout:3/121: dread dc/f14 [0,4194304] 0 2024-03-18T14:50:45.969 INFO:tasks.workunit.client.0.smithi012.stdout:4/103: dread d6/f12 [0,4194304] 0 2024-03-18T14:50:45.969 INFO:tasks.workunit.client.0.smithi012.stdout:4/104: chown d6/da/f13 1682444 1 2024-03-18T14:50:45.970 INFO:tasks.workunit.client.0.smithi012.stdout:8/165: dread d0/d12/f1e [0,4194304] 0 2024-03-18T14:50:45.974 INFO:tasks.workunit.client.0.smithi012.stdout:5/112: truncate d3/fc 1540858 0 2024-03-18T14:50:45.974 INFO:tasks.workunit.client.0.smithi012.stdout:5/113: chown d3/d1e/ce 2260 1 2024-03-18T14:50:45.976 INFO:tasks.workunit.client.0.smithi012.stdout:6/140: rmdir d17 39 2024-03-18T14:50:45.976 INFO:tasks.workunit.client.0.smithi012.stdout:6/141: fsync f8 0 2024-03-18T14:50:45.980 INFO:tasks.workunit.client.0.smithi012.stdout:0/151: dread d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:45.980 INFO:tasks.workunit.client.0.smithi012.stdout:0/152: chown d1/d16/d19 31 1 2024-03-18T14:50:45.987 INFO:tasks.workunit.client.0.smithi012.stdout:5/114: symlink d3/d1e/l26 0 2024-03-18T14:50:45.987 INFO:tasks.workunit.client.0.smithi012.stdout:5/115: write f2 [1007763,34081] 0 2024-03-18T14:50:45.989 INFO:tasks.workunit.client.0.smithi012.stdout:0/153: unlink d1/d20/d23/d26/l39 0 2024-03-18T14:50:45.989 INFO:tasks.workunit.client.0.smithi012.stdout:0/154: chown d1/d20/d23/d26 177116525 1 2024-03-18T14:50:45.990 INFO:tasks.workunit.client.0.smithi012.stdout:3/122: mkdir dc/d29 0 2024-03-18T14:50:45.991 INFO:tasks.workunit.client.0.smithi012.stdout:3/123: dread - dc/f22 zero size 2024-03-18T14:50:45.992 INFO:tasks.workunit.client.0.smithi012.stdout:2/130: dwrite d0/f1c [0,4194304] 0 2024-03-18T14:50:45.992 INFO:tasks.workunit.client.0.smithi012.stdout:2/131: chown d0/d10/l12 97614850 1 2024-03-18T14:50:45.993 INFO:tasks.workunit.client.0.smithi012.stdout:8/166: getdents d0/d18/d19 0 2024-03-18T14:50:45.993 INFO:tasks.workunit.client.0.smithi012.stdout:8/167: write d0/d15/f38 [4218876,48584] 0 2024-03-18T14:50:46.005 INFO:tasks.workunit.client.0.smithi012.stdout:8/168: symlink d0/l3b 0 2024-03-18T14:50:46.006 INFO:tasks.workunit.client.0.smithi012.stdout:7/98: dwrite dd/d13/f1c [4194304,4194304] 0 2024-03-18T14:50:46.009 INFO:tasks.workunit.client.0.smithi012.stdout:7/99: unlink dd/d13/f1c 0 2024-03-18T14:50:46.010 INFO:tasks.workunit.client.0.smithi012.stdout:8/169: dread d0/d18/d19/d1b/f30 [0,4194304] 0 2024-03-18T14:50:46.011 INFO:tasks.workunit.client.0.smithi012.stdout:8/170: fdatasync d0/d18/d19/d1b/f30 0 2024-03-18T14:50:46.011 INFO:tasks.workunit.client.0.smithi012.stdout:8/171: fdatasync d0/f6 0 2024-03-18T14:50:46.011 INFO:tasks.workunit.client.0.smithi012.stdout:7/100: creat dd/d13/d23/f27 x:0 0 0 2024-03-18T14:50:46.014 INFO:tasks.workunit.client.0.smithi012.stdout:8/172: dread d0/d12/f1e [0,4194304] 0 2024-03-18T14:50:46.014 INFO:tasks.workunit.client.0.smithi012.stdout:8/173: creat d0/d12/f3c x:0 0 0 2024-03-18T14:50:46.014 INFO:tasks.workunit.client.0.smithi012.stdout:8/174: chown d0/d18/d19/d1b 62777134 1 2024-03-18T14:50:46.014 INFO:tasks.workunit.client.0.smithi012.stdout:8/175: write d0/d12/d2c/f37 [1019166,73914] 0 2024-03-18T14:50:46.025 INFO:tasks.workunit.client.0.smithi012.stdout:2/132: write d0/d10/d18/f1b [691131,66601] 0 2024-03-18T14:50:46.026 INFO:tasks.workunit.client.0.smithi012.stdout:2/133: rename d0/l17 to d0/d10/d18/l26 0 2024-03-18T14:50:46.027 INFO:tasks.workunit.client.0.smithi012.stdout:2/134: chown d0/d10/d18/f24 290523 1 2024-03-18T14:50:46.027 INFO:tasks.workunit.client.0.smithi012.stdout:2/135: rename d0/f14 to d0/d10/d18/f27 0 2024-03-18T14:50:46.029 INFO:tasks.workunit.client.0.smithi012.stdout:6/142: dwrite ff [0,4194304] 0 2024-03-18T14:50:46.032 INFO:tasks.workunit.client.0.smithi012.stdout:6/143: rename c15 to d17/d19/c25 0 2024-03-18T14:50:46.041 INFO:tasks.workunit.client.0.smithi012.stdout:6/144: write d17/d18/f1e [2164244,78932] 0 2024-03-18T14:50:46.042 INFO:tasks.workunit.client.0.smithi012.stdout:6/145: truncate fe 4286521 0 2024-03-18T14:50:46.042 INFO:tasks.workunit.client.0.smithi012.stdout:5/116: dwrite d3/d15/f19 [0,4194304] 0 2024-03-18T14:50:46.042 INFO:tasks.workunit.client.0.smithi012.stdout:9/209: dwrite d3/f25 [0,4194304] 0 2024-03-18T14:50:46.043 INFO:tasks.workunit.client.0.smithi012.stdout:6/146: mkdir d17/d26 0 2024-03-18T14:50:46.045 INFO:tasks.workunit.client.0.smithi012.stdout:6/147: mkdir d17/d19/d27 0 2024-03-18T14:50:46.046 INFO:tasks.workunit.client.0.smithi012.stdout:6/148: write d17/d18/f1e [424745,123302] 0 2024-03-18T14:50:46.046 INFO:tasks.workunit.client.0.smithi012.stdout:5/117: truncate d3/d1e/fb 5114963 0 2024-03-18T14:50:46.047 INFO:tasks.workunit.client.0.smithi012.stdout:9/210: link d3/d9/f29 d3/d9/f39 0 2024-03-18T14:50:46.048 INFO:tasks.workunit.client.0.smithi012.stdout:4/105: dwrite d6/f9 [0,4194304] 0 2024-03-18T14:50:46.049 INFO:tasks.workunit.client.0.smithi012.stdout:6/149: truncate f11 2275778 0 2024-03-18T14:50:46.049 INFO:tasks.workunit.client.0.smithi012.stdout:6/150: write f12 [219797,60902] 0 2024-03-18T14:50:46.050 INFO:tasks.workunit.client.0.smithi012.stdout:5/118: link d3/d15/f19 d3/d15/d21/f27 0 2024-03-18T14:50:46.051 INFO:tasks.workunit.client.0.smithi012.stdout:5/119: chown d3/d15/d21 396 1 2024-03-18T14:50:46.051 INFO:tasks.workunit.client.0.smithi012.stdout:5/120: fdatasync d3/ff 0 2024-03-18T14:50:46.051 INFO:tasks.workunit.client.0.smithi012.stdout:4/106: mknod d6/d19/c24 0 2024-03-18T14:50:46.052 INFO:tasks.workunit.client.0.smithi012.stdout:4/107: chown d6/d1a 61394 1 2024-03-18T14:50:46.052 INFO:tasks.workunit.client.0.smithi012.stdout:4/108: read d6/f20 [438487,41525] 0 2024-03-18T14:50:46.052 INFO:tasks.workunit.client.0.smithi012.stdout:4/109: fsync d6/f12 0 2024-03-18T14:50:46.053 INFO:tasks.workunit.client.0.smithi012.stdout:0/155: dwrite d1/d16/d19/f1b [0,4194304] 0 2024-03-18T14:50:46.053 INFO:tasks.workunit.client.0.smithi012.stdout:3/124: dwrite f6 [0,4194304] 0 2024-03-18T14:50:46.053 INFO:tasks.workunit.client.0.smithi012.stdout:3/125: write dc/f26 [1038811,83130] 0 2024-03-18T14:50:46.053 INFO:tasks.workunit.client.0.smithi012.stdout:5/121: link d3/d15/f1c d3/d1e/f28 0 2024-03-18T14:50:46.054 INFO:tasks.workunit.client.0.smithi012.stdout:7/101: dwrite dd/f1e [0,4194304] 0 2024-03-18T14:50:46.054 INFO:tasks.workunit.client.0.smithi012.stdout:8/176: dwrite d0/fc [0,4194304] 0 2024-03-18T14:50:46.054 INFO:tasks.workunit.client.0.smithi012.stdout:2/136: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:46.054 INFO:tasks.workunit.client.0.smithi012.stdout:8/177: truncate d0/f16 154886 0 2024-03-18T14:50:46.057 INFO:tasks.workunit.client.0.smithi012.stdout:4/110: creat d6/f25 x:0 0 0 2024-03-18T14:50:46.057 INFO:tasks.workunit.client.0.smithi012.stdout:4/111: write d6/f20 [927834,117549] 0 2024-03-18T14:50:46.058 INFO:tasks.workunit.client.0.smithi012.stdout:3/126: write dc/ff [469211,32354] 0 2024-03-18T14:50:46.061 INFO:tasks.workunit.client.0.smithi012.stdout:6/151: dread f9 [0,4194304] 0 2024-03-18T14:50:46.061 INFO:tasks.workunit.client.0.smithi012.stdout:6/152: write f12 [374484,115608] 0 2024-03-18T14:50:46.067 INFO:tasks.workunit.client.0.smithi012.stdout:7/102: dread fa [0,4194304] 0 2024-03-18T14:50:46.070 INFO:tasks.workunit.client.0.smithi012.stdout:5/122: write d3/d15/d21/f27 [241489,102947] 0 2024-03-18T14:50:46.071 INFO:tasks.workunit.client.0.smithi012.stdout:0/156: write d1/d16/d31/f1d [3844425,98067] 0 2024-03-18T14:50:46.071 INFO:tasks.workunit.client.0.smithi012.stdout:8/178: fsync d0/d12/d2c/f37 0 2024-03-18T14:50:46.073 INFO:tasks.workunit.client.0.smithi012.stdout:9/211: dwrite d3/d9/f10 [0,4194304] 0 2024-03-18T14:50:46.085 INFO:tasks.workunit.client.0.smithi012.stdout:7/103: dread dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:46.102 INFO:tasks.workunit.client.0.smithi012.stdout:4/112: creat d6/d1a/f26 x:0 0 0 2024-03-18T14:50:46.104 INFO:tasks.workunit.client.0.smithi012.stdout:3/127: readlink dc/l18 0 2024-03-18T14:50:46.117 INFO:tasks.workunit.client.0.smithi012.stdout:5/123: mkdir d3/d29 0 2024-03-18T14:50:46.148 INFO:tasks.workunit.client.0.smithi012.stdout:6/153: dwrite f14 [0,4194304] 0 2024-03-18T14:50:46.165 INFO:tasks.workunit.client.0.smithi012.stdout:3/128: dread f6 [0,4194304] 0 2024-03-18T14:50:46.170 INFO:tasks.workunit.client.0.smithi012.stdout:6/154: dwrite d17/f22 [0,4194304] 0 2024-03-18T14:50:46.170 INFO:tasks.workunit.client.0.smithi012.stdout:6/155: fdatasync d17/d18/f1f 0 2024-03-18T14:50:46.173 INFO:tasks.workunit.client.0.smithi012.stdout:7/104: dwrite dd/f12 [0,4194304] 0 2024-03-18T14:50:46.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/124: link d3/l6 d3/d15/d21/l2a 0 2024-03-18T14:50:46.188 INFO:tasks.workunit.client.0.smithi012.stdout:3/129: symlink dc/d29/l2a 0 2024-03-18T14:50:46.192 INFO:tasks.workunit.client.0.smithi012.stdout:6/156: truncate f16 1169963 0 2024-03-18T14:50:46.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:45 smithi100 ceph-mon[37807]: pgmap v240: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 16 MiB/s wr, 554 op/s 2024-03-18T14:50:46.194 INFO:tasks.workunit.client.0.smithi012.stdout:7/105: truncate dd/f10 1239283 0 2024-03-18T14:50:46.195 INFO:tasks.workunit.client.0.smithi012.stdout:5/125: mknod d3/d29/c2b 0 2024-03-18T14:50:46.199 INFO:tasks.workunit.client.0.smithi012.stdout:5/126: write d3/d1e/fb [2281063,14688] 0 2024-03-18T14:50:46.199 INFO:tasks.workunit.client.0.smithi012.stdout:5/127: chown d3/l4 831 1 2024-03-18T14:50:46.213 INFO:tasks.workunit.client.0.smithi012.stdout:7/106: dwrite f5 [0,4194304] 0 2024-03-18T14:50:46.218 INFO:tasks.workunit.client.0.smithi012.stdout:7/107: symlink dd/d13/d23/l28 0 2024-03-18T14:50:46.218 INFO:tasks.workunit.client.0.smithi012.stdout:7/108: write f1 [786932,28388] 0 2024-03-18T14:50:46.221 INFO:tasks.workunit.client.0.smithi012.stdout:7/109: getdents dd 0 2024-03-18T14:50:46.230 INFO:tasks.workunit.client.0.smithi012.stdout:7/110: read dd/f12 [49532,44331] 0 2024-03-18T14:50:46.231 INFO:tasks.workunit.client.0.smithi012.stdout:7/111: read f3 [4910427,86529] 0 2024-03-18T14:50:46.236 INFO:tasks.workunit.client.0.smithi012.stdout:7/112: write dd/d13/f15 [1401134,81763] 0 2024-03-18T14:50:46.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:45 smithi012 ceph-mon[29182]: pgmap v240: 65 pgs: 65 active+clean; 267 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 16 MiB/s wr, 554 op/s 2024-03-18T14:50:46.249 INFO:tasks.workunit.client.0.smithi012.stdout:0/157: rmdir d1/d16/d18 39 2024-03-18T14:50:46.249 INFO:tasks.workunit.client.0.smithi012.stdout:0/158: chown d1/d20 28111 1 2024-03-18T14:50:46.250 INFO:tasks.workunit.client.0.smithi012.stdout:7/113: read dd/f10 [313384,38483] 0 2024-03-18T14:50:46.250 INFO:tasks.workunit.client.0.smithi012.stdout:7/114: mkdir dd/d13/d23/d29 0 2024-03-18T14:50:46.251 INFO:tasks.workunit.client.0.smithi012.stdout:7/115: rmdir dd 39 2024-03-18T14:50:46.253 INFO:tasks.workunit.client.0.smithi012.stdout:7/116: fdatasync f1 0 2024-03-18T14:50:46.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/117: mkdir dd/d13/d23/d2a 0 2024-03-18T14:50:46.255 INFO:tasks.workunit.client.0.smithi012.stdout:6/157: read f16 [936071,101456] 0 2024-03-18T14:50:46.255 INFO:tasks.workunit.client.0.smithi012.stdout:6/158: write f14 [4749944,37041] 0 2024-03-18T14:50:46.256 INFO:tasks.workunit.client.0.smithi012.stdout:7/118: write dd/f22 [762397,35378] 0 2024-03-18T14:50:46.256 INFO:tasks.workunit.client.0.smithi012.stdout:7/119: readlink dd/d13/d23/l28 0 2024-03-18T14:50:46.256 INFO:tasks.workunit.client.0.smithi012.stdout:7/120: stat dd/d13/d23/d2a 0 2024-03-18T14:50:46.257 INFO:tasks.workunit.client.0.smithi012.stdout:6/159: unlink d17/d18/f1e 0 2024-03-18T14:50:46.258 INFO:tasks.workunit.client.0.smithi012.stdout:7/121: unlink dd/f22 0 2024-03-18T14:50:46.258 INFO:tasks.workunit.client.0.smithi012.stdout:7/122: creat dd/d13/d23/f2b x:0 0 0 2024-03-18T14:50:46.261 INFO:tasks.workunit.client.0.smithi012.stdout:7/123: link dd/c20 dd/d13/d23/d29/c2c 0 2024-03-18T14:50:46.261 INFO:tasks.workunit.client.0.smithi012.stdout:7/124: write dd/f12 [2332022,38797] 0 2024-03-18T14:50:46.261 INFO:tasks.workunit.client.0.smithi012.stdout:7/125: fsync dd/d13/d23/f27 0 2024-03-18T14:50:46.262 INFO:tasks.workunit.client.0.smithi012.stdout:1/138: sync 2024-03-18T14:50:46.262 INFO:tasks.workunit.client.0.smithi012.stdout:1/139: write d8/db/ff [192702,1345] 0 2024-03-18T14:50:46.262 INFO:tasks.workunit.client.0.smithi012.stdout:1/140: write d8/db/d10/d16/d1c/f2c [1022343,38711] 0 2024-03-18T14:50:46.262 INFO:tasks.workunit.client.0.smithi012.stdout:1/141: dread - d8/db/d10/d16/d1c/f2a zero size 2024-03-18T14:50:46.263 INFO:tasks.workunit.client.0.smithi012.stdout:7/126: mknod dd/c2d 0 2024-03-18T14:50:46.265 INFO:tasks.workunit.client.0.smithi012.stdout:1/142: rmdir d8/db/d10 39 2024-03-18T14:50:46.266 INFO:tasks.workunit.client.0.smithi012.stdout:7/127: unlink f3 0 2024-03-18T14:50:46.267 INFO:tasks.workunit.client.0.smithi012.stdout:7/128: dread - dd/d13/d23/f2b zero size 2024-03-18T14:50:46.272 INFO:tasks.workunit.client.0.smithi012.stdout:4/113: mkdir d6/d27 0 2024-03-18T14:50:46.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/114: chown d6/f12 142663742 1 2024-03-18T14:50:46.277 INFO:tasks.workunit.client.0.smithi012.stdout:9/212: rename d3/d9/f32 to d3/d9/d19/f3a 0 2024-03-18T14:50:46.277 INFO:tasks.workunit.client.0.smithi012.stdout:9/213: write d3/d1e/d21/f30 [509050,101222] 0 2024-03-18T14:50:46.277 INFO:tasks.workunit.client.0.smithi012.stdout:9/214: dread - d3/d9/f39 zero size 2024-03-18T14:50:46.278 INFO:tasks.workunit.client.0.smithi012.stdout:1/143: dread d8/db/fd [0,4194304] 0 2024-03-18T14:50:46.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/128: rename d3/d1e/fb to d3/d15/f2c 0 2024-03-18T14:50:46.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/129: creat d3/d15/f2d x:0 0 0 2024-03-18T14:50:46.282 INFO:tasks.workunit.client.0.smithi012.stdout:1/144: mkdir d8/db/d10/d16/d1c/d2e 0 2024-03-18T14:50:46.284 INFO:tasks.workunit.client.0.smithi012.stdout:0/159: rename d1/c15 to d1/d2d/c3e 0 2024-03-18T14:50:46.284 INFO:tasks.workunit.client.0.smithi012.stdout:0/160: fdatasync d1/d16/d19/f2a 0 2024-03-18T14:50:46.287 INFO:tasks.workunit.client.0.smithi012.stdout:5/130: unlink d3/d15/d21/f23 0 2024-03-18T14:50:46.287 INFO:tasks.workunit.client.0.smithi012.stdout:6/160: truncate f14 3861711 0 2024-03-18T14:50:46.290 INFO:tasks.workunit.client.0.smithi012.stdout:1/145: creat d8/d1e/f2f x:0 0 0 2024-03-18T14:50:46.290 INFO:tasks.workunit.client.0.smithi012.stdout:1/146: truncate d8/fc 1641079 0 2024-03-18T14:50:46.291 INFO:tasks.workunit.client.0.smithi012.stdout:0/161: creat d1/d16/d31/f3f x:0 0 0 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:5/131: mknod d3/d1e/c2e 0 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/129: dwrite fa [0,4194304] 0 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:5/132: chown d3/d15/f2c 1 1 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/130: truncate dd/d13/d23/f27 591991 0 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/131: readlink dd/l16 0 2024-03-18T14:50:46.293 INFO:tasks.workunit.client.0.smithi012.stdout:4/115: dwrite d6/f25 [0,4194304] 0 2024-03-18T14:50:46.294 INFO:tasks.workunit.client.0.smithi012.stdout:6/161: creat d17/d26/f28 x:0 0 0 2024-03-18T14:50:46.295 INFO:tasks.workunit.client.0.smithi012.stdout:1/147: rmdir d8/db 39 2024-03-18T14:50:46.295 INFO:tasks.workunit.client.0.smithi012.stdout:1/148: stat d8/db/d10/l1d 0 2024-03-18T14:50:46.297 INFO:tasks.workunit.client.0.smithi012.stdout:0/162: rename d1/d16/d19/f1b to d1/d20/d23/d26/f40 0 2024-03-18T14:50:46.298 INFO:tasks.workunit.client.0.smithi012.stdout:0/163: creat d1/d16/d31/f41 x:0 0 0 2024-03-18T14:50:46.298 INFO:tasks.workunit.client.0.smithi012.stdout:0/164: chown d1/d20/d23/d26/f36 4232611 1 2024-03-18T14:50:46.299 INFO:tasks.workunit.client.0.smithi012.stdout:5/133: mknod d3/d15/c2f 0 2024-03-18T14:50:46.300 INFO:tasks.workunit.client.0.smithi012.stdout:7/132: creat dd/f2e x:0 0 0 2024-03-18T14:50:46.304 INFO:tasks.workunit.client.0.smithi012.stdout:4/116: creat d6/f28 x:0 0 0 2024-03-18T14:50:46.304 INFO:tasks.workunit.client.0.smithi012.stdout:4/117: stat d6/f28 0 2024-03-18T14:50:46.304 INFO:tasks.workunit.client.0.smithi012.stdout:1/149: rmdir d8/db 39 2024-03-18T14:50:46.306 INFO:tasks.workunit.client.0.smithi012.stdout:8/179: sync 2024-03-18T14:50:46.306 INFO:tasks.workunit.client.0.smithi012.stdout:2/137: sync 2024-03-18T14:50:46.306 INFO:tasks.workunit.client.0.smithi012.stdout:2/138: readlink d0/d10/l23 0 2024-03-18T14:50:46.308 INFO:tasks.workunit.client.0.smithi012.stdout:5/134: getdents d3/d29 0 2024-03-18T14:50:46.309 INFO:tasks.workunit.client.0.smithi012.stdout:7/133: rename dd/d13/d23/d29 to dd/d13/d23/d2f 0 2024-03-18T14:50:46.310 INFO:tasks.workunit.client.0.smithi012.stdout:2/139: write d0/d10/f13 [2325564,37960] 0 2024-03-18T14:50:46.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/150: creat d8/db/f30 x:0 0 0 2024-03-18T14:50:46.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/151: truncate d8/db/d10/d16/d1c/f2a 904019 0 2024-03-18T14:50:46.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/152: stat d8/db/d10/d16/d1c/f2c 0 2024-03-18T14:50:46.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/153: dread - d8/db/d10/d16/d1c/f23 zero size 2024-03-18T14:50:46.313 INFO:tasks.workunit.client.0.smithi012.stdout:8/180: mknod d0/d12/c3d 0 2024-03-18T14:50:46.315 INFO:tasks.workunit.client.0.smithi012.stdout:0/165: unlink d1/d16/d31/f41 0 2024-03-18T14:50:46.317 INFO:tasks.workunit.client.0.smithi012.stdout:5/135: symlink d3/l30 0 2024-03-18T14:50:46.317 INFO:tasks.workunit.client.0.smithi012.stdout:5/136: chown d3/d1e/l12 209855 1 2024-03-18T14:50:46.317 INFO:tasks.workunit.client.0.smithi012.stdout:5/137: creat d3/d15/f31 x:0 0 0 2024-03-18T14:50:46.317 INFO:tasks.workunit.client.0.smithi012.stdout:5/138: readlink d3/l25 0 2024-03-18T14:50:46.317 INFO:tasks.workunit.client.0.smithi012.stdout:5/139: chown d3/d1e/c20 47544 1 2024-03-18T14:50:46.319 INFO:tasks.workunit.client.0.smithi012.stdout:2/140: symlink d0/d10/l28 0 2024-03-18T14:50:46.321 INFO:tasks.workunit.client.0.smithi012.stdout:8/181: mkdir d0/d18/d19/d1b/d2a/d3e 0 2024-03-18T14:50:46.322 INFO:tasks.workunit.client.0.smithi012.stdout:0/166: mknod d1/d20/d23/d3b/d33/c42 0 2024-03-18T14:50:46.327 INFO:tasks.workunit.client.0.smithi012.stdout:2/141: symlink d0/l29 0 2024-03-18T14:50:46.328 INFO:tasks.workunit.client.0.smithi012.stdout:5/140: dread d3/d15/d21/f27 [0,4194304] 0 2024-03-18T14:50:46.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/154: getdents d8/db/d10/d16/d1c 0 2024-03-18T14:50:46.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/155: truncate d8/db/d10/d16/d1c/f2b 781462 0 2024-03-18T14:50:46.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/156: write d8/db/f30 [438302,91490] 0 2024-03-18T14:50:46.330 INFO:tasks.workunit.client.0.smithi012.stdout:8/182: link d0/f10 d0/d18/d19/f3f 0 2024-03-18T14:50:46.331 INFO:tasks.workunit.client.0.smithi012.stdout:0/167: symlink d1/d20/l43 0 2024-03-18T14:50:46.333 INFO:tasks.workunit.client.0.smithi012.stdout:5/141: link d3/l4 d3/d1e/l32 0 2024-03-18T14:50:46.334 INFO:tasks.workunit.client.0.smithi012.stdout:7/134: dwrite dd/f12 [0,4194304] 0 2024-03-18T14:50:46.335 INFO:tasks.workunit.client.0.smithi012.stdout:4/118: getdents d6 0 2024-03-18T14:50:46.337 INFO:tasks.workunit.client.0.smithi012.stdout:1/157: mkdir d8/db/d10/d31 0 2024-03-18T14:50:46.338 INFO:tasks.workunit.client.0.smithi012.stdout:5/142: mkdir d3/d1e/d33 0 2024-03-18T14:50:46.340 INFO:tasks.workunit.client.0.smithi012.stdout:7/135: rename dd/d13/c21 to dd/d13/d23/c30 0 2024-03-18T14:50:46.341 INFO:tasks.workunit.client.0.smithi012.stdout:2/142: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:46.341 INFO:tasks.workunit.client.0.smithi012.stdout:8/183: dwrite d0/f7 [0,4194304] 0 2024-03-18T14:50:46.343 INFO:tasks.workunit.client.0.smithi012.stdout:4/119: unlink d6/f9 0 2024-03-18T14:50:46.344 INFO:tasks.workunit.client.0.smithi012.stdout:1/158: truncate d8/f1a 3744451 0 2024-03-18T14:50:46.346 INFO:tasks.workunit.client.0.smithi012.stdout:5/143: creat d3/d1e/d33/f34 x:0 0 0 2024-03-18T14:50:46.346 INFO:tasks.workunit.client.0.smithi012.stdout:5/144: dread - d3/d15/f2d zero size 2024-03-18T14:50:46.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/136: truncate dd/d13/f15 2334993 0 2024-03-18T14:50:46.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/137: stat dd/l1f 0 2024-03-18T14:50:46.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/138: creat dd/f31 x:0 0 0 2024-03-18T14:50:46.348 INFO:tasks.workunit.client.0.smithi012.stdout:8/184: symlink d0/d18/l40 0 2024-03-18T14:50:46.352 INFO:tasks.workunit.client.0.smithi012.stdout:1/159: link d8/db/d10/l29 d8/db/d10/l32 0 2024-03-18T14:50:46.354 INFO:tasks.workunit.client.0.smithi012.stdout:8/185: dread d0/d12/f2d [0,4194304] 0 2024-03-18T14:50:46.354 INFO:tasks.workunit.client.0.smithi012.stdout:8/186: rename d0/d18/d19/d1b/d2a to d0/d18/d19/d1b/d2a/d3e/d41 22 2024-03-18T14:50:46.356 INFO:tasks.workunit.client.0.smithi012.stdout:7/139: creat dd/d13/d23/d2f/f32 x:0 0 0 2024-03-18T14:50:46.358 INFO:tasks.workunit.client.0.smithi012.stdout:7/140: mkdir dd/d13/d23/d33 0 2024-03-18T14:50:46.358 INFO:tasks.workunit.client.0.smithi012.stdout:7/141: chown dd/c25 144613 1 2024-03-18T14:50:46.359 INFO:tasks.workunit.client.0.smithi012.stdout:7/142: symlink dd/d13/d23/d2a/l34 0 2024-03-18T14:50:46.359 INFO:tasks.workunit.client.0.smithi012.stdout:7/143: readlink dd/l16 0 2024-03-18T14:50:46.360 INFO:tasks.workunit.client.0.smithi012.stdout:7/144: stat dd/l1b 0 2024-03-18T14:50:46.361 INFO:tasks.workunit.client.0.smithi012.stdout:4/120: write d6/f25 [3142698,25240] 0 2024-03-18T14:50:46.363 INFO:tasks.workunit.client.0.smithi012.stdout:2/143: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:46.364 INFO:tasks.workunit.client.0.smithi012.stdout:8/187: dread d0/d18/d19/f29 [0,4194304] 0 2024-03-18T14:50:46.368 INFO:tasks.workunit.client.0.smithi012.stdout:5/145: dwrite d3/d15/f2c [0,4194304] 0 2024-03-18T14:50:46.369 INFO:tasks.workunit.client.0.smithi012.stdout:4/121: truncate d6/f12 2472179 0 2024-03-18T14:50:46.371 INFO:tasks.workunit.client.0.smithi012.stdout:2/144: getdents d0/d10 0 2024-03-18T14:50:46.373 INFO:tasks.workunit.client.0.smithi012.stdout:4/122: creat d6/d1a/f29 x:0 0 0 2024-03-18T14:50:46.375 INFO:tasks.workunit.client.0.smithi012.stdout:2/145: creat d0/d10/f2a x:0 0 0 2024-03-18T14:50:46.376 INFO:tasks.workunit.client.0.smithi012.stdout:5/146: rmdir d3/d15/d21 39 2024-03-18T14:50:46.379 INFO:tasks.workunit.client.0.smithi012.stdout:5/147: dread d3/d15/f1c [0,4194304] 0 2024-03-18T14:50:46.379 INFO:tasks.workunit.client.0.smithi012.stdout:5/148: chown d3/d1e/c9 43 1 2024-03-18T14:50:46.379 INFO:tasks.workunit.client.0.smithi012.stdout:5/149: fsync d3/d1e/f28 0 2024-03-18T14:50:46.381 INFO:tasks.workunit.client.0.smithi012.stdout:5/150: dread d3/d1e/f28 [0,4194304] 0 2024-03-18T14:50:46.381 INFO:tasks.workunit.client.0.smithi012.stdout:5/151: write d3/d15/f1b [99569,26252] 0 2024-03-18T14:50:46.381 INFO:tasks.workunit.client.0.smithi012.stdout:4/123: rename d6/f20 to d6/d1a/f2a 0 2024-03-18T14:50:46.382 INFO:tasks.workunit.client.0.smithi012.stdout:2/146: symlink d0/d10/d18/l2b 0 2024-03-18T14:50:46.382 INFO:tasks.workunit.client.0.smithi012.stdout:2/147: creat d0/f2c x:0 0 0 2024-03-18T14:50:46.383 INFO:tasks.workunit.client.0.smithi012.stdout:5/152: mkdir d3/d29/d35 0 2024-03-18T14:50:46.384 INFO:tasks.workunit.client.0.smithi012.stdout:7/145: rmdir dd/d13/d23 39 2024-03-18T14:50:46.385 INFO:tasks.workunit.client.0.smithi012.stdout:4/124: fdatasync d6/d19/f1d 0 2024-03-18T14:50:46.386 INFO:tasks.workunit.client.0.smithi012.stdout:8/188: dwrite d0/d18/f1d [0,4194304] 0 2024-03-18T14:50:46.386 INFO:tasks.workunit.client.0.smithi012.stdout:8/189: creat d0/d15/f42 x:0 0 0 2024-03-18T14:50:46.387 INFO:tasks.workunit.client.0.smithi012.stdout:2/148: symlink d0/d10/d18/l2d 0 2024-03-18T14:50:46.389 INFO:tasks.workunit.client.0.smithi012.stdout:5/153: truncate d3/d1e/f13 4132703 0 2024-03-18T14:50:46.390 INFO:tasks.workunit.client.0.smithi012.stdout:7/146: symlink dd/d13/d23/d2a/l35 0 2024-03-18T14:50:46.390 INFO:tasks.workunit.client.0.smithi012.stdout:7/147: fsync dd/d13/d23/d2f/f32 0 2024-03-18T14:50:46.394 INFO:tasks.workunit.client.0.smithi012.stdout:2/149: chown d0/d10/l12 2653209 1 2024-03-18T14:50:46.395 INFO:tasks.workunit.client.0.smithi012.stdout:1/160: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:50:46.395 INFO:tasks.workunit.client.0.smithi012.stdout:1/161: fdatasync f5 0 2024-03-18T14:50:46.395 INFO:tasks.workunit.client.0.smithi012.stdout:1/162: fdatasync d8/db/d10/d16/d1c/f2a 0 2024-03-18T14:50:46.396 INFO:tasks.workunit.client.0.smithi012.stdout:5/154: mkdir d3/d29/d36 0 2024-03-18T14:50:46.399 INFO:tasks.workunit.client.0.smithi012.stdout:7/148: creat dd/d13/d23/d2a/f36 x:0 0 0 2024-03-18T14:50:46.399 INFO:tasks.workunit.client.0.smithi012.stdout:7/149: write dd/d13/d23/f2b [924261,63396] 0 2024-03-18T14:50:46.401 INFO:tasks.workunit.client.0.smithi012.stdout:8/190: unlink d0/d15/f42 0 2024-03-18T14:50:46.401 INFO:tasks.workunit.client.0.smithi012.stdout:8/191: write d0/d12/f2d [1062253,77391] 0 2024-03-18T14:50:46.405 INFO:tasks.workunit.client.0.smithi012.stdout:7/150: mkdir dd/d13/d23/d33/d37 0 2024-03-18T14:50:46.405 INFO:tasks.workunit.client.0.smithi012.stdout:5/155: write d3/d15/f2c [3449514,124759] 0 2024-03-18T14:50:46.405 INFO:tasks.workunit.client.0.smithi012.stdout:5/156: creat d3/d15/f37 x:0 0 0 2024-03-18T14:50:46.407 INFO:tasks.workunit.client.0.smithi012.stdout:8/192: link d0/d18/d19/f29 d0/d15/f43 0 2024-03-18T14:50:46.409 INFO:tasks.workunit.client.0.smithi012.stdout:7/151: rename dd/d13/d23/d2f/c2c to dd/d13/d23/d33/d37/c38 0 2024-03-18T14:50:46.413 INFO:tasks.workunit.client.0.smithi012.stdout:5/157: link d3/d29/c2b d3/d29/c38 0 2024-03-18T14:50:46.413 INFO:tasks.workunit.client.0.smithi012.stdout:5/158: chown f2 1792003 1 2024-03-18T14:50:46.414 INFO:tasks.workunit.client.0.smithi012.stdout:7/152: mkdir dd/d13/d23/d39 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/159: rename d3/d29/c2b to d3/c39 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/160: truncate d3/d15/f37 199083 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/161: creat d3/f3a x:0 0 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/162: chown d3/d1e/l32 26539105 1 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/163: creat d3/f3b x:0 0 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/164: chown d3/l4 179775766 1 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/165: write d3/d15/f1d [196656,57936] 0 2024-03-18T14:50:46.416 INFO:tasks.workunit.client.0.smithi012.stdout:7/153: symlink dd/d13/d23/l3a 0 2024-03-18T14:50:46.418 INFO:tasks.workunit.client.0.smithi012.stdout:5/166: symlink d3/d29/d35/l3c 0 2024-03-18T14:50:46.419 INFO:tasks.workunit.client.0.smithi012.stdout:5/167: mkdir d3/d29/d36/d3d 0 2024-03-18T14:50:46.419 INFO:tasks.workunit.client.0.smithi012.stdout:5/168: stat d3/fc 0 2024-03-18T14:50:46.419 INFO:tasks.workunit.client.0.smithi012.stdout:5/169: write d3/d1e/f1a [575295,107786] 0 2024-03-18T14:50:46.420 INFO:tasks.workunit.client.0.smithi012.stdout:5/170: creat d3/d29/d36/d3d/f3e x:0 0 0 2024-03-18T14:50:46.420 INFO:tasks.workunit.client.0.smithi012.stdout:5/171: creat d3/d29/d36/d3d/f3f x:0 0 0 2024-03-18T14:50:46.421 INFO:tasks.workunit.client.0.smithi012.stdout:5/172: symlink d3/d1e/l40 0 2024-03-18T14:50:46.455 INFO:tasks.workunit.client.0.smithi012.stdout:7/154: dread f1 [0,4194304] 0 2024-03-18T14:50:46.456 INFO:tasks.workunit.client.0.smithi012.stdout:4/125: dwrite d6/da/fd [0,4194304] 0 2024-03-18T14:50:46.456 INFO:tasks.workunit.client.0.smithi012.stdout:7/155: symlink dd/d13/d23/d2f/l3b 0 2024-03-18T14:50:46.458 INFO:tasks.workunit.client.0.smithi012.stdout:4/126: truncate d6/f21 828045 0 2024-03-18T14:50:46.460 INFO:tasks.workunit.client.0.smithi012.stdout:7/156: rename l4 to dd/d13/d23/d2f/l3c 0 2024-03-18T14:50:46.460 INFO:tasks.workunit.client.0.smithi012.stdout:7/157: truncate dd/f10 1809801 0 2024-03-18T14:50:46.470 INFO:tasks.workunit.client.0.smithi012.stdout:7/158: unlink dd/le 0 2024-03-18T14:50:46.470 INFO:tasks.workunit.client.0.smithi012.stdout:4/127: creat d6/d27/f2b x:0 0 0 2024-03-18T14:50:46.470 INFO:tasks.workunit.client.0.smithi012.stdout:4/128: fdatasync d6/d19/f1c 0 2024-03-18T14:50:46.472 INFO:tasks.workunit.client.0.smithi012.stdout:7/159: link f5 dd/d13/d23/d2f/f3d 0 2024-03-18T14:50:46.472 INFO:tasks.workunit.client.0.smithi012.stdout:7/160: chown f1 6838621 1 2024-03-18T14:50:46.472 INFO:tasks.workunit.client.0.smithi012.stdout:7/161: creat dd/d13/d23/d2a/f3e x:0 0 0 2024-03-18T14:50:46.472 INFO:tasks.workunit.client.0.smithi012.stdout:7/162: write dd/d13/d23/d2f/f32 [108848,62618] 0 2024-03-18T14:50:46.473 INFO:tasks.workunit.client.0.smithi012.stdout:4/129: rename d6/d27/f2b to d6/d19/f2c 0 2024-03-18T14:50:46.481 INFO:tasks.workunit.client.0.smithi012.stdout:2/150: dwrite d0/f1c [0,4194304] 0 2024-03-18T14:50:46.484 INFO:tasks.workunit.client.0.smithi012.stdout:5/173: dwrite d3/f1f [0,4194304] 0 2024-03-18T14:50:46.484 INFO:tasks.workunit.client.0.smithi012.stdout:5/174: write d3/d29/d36/d3d/f3f [222771,66726] 0 2024-03-18T14:50:46.484 INFO:tasks.workunit.client.0.smithi012.stdout:8/193: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:46.485 INFO:tasks.workunit.client.0.smithi012.stdout:5/175: rmdir d3/d29/d35 39 2024-03-18T14:50:46.488 INFO:tasks.workunit.client.0.smithi012.stdout:8/194: fsync d0/d15/f43 0 2024-03-18T14:50:46.488 INFO:tasks.workunit.client.0.smithi012.stdout:8/195: fdatasync d0/d12/f20 0 2024-03-18T14:50:46.489 INFO:tasks.workunit.client.0.smithi012.stdout:8/196: dread - d0/f35 zero size 2024-03-18T14:50:46.489 INFO:tasks.workunit.client.0.smithi012.stdout:8/197: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:46.490 INFO:tasks.workunit.client.0.smithi012.stdout:1/163: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:50:46.493 INFO:tasks.workunit.client.0.smithi012.stdout:2/151: dread d0/f5 [0,4194304] 0 2024-03-18T14:50:46.493 INFO:tasks.workunit.client.0.smithi012.stdout:5/176: creat d3/d29/f41 x:0 0 0 2024-03-18T14:50:46.494 INFO:tasks.workunit.client.0.smithi012.stdout:8/198: creat d0/d12/d33/f44 x:0 0 0 2024-03-18T14:50:46.495 INFO:tasks.workunit.client.0.smithi012.stdout:7/163: dwrite dd/f12 [0,4194304] 0 2024-03-18T14:50:46.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/164: creat d8/db/d10/f33 x:0 0 0 2024-03-18T14:50:46.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/165: fdatasync d8/db/f25 0 2024-03-18T14:50:46.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/166: creat d8/db/d10/f34 x:0 0 0 2024-03-18T14:50:46.497 INFO:tasks.workunit.client.0.smithi012.stdout:1/167: write d8/fa [1616835,98566] 0 2024-03-18T14:50:46.499 INFO:tasks.workunit.client.0.smithi012.stdout:2/152: rename d0/ce to d0/c2e 0 2024-03-18T14:50:46.499 INFO:tasks.workunit.client.0.smithi012.stdout:2/153: fsync d0/f5 0 2024-03-18T14:50:46.500 INFO:tasks.workunit.client.0.smithi012.stdout:5/177: mkdir d3/d15/d42 0 2024-03-18T14:50:46.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/164: mknod dd/d13/d23/d33/d37/c3f 0 2024-03-18T14:50:46.503 INFO:tasks.workunit.client.0.smithi012.stdout:8/199: write d0/fc [3657696,55986] 0 2024-03-18T14:50:46.504 INFO:tasks.workunit.client.0.smithi012.stdout:7/165: dread dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:50:46.505 INFO:tasks.workunit.client.0.smithi012.stdout:7/166: write dd/d13/d23/f27 [455001,129347] 0 2024-03-18T14:50:46.505 INFO:tasks.workunit.client.0.smithi012.stdout:7/167: fsync dd/f1d 0 2024-03-18T14:50:46.505 INFO:tasks.workunit.client.0.smithi012.stdout:7/168: write fa [4918500,95725] 0 2024-03-18T14:50:46.508 INFO:tasks.workunit.client.0.smithi012.stdout:2/154: chown d0/d10/f13 356348 1 2024-03-18T14:50:46.513 INFO:tasks.workunit.client.0.smithi012.stdout:8/200: write d0/f7 [756821,8141] 0 2024-03-18T14:50:46.522 INFO:tasks.workunit.client.0.smithi012.stdout:4/130: rename d6/d19/f2c to d6/d1a/f2d 0 2024-03-18T14:50:46.523 INFO:tasks.workunit.client.0.smithi012.stdout:8/201: write d0/d18/f1d [558608,121689] 0 2024-03-18T14:50:46.524 INFO:tasks.workunit.client.0.smithi012.stdout:4/131: creat d6/f2e x:0 0 0 2024-03-18T14:50:46.525 INFO:tasks.workunit.client.0.smithi012.stdout:4/132: chown d6/l7 1 1 2024-03-18T14:50:46.528 INFO:tasks.workunit.client.0.smithi012.stdout:1/168: dwrite d8/db/d10/d16/d1c/f2c [0,4194304] 0 2024-03-18T14:50:46.528 INFO:tasks.workunit.client.0.smithi012.stdout:1/169: creat d8/d1e/f35 x:0 0 0 2024-03-18T14:50:46.529 INFO:tasks.workunit.client.0.smithi012.stdout:1/170: stat d8/l24 0 2024-03-18T14:50:46.529 INFO:tasks.workunit.client.0.smithi012.stdout:1/171: truncate d8/db/d10/d16/d1c/f2a 1929928 0 2024-03-18T14:50:46.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/178: dwrite d3/d15/f2d [0,4194304] 0 2024-03-18T14:50:46.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/179: write d3/d15/f31 [870998,18756] 0 2024-03-18T14:50:46.531 INFO:tasks.workunit.client.0.smithi012.stdout:7/169: dwrite dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:46.531 INFO:tasks.workunit.client.0.smithi012.stdout:8/202: dread d0/fa [0,4194304] 0 2024-03-18T14:50:46.532 INFO:tasks.workunit.client.0.smithi012.stdout:8/203: fdatasync d0/f6 0 2024-03-18T14:50:46.533 INFO:tasks.workunit.client.0.smithi012.stdout:8/204: read d0/f10 [547787,120557] 0 2024-03-18T14:50:46.533 INFO:tasks.workunit.client.0.smithi012.stdout:2/155: rmdir d0 39 2024-03-18T14:50:46.534 INFO:tasks.workunit.client.0.smithi012.stdout:8/205: read d0/f6 [135357,32317] 0 2024-03-18T14:50:46.536 INFO:tasks.workunit.client.0.smithi012.stdout:8/206: write d0/d18/d19/d1b/f30 [924181,124743] 0 2024-03-18T14:50:46.536 INFO:tasks.workunit.client.0.smithi012.stdout:1/172: symlink d8/d1e/l36 0 2024-03-18T14:50:46.545 INFO:tasks.workunit.client.0.smithi012.stdout:8/207: symlink d0/d18/l45 0 2024-03-18T14:50:46.546 INFO:tasks.workunit.client.0.smithi012.stdout:8/208: creat d0/d12/f46 x:0 0 0 2024-03-18T14:50:46.546 INFO:tasks.workunit.client.0.smithi012.stdout:8/209: write d0/fc [3995215,72519] 0 2024-03-18T14:50:46.546 INFO:tasks.workunit.client.0.smithi012.stdout:2/156: link d0/l1d d0/d10/l2f 0 2024-03-18T14:50:46.551 INFO:tasks.workunit.client.0.smithi012.stdout:2/157: dread d0/d10/d18/f1f [0,4194304] 0 2024-03-18T14:50:46.551 INFO:tasks.workunit.client.0.smithi012.stdout:2/158: write d0/f2c [506163,66303] 0 2024-03-18T14:50:46.551 INFO:tasks.workunit.client.0.smithi012.stdout:2/159: fdatasync d0/d10/d18/f1f 0 2024-03-18T14:50:46.551 INFO:tasks.workunit.client.0.smithi012.stdout:2/160: read d0/f8 [2103471,24228] 0 2024-03-18T14:50:46.552 INFO:tasks.workunit.client.0.smithi012.stdout:8/210: dread d0/d18/f25 [0,4194304] 0 2024-03-18T14:50:46.553 INFO:tasks.workunit.client.0.smithi012.stdout:8/211: truncate d0/d15/f38 1540202 0 2024-03-18T14:50:46.553 INFO:tasks.workunit.client.0.smithi012.stdout:8/212: readlink d0/d18/l40 0 2024-03-18T14:50:46.554 INFO:tasks.workunit.client.0.smithi012.stdout:7/170: dread f5 [0,4194304] 0 2024-03-18T14:50:46.554 INFO:tasks.workunit.client.0.smithi012.stdout:7/171: fdatasync f5 0 2024-03-18T14:50:46.555 INFO:tasks.workunit.client.0.smithi012.stdout:7/172: unlink dd/f1d 0 2024-03-18T14:50:46.556 INFO:tasks.workunit.client.0.smithi012.stdout:7/173: symlink dd/d13/d23/d33/d37/l40 0 2024-03-18T14:50:46.556 INFO:tasks.workunit.client.0.smithi012.stdout:4/133: dwrite d6/f25 [0,4194304] 0 2024-03-18T14:50:46.556 INFO:tasks.workunit.client.0.smithi012.stdout:4/134: fdatasync d6/d1a/f29 0 2024-03-18T14:50:46.557 INFO:tasks.workunit.client.0.smithi012.stdout:5/180: dwrite d3/f3a [0,4194304] 0 2024-03-18T14:50:46.557 INFO:tasks.workunit.client.0.smithi012.stdout:7/174: link dd/c2d dd/d13/d23/d33/c41 0 2024-03-18T14:50:46.560 INFO:tasks.workunit.client.0.smithi012.stdout:4/135: truncate d6/d1a/f2a 687280 0 2024-03-18T14:50:46.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/161: dread d0/f1c [0,4194304] 0 2024-03-18T14:50:46.566 INFO:tasks.workunit.client.0.smithi012.stdout:7/175: rename dd/f12 to dd/f42 0 2024-03-18T14:50:46.566 INFO:tasks.workunit.client.0.smithi012.stdout:7/176: dread - dd/f31 zero size 2024-03-18T14:50:46.568 INFO:tasks.workunit.client.0.smithi012.stdout:2/162: getdents d0/d10/d18 0 2024-03-18T14:50:46.569 INFO:tasks.workunit.client.0.smithi012.stdout:5/181: rmdir d3/d29/d36/d3d 39 2024-03-18T14:50:46.569 INFO:tasks.workunit.client.0.smithi012.stdout:3/130: sync 2024-03-18T14:50:46.570 INFO:tasks.workunit.client.0.smithi012.stdout:7/177: unlink dd/d13/d23/d33/d37/c38 0 2024-03-18T14:50:46.571 INFO:tasks.workunit.client.0.smithi012.stdout:4/136: rmdir d6/d19 39 2024-03-18T14:50:46.572 INFO:tasks.workunit.client.0.smithi012.stdout:2/163: rmdir d0/d10 39 2024-03-18T14:50:46.572 INFO:tasks.workunit.client.0.smithi012.stdout:2/164: chown d0/d10/d18/l2d 266468130 1 2024-03-18T14:50:46.573 INFO:tasks.workunit.client.0.smithi012.stdout:5/182: unlink d3/c7 0 2024-03-18T14:50:46.575 INFO:tasks.workunit.client.0.smithi012.stdout:5/183: read d3/d29/d36/d3d/f3f [70332,85338] 0 2024-03-18T14:50:46.575 INFO:tasks.workunit.client.0.smithi012.stdout:5/184: chown d3/d15/l16 1499 1 2024-03-18T14:50:46.576 INFO:tasks.workunit.client.0.smithi012.stdout:7/178: unlink dd/d13/d23/d2a/f36 0 2024-03-18T14:50:46.577 INFO:tasks.workunit.client.0.smithi012.stdout:5/185: dread d3/d15/f1c [0,4194304] 0 2024-03-18T14:50:46.578 INFO:tasks.workunit.client.0.smithi012.stdout:4/137: creat d6/d19/f2f x:0 0 0 2024-03-18T14:50:46.579 INFO:tasks.workunit.client.0.smithi012.stdout:7/179: symlink dd/d13/d23/d2f/l43 0 2024-03-18T14:50:46.579 INFO:tasks.workunit.client.0.smithi012.stdout:7/180: write dd/f42 [5050115,106372] 0 2024-03-18T14:50:46.580 INFO:tasks.workunit.client.0.smithi012.stdout:7/181: dread - dd/d13/d23/d2a/f3e zero size 2024-03-18T14:50:46.580 INFO:tasks.workunit.client.0.smithi012.stdout:7/182: getdents dd/d13/d23/d39 0 2024-03-18T14:50:46.582 INFO:tasks.workunit.client.0.smithi012.stdout:5/186: rename d3/f24 to d3/f43 0 2024-03-18T14:50:46.590 INFO:tasks.workunit.client.0.smithi012.stdout:0/168: sync 2024-03-18T14:50:46.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/215: sync 2024-03-18T14:50:46.591 INFO:tasks.workunit.client.0.smithi012.stdout:6/162: sync 2024-03-18T14:50:46.591 INFO:tasks.workunit.client.0.smithi012.stdout:6/163: chown c10 1448 1 2024-03-18T14:50:46.592 INFO:tasks.workunit.client.0.smithi012.stdout:6/164: rmdir d17/d18 39 2024-03-18T14:50:46.592 INFO:tasks.workunit.client.0.smithi012.stdout:6/165: write f14 [4426542,26381] 0 2024-03-18T14:50:46.595 INFO:tasks.workunit.client.0.smithi012.stdout:6/166: mknod d17/d26/c29 0 2024-03-18T14:50:46.596 INFO:tasks.workunit.client.0.smithi012.stdout:6/167: truncate ff 2120605 0 2024-03-18T14:50:46.597 INFO:tasks.workunit.client.0.smithi012.stdout:9/216: write d3/d9/f13 [1200165,98033] 0 2024-03-18T14:50:46.597 INFO:tasks.workunit.client.0.smithi012.stdout:9/217: readlink d3/d9/d2c/l2d 0 2024-03-18T14:50:46.598 INFO:tasks.workunit.client.0.smithi012.stdout:2/165: dwrite d0/d10/d18/f1f [0,4194304] 0 2024-03-18T14:50:46.598 INFO:tasks.workunit.client.0.smithi012.stdout:6/168: rmdir d17/d19 39 2024-03-18T14:50:46.601 INFO:tasks.workunit.client.0.smithi012.stdout:9/218: creat d3/f3b x:0 0 0 2024-03-18T14:50:46.604 INFO:tasks.workunit.client.0.smithi012.stdout:7/183: rmdir dd/d13 39 2024-03-18T14:50:46.605 INFO:tasks.workunit.client.0.smithi012.stdout:9/219: mknod d3/d9/d2c/c3c 0 2024-03-18T14:50:46.606 INFO:tasks.workunit.client.0.smithi012.stdout:2/166: dread d0/fa [0,4194304] 0 2024-03-18T14:50:46.606 INFO:tasks.workunit.client.0.smithi012.stdout:2/167: getdents d0/d10/d18 0 2024-03-18T14:50:46.606 INFO:tasks.workunit.client.0.smithi012.stdout:2/168: stat d0/d10/d18/l2b 0 2024-03-18T14:50:46.607 INFO:tasks.workunit.client.0.smithi012.stdout:7/184: unlink dd/lf 0 2024-03-18T14:50:46.607 INFO:tasks.workunit.client.0.smithi012.stdout:7/185: stat dd/d13/d23/d2a/f3e 0 2024-03-18T14:50:46.609 INFO:tasks.workunit.client.0.smithi012.stdout:9/220: rename d3/l2f to d3/d1e/d21/d35/l3d 0 2024-03-18T14:50:46.609 INFO:tasks.workunit.client.0.smithi012.stdout:9/221: chown d3/d1e/f2b 794708 1 2024-03-18T14:50:46.610 INFO:tasks.workunit.client.0.smithi012.stdout:7/186: getdents dd/d13/d23/d2f 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:0/169: dread d1/f9 [0,4194304] 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:9/222: rename d3/f16 to d3/f3e 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:0/170: write d1/d20/d23/d3b/f2c [294118,83094] 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:0/171: fsync d1/d16/d19/f37 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:9/223: write d3/d9/f13 [3555754,15405] 0 2024-03-18T14:50:46.613 INFO:tasks.workunit.client.0.smithi012.stdout:9/224: mknod d3/d1e/c3f 0 2024-03-18T14:50:46.615 INFO:tasks.workunit.client.0.smithi012.stdout:5/187: dwrite f2 [0,4194304] 0 2024-03-18T14:50:46.615 INFO:tasks.workunit.client.0.smithi012.stdout:5/188: read d3/d15/f19 [3521882,47281] 0 2024-03-18T14:50:46.615 INFO:tasks.workunit.client.0.smithi012.stdout:5/189: write d3/d15/f1b [335847,69490] 0 2024-03-18T14:50:46.615 INFO:tasks.workunit.client.0.smithi012.stdout:7/187: rmdir dd/d13/d23/d39 0 2024-03-18T14:50:46.616 INFO:tasks.workunit.client.0.smithi012.stdout:6/169: write f14 [883341,62572] 0 2024-03-18T14:50:46.619 INFO:tasks.workunit.client.0.smithi012.stdout:5/190: unlink d3/ff 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:4/138: dwrite d6/d1a/f26 [0,4194304] 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:2/169: write d0/d10/d18/f27 [2906227,97323] 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:4/139: fsync d6/da/f16 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:4/140: write d6/d19/f2f [515874,4595] 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:4/141: write d6/da/f13 [3199514,23980] 0 2024-03-18T14:50:46.620 INFO:tasks.workunit.client.0.smithi012.stdout:7/188: mknod dd/d13/d23/d33/d37/c44 0 2024-03-18T14:50:46.622 INFO:tasks.workunit.client.0.smithi012.stdout:6/170: symlink d17/l2a 0 2024-03-18T14:50:46.622 INFO:tasks.workunit.client.0.smithi012.stdout:6/171: read f4 [568024,50117] 0 2024-03-18T14:50:46.624 INFO:tasks.workunit.client.0.smithi012.stdout:5/191: symlink d3/d15/l44 0 2024-03-18T14:50:46.628 INFO:tasks.workunit.client.0.smithi012.stdout:2/170: symlink d0/l30 0 2024-03-18T14:50:46.629 INFO:tasks.workunit.client.0.smithi012.stdout:0/172: dwrite d1/d16/d19/f2a [0,4194304] 0 2024-03-18T14:50:46.629 INFO:tasks.workunit.client.0.smithi012.stdout:0/173: write d1/d20/f2e [872588,120109] 0 2024-03-18T14:50:46.629 INFO:tasks.workunit.client.0.smithi012.stdout:0/174: write d1/d20/d23/d26/f36 [149834,67842] 0 2024-03-18T14:50:46.630 INFO:tasks.workunit.client.0.smithi012.stdout:4/142: mkdir d6/d1a/d30 0 2024-03-18T14:50:46.631 INFO:tasks.workunit.client.0.smithi012.stdout:7/189: dread fa [4194304,4194304] 0 2024-03-18T14:50:46.631 INFO:tasks.workunit.client.0.smithi012.stdout:5/192: rename d3/d15/f2d to d3/d29/d36/f45 0 2024-03-18T14:50:46.634 INFO:tasks.workunit.client.0.smithi012.stdout:0/175: mknod d1/d2d/c44 0 2024-03-18T14:50:46.636 INFO:tasks.workunit.client.0.smithi012.stdout:5/193: unlink d3/d29/d36/d3d/f3f 0 2024-03-18T14:50:46.637 INFO:tasks.workunit.client.0.smithi012.stdout:0/176: symlink d1/d16/d19/l45 0 2024-03-18T14:50:46.638 INFO:tasks.workunit.client.0.smithi012.stdout:0/177: truncate d1/f10 195799 0 2024-03-18T14:50:46.638 INFO:tasks.workunit.client.0.smithi012.stdout:0/178: chown d1/d20/d23/l3a 0 1 2024-03-18T14:50:46.641 INFO:tasks.workunit.client.0.smithi012.stdout:9/225: dwrite d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:46.642 INFO:tasks.workunit.client.0.smithi012.stdout:7/190: dread dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:46.642 INFO:tasks.workunit.client.0.smithi012.stdout:9/226: fsync d3/d9/f13 0 2024-03-18T14:50:46.642 INFO:tasks.workunit.client.0.smithi012.stdout:7/191: read dd/f42 [4484444,97099] 0 2024-03-18T14:50:46.645 INFO:tasks.workunit.client.0.smithi012.stdout:9/227: creat d3/f40 x:0 0 0 2024-03-18T14:50:46.645 INFO:tasks.workunit.client.0.smithi012.stdout:6/172: dwrite f4 [0,4194304] 0 2024-03-18T14:50:46.649 INFO:tasks.workunit.client.0.smithi012.stdout:9/228: rename d3/d1e/d21/f36 to d3/f41 0 2024-03-18T14:50:46.649 INFO:tasks.workunit.client.0.smithi012.stdout:9/229: write d3/d9/f1c [523886,53727] 0 2024-03-18T14:50:46.649 INFO:tasks.workunit.client.0.smithi012.stdout:2/171: write d0/d10/f16 [3018817,66424] 0 2024-03-18T14:50:46.649 INFO:tasks.workunit.client.0.smithi012.stdout:2/172: write d0/d10/f16 [329003,89015] 0 2024-03-18T14:50:46.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/173: chown d0/d10/d18 5 1 2024-03-18T14:50:46.650 INFO:tasks.workunit.client.0.smithi012.stdout:9/230: mknod d3/d1e/c42 0 2024-03-18T14:50:46.650 INFO:tasks.workunit.client.0.smithi012.stdout:9/231: write d3/f3b [463556,41533] 0 2024-03-18T14:50:46.651 INFO:tasks.workunit.client.0.smithi012.stdout:7/192: dread f1 [0,4194304] 0 2024-03-18T14:50:46.652 INFO:tasks.workunit.client.0.smithi012.stdout:2/174: mkdir d0/d10/d18/d31 0 2024-03-18T14:50:46.652 INFO:tasks.workunit.client.0.smithi012.stdout:2/175: readlink d0/d10/d18/l26 0 2024-03-18T14:50:46.653 INFO:tasks.workunit.client.0.smithi012.stdout:4/143: dread d6/f11 [0,4194304] 0 2024-03-18T14:50:46.653 INFO:tasks.workunit.client.0.smithi012.stdout:4/144: creat d6/d1a/f31 x:0 0 0 2024-03-18T14:50:46.655 INFO:tasks.workunit.client.0.smithi012.stdout:0/179: write d1/d16/d19/f37 [653322,61636] 0 2024-03-18T14:50:46.655 INFO:tasks.workunit.client.0.smithi012.stdout:6/173: dread d17/f22 [0,4194304] 0 2024-03-18T14:50:46.656 INFO:tasks.workunit.client.0.smithi012.stdout:9/232: rename d3/d9/f33 to d3/d20/f43 0 2024-03-18T14:50:46.658 INFO:tasks.workunit.client.0.smithi012.stdout:5/194: dwrite d3/d1e/f28 [0,4194304] 0 2024-03-18T14:50:46.658 INFO:tasks.workunit.client.0.smithi012.stdout:2/176: creat d0/d10/f32 x:0 0 0 2024-03-18T14:50:46.662 INFO:tasks.workunit.client.0.smithi012.stdout:9/233: read d3/f3b [319141,64963] 0 2024-03-18T14:50:46.663 INFO:tasks.workunit.client.0.smithi012.stdout:6/174: creat d17/d19/f2b x:0 0 0 2024-03-18T14:50:46.677 INFO:tasks.workunit.client.0.smithi012.stdout:2/177: dwrite d0/d10/f16 [0,4194304] 0 2024-03-18T14:50:46.677 INFO:tasks.workunit.client.0.smithi012.stdout:2/178: fdatasync d0/fa 0 2024-03-18T14:50:46.687 INFO:tasks.workunit.client.0.smithi012.stdout:6/175: dwrite f11 [0,4194304] 0 2024-03-18T14:50:46.711 INFO:tasks.workunit.client.0.smithi012.stdout:5/195: rmdir d3/d29/d35 39 2024-03-18T14:50:46.718 INFO:tasks.workunit.client.0.smithi012.stdout:9/234: getdents d3/d1e/d21/d35 0 2024-03-18T14:50:46.718 INFO:tasks.workunit.client.0.smithi012.stdout:9/235: write d3/d1e/f2b [1692773,37399] 0 2024-03-18T14:50:46.718 INFO:tasks.workunit.client.0.smithi012.stdout:4/145: mknod d6/da/c32 0 2024-03-18T14:50:46.719 INFO:tasks.workunit.client.0.smithi012.stdout:2/179: mkdir d0/d10/d18/d33 0 2024-03-18T14:50:46.724 INFO:tasks.workunit.client.0.smithi012.stdout:6/176: unlink d17/d18/f21 0 2024-03-18T14:50:46.727 INFO:tasks.workunit.client.0.smithi012.stdout:5/196: rename d3/l4 to d3/l46 0 2024-03-18T14:50:46.728 INFO:tasks.workunit.client.0.smithi012.stdout:9/236: creat d3/d1e/f44 x:0 0 0 2024-03-18T14:50:46.729 INFO:tasks.workunit.client.0.smithi012.stdout:2/180: symlink d0/d10/d18/d31/l34 0 2024-03-18T14:50:46.729 INFO:tasks.workunit.client.0.smithi012.stdout:2/181: fdatasync d0/d10/d18/f1b 0 2024-03-18T14:50:46.730 INFO:tasks.workunit.client.0.smithi012.stdout:6/177: creat d17/f2c x:0 0 0 2024-03-18T14:50:46.730 INFO:tasks.workunit.client.0.smithi012.stdout:5/197: link d3/d29/d36/d3d/f3e d3/d29/d36/d3d/f47 0 2024-03-18T14:50:46.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/178: truncate d17/d18/f1f 567230 0 2024-03-18T14:50:46.733 INFO:tasks.workunit.client.0.smithi012.stdout:4/146: dwrite d6/da/f18 [0,4194304] 0 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:5/198: rmdir d3/d15 39 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:5/199: fsync d3/fc 0 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:5/200: write d3/d29/d36/d3d/f3e [426958,124882] 0 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:5/201: chown d3/d1e/l40 8255274 1 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:2/182: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:46.736 INFO:tasks.workunit.client.0.smithi012.stdout:2/183: truncate d0/fa 5004901 0 2024-03-18T14:50:46.739 INFO:tasks.workunit.client.0.smithi012.stdout:6/179: getdents d17/d26 0 2024-03-18T14:50:46.739 INFO:tasks.workunit.client.0.smithi012.stdout:6/180: fsync d17/d26/f28 0 2024-03-18T14:50:46.739 INFO:tasks.workunit.client.0.smithi012.stdout:6/181: fsync d17/d18/f23 0 2024-03-18T14:50:46.739 INFO:tasks.workunit.client.0.smithi012.stdout:6/182: chown f9 81198 1 2024-03-18T14:50:46.739 INFO:tasks.workunit.client.0.smithi012.stdout:6/183: fdatasync fe 0 2024-03-18T14:50:46.741 INFO:tasks.workunit.client.0.smithi012.stdout:2/184: rename d0/f1c to d0/d10/d18/d31/f35 0 2024-03-18T14:50:46.741 INFO:tasks.workunit.client.0.smithi012.stdout:2/185: write d0/f2c [1465916,47579] 0 2024-03-18T14:50:46.744 INFO:tasks.workunit.client.0.smithi012.stdout:2/186: mknod d0/c36 0 2024-03-18T14:50:46.744 INFO:tasks.workunit.client.0.smithi012.stdout:2/187: write d0/d10/d18/d31/f35 [5140470,52509] 0 2024-03-18T14:50:46.744 INFO:tasks.workunit.client.0.smithi012.stdout:6/184: getdents d17/d19 0 2024-03-18T14:50:46.746 INFO:tasks.workunit.client.0.smithi012.stdout:2/188: creat d0/d10/d18/d33/f37 x:0 0 0 2024-03-18T14:50:46.746 INFO:tasks.workunit.client.0.smithi012.stdout:2/189: chown d0/l3 2 1 2024-03-18T14:50:46.749 INFO:tasks.workunit.client.0.smithi012.stdout:2/190: rename d0/d10/l23 to d0/d10/d18/d31/l38 0 2024-03-18T14:50:46.749 INFO:tasks.workunit.client.0.smithi012.stdout:2/191: chown d0/d10/d18/d31/l34 3255 1 2024-03-18T14:50:46.749 INFO:tasks.workunit.client.0.smithi012.stdout:2/192: chown d0/c9 199986 1 2024-03-18T14:50:46.752 INFO:tasks.workunit.client.0.smithi012.stdout:9/237: getdents d3/d1e 0 2024-03-18T14:50:46.752 INFO:tasks.workunit.client.0.smithi012.stdout:9/238: chown d3/f12 12 1 2024-03-18T14:50:46.754 INFO:tasks.workunit.client.0.smithi012.stdout:9/239: unlink d3/d9/d19/c34 0 2024-03-18T14:50:46.760 INFO:tasks.workunit.client.0.smithi012.stdout:6/185: dread f11 [0,4194304] 0 2024-03-18T14:50:46.761 INFO:tasks.workunit.client.0.smithi012.stdout:1/173: sync 2024-03-18T14:50:46.761 INFO:tasks.workunit.client.0.smithi012.stdout:1/174: chown d8/d1e 1766603001 1 2024-03-18T14:50:46.773 INFO:tasks.workunit.client.0.smithi012.stdout:5/202: dwrite d3/d15/f1b [0,4194304] 0 2024-03-18T14:50:46.774 INFO:tasks.workunit.client.0.smithi012.stdout:4/147: dwrite d6/d1a/f2d [0,4194304] 0 2024-03-18T14:50:46.774 INFO:tasks.workunit.client.0.smithi012.stdout:9/240: dwrite d3/d20/f43 [4194304,4194304] 0 2024-03-18T14:50:46.776 INFO:tasks.workunit.client.0.smithi012.stdout:5/203: rename d3/f3b to d3/d29/d35/f48 0 2024-03-18T14:50:46.779 INFO:tasks.workunit.client.0.smithi012.stdout:4/148: creat d6/f33 x:0 0 0 2024-03-18T14:50:46.779 INFO:tasks.workunit.client.0.smithi012.stdout:4/149: creat d6/d27/f34 x:0 0 0 2024-03-18T14:50:46.781 INFO:tasks.workunit.client.0.smithi012.stdout:4/150: link f1 d6/d1a/f35 0 2024-03-18T14:50:46.782 INFO:tasks.workunit.client.0.smithi012.stdout:4/151: symlink d6/d1a/d30/l36 0 2024-03-18T14:50:46.782 INFO:tasks.workunit.client.0.smithi012.stdout:1/175: dwrite d8/fc [0,4194304] 0 2024-03-18T14:50:46.784 INFO:tasks.workunit.client.0.smithi012.stdout:1/176: getdents d8/d1e 0 2024-03-18T14:50:46.785 INFO:tasks.workunit.client.0.smithi012.stdout:1/177: mknod d8/db/d10/d16/d1c/d2e/c37 0 2024-03-18T14:50:46.785 INFO:tasks.workunit.client.0.smithi012.stdout:1/178: creat d8/d1e/d27/f38 x:0 0 0 2024-03-18T14:50:46.787 INFO:tasks.workunit.client.0.smithi012.stdout:1/179: mkdir d8/db/d10/d16/d1c/d2e/d39 0 2024-03-18T14:50:46.789 INFO:tasks.workunit.client.0.smithi012.stdout:4/152: dread d6/f12 [0,4194304] 0 2024-03-18T14:50:46.789 INFO:tasks.workunit.client.0.smithi012.stdout:1/180: unlink d8/db/d10/d16/d1c/f2a 0 2024-03-18T14:50:46.792 INFO:tasks.workunit.client.0.smithi012.stdout:4/153: rmdir d6/d19 39 2024-03-18T14:50:46.794 INFO:tasks.workunit.client.0.smithi012.stdout:5/204: dwrite d3/d29/d35/f48 [0,4194304] 0 2024-03-18T14:50:46.794 INFO:tasks.workunit.client.0.smithi012.stdout:5/205: stat d3/d1e/l12 0 2024-03-18T14:50:46.794 INFO:tasks.workunit.client.0.smithi012.stdout:5/206: truncate d3/d29/f41 666770 0 2024-03-18T14:50:46.794 INFO:tasks.workunit.client.0.smithi012.stdout:5/207: read d3/d29/f41 [430935,90908] 0 2024-03-18T14:50:46.798 INFO:tasks.workunit.client.0.smithi012.stdout:5/208: creat d3/d15/d21/f49 x:0 0 0 2024-03-18T14:50:46.799 INFO:tasks.workunit.client.0.smithi012.stdout:1/181: dread d8/db/fd [0,4194304] 0 2024-03-18T14:50:46.800 INFO:tasks.workunit.client.0.smithi012.stdout:5/209: creat d3/d29/d36/d3d/f4a x:0 0 0 2024-03-18T14:50:46.800 INFO:tasks.workunit.client.0.smithi012.stdout:8/213: sync 2024-03-18T14:50:46.801 INFO:tasks.workunit.client.0.smithi012.stdout:8/214: mkdir d0/d47 0 2024-03-18T14:50:46.802 INFO:tasks.workunit.client.0.smithi012.stdout:8/215: symlink d0/d18/d19/d1b/l48 0 2024-03-18T14:50:46.804 INFO:tasks.workunit.client.0.smithi012.stdout:8/216: symlink d0/d18/d19/d1b/l49 0 2024-03-18T14:50:46.805 INFO:tasks.workunit.client.0.smithi012.stdout:1/182: write d8/db/d10/f13 [1718193,64102] 0 2024-03-18T14:50:46.805 INFO:tasks.workunit.client.0.smithi012.stdout:8/217: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:46.809 INFO:tasks.workunit.client.0.smithi012.stdout:8/218: unlink d0/d12/d2c/f2f 0 2024-03-18T14:50:46.816 INFO:tasks.workunit.client.0.smithi012.stdout:5/210: dread d3/d1e/f28 [0,4194304] 0 2024-03-18T14:50:46.816 INFO:tasks.workunit.client.0.smithi012.stdout:4/154: dwrite d6/f28 [0,4194304] 0 2024-03-18T14:50:46.817 INFO:tasks.workunit.client.0.smithi012.stdout:4/155: readlink d6/d1a/d30/l36 0 2024-03-18T14:50:46.817 INFO:tasks.workunit.client.0.smithi012.stdout:4/156: readlink d6/d1a/d30/l36 0 2024-03-18T14:50:46.817 INFO:tasks.workunit.client.0.smithi012.stdout:5/211: dread d3/d15/f37 [0,4194304] 0 2024-03-18T14:50:46.817 INFO:tasks.workunit.client.0.smithi012.stdout:5/212: creat d3/d1e/d33/f4b x:0 0 0 2024-03-18T14:50:46.818 INFO:tasks.workunit.client.0.smithi012.stdout:4/157: mkdir d6/d1a/d37 0 2024-03-18T14:50:46.819 INFO:tasks.workunit.client.0.smithi012.stdout:4/158: mknod d6/d1a/c38 0 2024-03-18T14:50:46.820 INFO:tasks.workunit.client.0.smithi012.stdout:4/159: mkdir d6/d19/d39 0 2024-03-18T14:50:46.821 INFO:tasks.workunit.client.0.smithi012.stdout:4/160: symlink d6/l3a 0 2024-03-18T14:50:46.825 INFO:tasks.workunit.client.0.smithi012.stdout:4/161: dread d6/da/f13 [0,4194304] 0 2024-03-18T14:50:46.826 INFO:tasks.workunit.client.0.smithi012.stdout:4/162: creat d6/da/f3b x:0 0 0 2024-03-18T14:50:46.826 INFO:tasks.workunit.client.0.smithi012.stdout:1/183: dwrite f5 [4194304,4194304] 0 2024-03-18T14:50:46.827 INFO:tasks.workunit.client.0.smithi012.stdout:8/219: dwrite d0/f35 [0,4194304] 0 2024-03-18T14:50:46.828 INFO:tasks.workunit.client.0.smithi012.stdout:1/184: mknod d8/db/d10/c3a 0 2024-03-18T14:50:46.830 INFO:tasks.workunit.client.0.smithi012.stdout:5/213: read d3/d29/d35/f48 [1974373,80643] 0 2024-03-18T14:50:46.830 INFO:tasks.workunit.client.0.smithi012.stdout:5/214: readlink d3/d15/d21/l2a 0 2024-03-18T14:50:46.831 INFO:tasks.workunit.client.0.smithi012.stdout:4/163: read d6/d19/f2f [233124,130314] 0 2024-03-18T14:50:46.831 INFO:tasks.workunit.client.0.smithi012.stdout:4/164: chown d6/d1a/f2a 122786 1 2024-03-18T14:50:46.831 INFO:tasks.workunit.client.0.smithi012.stdout:8/220: creat d0/d18/d19/d1b/d2a/d3e/f4a x:0 0 0 2024-03-18T14:50:46.832 INFO:tasks.workunit.client.0.smithi012.stdout:8/221: dread - d0/d12/d33/f44 zero size 2024-03-18T14:50:46.833 INFO:tasks.workunit.client.0.smithi012.stdout:4/165: truncate d6/f25 4191676 0 2024-03-18T14:50:46.833 INFO:tasks.workunit.client.0.smithi012.stdout:4/166: chown d6/d19/d39 7 1 2024-03-18T14:50:46.833 INFO:tasks.workunit.client.0.smithi012.stdout:4/167: read - d6/d1a/f31 zero size 2024-03-18T14:50:46.834 INFO:tasks.workunit.client.0.smithi012.stdout:4/168: truncate d6/d1a/f1b 1032400 0 2024-03-18T14:50:46.834 INFO:tasks.workunit.client.0.smithi012.stdout:4/169: truncate d6/d1a/f31 476913 0 2024-03-18T14:50:46.834 INFO:tasks.workunit.client.0.smithi012.stdout:4/170: mknod d6/d1a/c3c 0 2024-03-18T14:50:46.840 INFO:tasks.workunit.client.0.smithi012.stdout:1/185: write d8/db/ff [584964,31053] 0 2024-03-18T14:50:46.841 INFO:tasks.workunit.client.0.smithi012.stdout:8/222: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:46.844 INFO:tasks.workunit.client.0.smithi012.stdout:8/223: rename d0/f35 to d0/d12/f4b 0 2024-03-18T14:50:46.844 INFO:tasks.workunit.client.0.smithi012.stdout:8/224: readlink d0/d18/l45 0 2024-03-18T14:50:46.844 INFO:tasks.workunit.client.0.smithi012.stdout:8/225: fdatasync d0/d12/f20 0 2024-03-18T14:50:46.845 INFO:tasks.workunit.client.0.smithi012.stdout:8/226: rename d0/c32 to d0/d47/c4c 0 2024-03-18T14:50:46.847 INFO:tasks.workunit.client.0.smithi012.stdout:8/227: unlink d0/d18/l45 0 2024-03-18T14:50:46.848 INFO:tasks.workunit.client.0.smithi012.stdout:8/228: dread - d0/d12/d33/f44 zero size 2024-03-18T14:50:46.852 INFO:tasks.workunit.client.0.smithi012.stdout:1/186: dread f5 [4194304,4194304] 0 2024-03-18T14:50:46.855 INFO:tasks.workunit.client.0.smithi012.stdout:5/215: dwrite d3/d15/f1c [0,4194304] 0 2024-03-18T14:50:46.856 INFO:tasks.workunit.client.0.smithi012.stdout:1/187: rmdir d8/db/d10/d16 39 2024-03-18T14:50:46.856 INFO:tasks.workunit.client.0.smithi012.stdout:1/188: chown d8/d1e/d27/f38 18588 1 2024-03-18T14:50:46.859 INFO:tasks.workunit.client.0.smithi012.stdout:5/216: rename d3/d1e/l26 to d3/d15/l4c 0 2024-03-18T14:50:46.860 INFO:tasks.workunit.client.0.smithi012.stdout:5/217: mkdir d3/d15/d4d 0 2024-03-18T14:50:46.863 INFO:tasks.workunit.client.0.smithi012.stdout:4/171: dwrite d6/d1a/f31 [0,4194304] 0 2024-03-18T14:50:46.867 INFO:tasks.workunit.client.0.smithi012.stdout:8/229: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:46.867 INFO:tasks.workunit.client.0.smithi012.stdout:8/230: write d0/d12/f3c [449787,19764] 0 2024-03-18T14:50:46.870 INFO:tasks.workunit.client.0.smithi012.stdout:8/231: dread d0/d12/d2c/f37 [0,4194304] 0 2024-03-18T14:50:46.876 INFO:tasks.workunit.client.0.smithi012.stdout:4/172: write d6/d1a/f31 [4444776,21906] 0 2024-03-18T14:50:46.878 INFO:tasks.workunit.client.0.smithi012.stdout:4/173: rename d6/d1a/c38 to d6/d19/c3d 0 2024-03-18T14:50:46.878 INFO:tasks.workunit.client.0.smithi012.stdout:4/174: chown d6/d1a/f31 56 1 2024-03-18T14:50:46.878 INFO:tasks.workunit.client.0.smithi012.stdout:4/175: write d6/f12 [235482,30548] 0 2024-03-18T14:50:46.882 INFO:tasks.workunit.client.0.smithi012.stdout:5/218: dwrite f2 [0,4194304] 0 2024-03-18T14:50:46.884 INFO:tasks.workunit.client.0.smithi012.stdout:5/219: rename d3/d15/d4d to d3/d29/d36/d4e 0 2024-03-18T14:50:46.895 INFO:tasks.workunit.client.0.smithi012.stdout:8/232: dwrite d0/d18/f1d [0,4194304] 0 2024-03-18T14:50:46.898 INFO:tasks.workunit.client.0.smithi012.stdout:8/233: unlink d0/d18/l40 0 2024-03-18T14:50:46.908 INFO:tasks.workunit.client.0.smithi012.stdout:8/234: dread d0/d12/f4b [0,4194304] 0 2024-03-18T14:50:46.912 INFO:tasks.workunit.client.0.smithi012.stdout:5/220: dwrite d3/d29/d35/f48 [0,4194304] 0 2024-03-18T14:50:46.912 INFO:tasks.workunit.client.0.smithi012.stdout:4/176: dwrite d6/d1a/f31 [4194304,4194304] 0 2024-03-18T14:50:46.913 INFO:tasks.workunit.client.0.smithi012.stdout:4/177: write d6/f21 [642943,32556] 0 2024-03-18T14:50:46.927 INFO:tasks.workunit.client.0.smithi012.stdout:4/178: write d6/d1a/f26 [2342923,16714] 0 2024-03-18T14:50:46.928 INFO:tasks.workunit.client.0.smithi012.stdout:4/179: creat d6/d19/f3e x:0 0 0 2024-03-18T14:50:46.929 INFO:tasks.workunit.client.0.smithi012.stdout:4/180: truncate d6/f11 836859 0 2024-03-18T14:50:46.931 INFO:tasks.workunit.client.0.smithi012.stdout:4/181: rmdir d6/d1a/d37 0 2024-03-18T14:50:46.932 INFO:tasks.workunit.client.0.smithi012.stdout:4/182: symlink d6/da/l3f 0 2024-03-18T14:50:46.932 INFO:tasks.workunit.client.0.smithi012.stdout:4/183: fsync d6/da/f18 0 2024-03-18T14:50:46.935 INFO:tasks.workunit.client.0.smithi012.stdout:8/235: dwrite d0/f7 [0,4194304] 0 2024-03-18T14:50:46.935 INFO:tasks.workunit.client.0.smithi012.stdout:8/236: fsync d0/d12/d33/f44 0 2024-03-18T14:50:46.935 INFO:tasks.workunit.client.0.smithi012.stdout:8/237: creat d0/d12/f4d x:0 0 0 2024-03-18T14:50:46.936 INFO:tasks.workunit.client.0.smithi012.stdout:5/221: dwrite d3/d29/d35/f48 [4194304,4194304] 0 2024-03-18T14:50:46.936 INFO:tasks.workunit.client.0.smithi012.stdout:5/222: dread - d3/d1e/d33/f4b zero size 2024-03-18T14:50:46.938 INFO:tasks.workunit.client.0.smithi012.stdout:5/223: dread d3/f22 [0,4194304] 0 2024-03-18T14:50:46.940 INFO:tasks.workunit.client.0.smithi012.stdout:5/224: read d3/d15/f31 [104812,126569] 0 2024-03-18T14:50:46.943 INFO:tasks.workunit.client.0.smithi012.stdout:5/225: dread d3/d15/f31 [0,4194304] 0 2024-03-18T14:50:46.943 INFO:tasks.workunit.client.0.smithi012.stdout:5/226: dread - d3/d15/d21/f49 zero size 2024-03-18T14:50:46.944 INFO:tasks.workunit.client.0.smithi012.stdout:4/184: dread d6/da/f18 [0,4194304] 0 2024-03-18T14:50:46.944 INFO:tasks.workunit.client.0.smithi012.stdout:4/185: write d6/d19/f1c [930410,89869] 0 2024-03-18T14:50:46.945 INFO:tasks.workunit.client.0.smithi012.stdout:4/186: stat d6/fe 0 2024-03-18T14:50:46.947 INFO:tasks.workunit.client.0.smithi012.stdout:8/238: write d0/d15/f38 [591073,123686] 0 2024-03-18T14:50:46.949 INFO:tasks.workunit.client.0.smithi012.stdout:8/239: mkdir d0/d15/d4e 0 2024-03-18T14:50:46.951 INFO:tasks.workunit.client.0.smithi012.stdout:8/240: rename d0/d12/f1e to d0/d47/f4f 0 2024-03-18T14:50:46.951 INFO:tasks.workunit.client.0.smithi012.stdout:8/241: write d0/d15/f1a [778999,20249] 0 2024-03-18T14:50:46.951 INFO:tasks.workunit.client.0.smithi012.stdout:8/242: mkdir d0/d12/d33/d50 0 2024-03-18T14:50:46.954 INFO:tasks.workunit.client.0.smithi012.stdout:8/243: dread d0/d18/f25 [0,4194304] 0 2024-03-18T14:50:46.955 INFO:tasks.workunit.client.0.smithi012.stdout:7/193: sync 2024-03-18T14:50:46.955 INFO:tasks.workunit.client.0.smithi012.stdout:3/131: sync 2024-03-18T14:50:46.955 INFO:tasks.workunit.client.0.smithi012.stdout:0/180: sync 2024-03-18T14:50:46.956 INFO:tasks.workunit.client.0.smithi012.stdout:0/181: write d1/d16/d31/f1d [673245,14331] 0 2024-03-18T14:50:46.956 INFO:tasks.workunit.client.0.smithi012.stdout:0/182: chown d1/d16/d19/c1e 5 1 2024-03-18T14:50:46.957 INFO:tasks.workunit.client.0.smithi012.stdout:7/194: truncate fa 2682373 0 2024-03-18T14:50:46.957 INFO:tasks.workunit.client.0.smithi012.stdout:7/195: fsync f5 0 2024-03-18T14:50:46.957 INFO:tasks.workunit.client.0.smithi012.stdout:7/196: creat dd/f45 x:0 0 0 2024-03-18T14:50:46.959 INFO:tasks.workunit.client.0.smithi012.stdout:4/187: truncate d6/d19/f1c 346379 0 2024-03-18T14:50:46.960 INFO:tasks.workunit.client.0.smithi012.stdout:0/183: creat d1/d20/d23/f46 x:0 0 0 2024-03-18T14:50:46.961 INFO:tasks.workunit.client.0.smithi012.stdout:0/184: creat d1/d16/d19/f47 x:0 0 0 2024-03-18T14:50:46.962 INFO:tasks.workunit.client.0.smithi012.stdout:0/185: write d1/d16/d18/ff [1289442,24194] 0 2024-03-18T14:50:46.963 INFO:tasks.workunit.client.0.smithi012.stdout:7/197: unlink dd/f2e 0 2024-03-18T14:50:46.963 INFO:tasks.workunit.client.0.smithi012.stdout:7/198: chown dd/d13/d23/d33/d37/l40 407492 1 2024-03-18T14:50:46.965 INFO:tasks.workunit.client.0.smithi012.stdout:4/188: rename d6/f11 to d6/d19/d39/f40 0 2024-03-18T14:50:46.968 INFO:tasks.workunit.client.0.smithi012.stdout:0/186: link d1/d16/d19/c2b d1/c48 0 2024-03-18T14:50:46.968 INFO:tasks.workunit.client.0.smithi012.stdout:0/187: write d1/fa [5248,126224] 0 2024-03-18T14:50:46.968 INFO:tasks.workunit.client.0.smithi012.stdout:7/199: write f5 [2551807,131043] 0 2024-03-18T14:50:46.969 INFO:tasks.workunit.client.0.smithi012.stdout:3/132: write dc/f1f [1082753,12547] 0 2024-03-18T14:50:46.970 INFO:tasks.workunit.client.0.smithi012.stdout:4/189: creat d6/d19/d39/f41 x:0 0 0 2024-03-18T14:50:46.971 INFO:tasks.workunit.client.0.smithi012.stdout:8/244: dwrite d0/fa [4194304,4194304] 0 2024-03-18T14:50:46.972 INFO:tasks.workunit.client.0.smithi012.stdout:5/227: dwrite d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:46.972 INFO:tasks.workunit.client.0.smithi012.stdout:5/228: write d3/d1e/d33/f4b [834021,54336] 0 2024-03-18T14:50:46.972 INFO:tasks.workunit.client.0.smithi012.stdout:5/229: write d3/d1e/f1a [1453180,112749] 0 2024-03-18T14:50:46.972 INFO:tasks.workunit.client.0.smithi012.stdout:7/200: symlink dd/d13/d23/d33/l46 0 2024-03-18T14:50:46.978 INFO:tasks.workunit.client.0.smithi012.stdout:3/133: dread dc/f1f [0,4194304] 0 2024-03-18T14:50:46.980 INFO:tasks.workunit.client.0.smithi012.stdout:7/201: unlink dd/d13/d23/d2f/l3b 0 2024-03-18T14:50:46.982 INFO:tasks.workunit.client.0.smithi012.stdout:8/245: dread d0/fa [4194304,4194304] 0 2024-03-18T14:50:46.982 INFO:tasks.workunit.client.0.smithi012.stdout:8/246: chown d0/d18/d19/d1b/l48 67950288 1 2024-03-18T14:50:46.982 INFO:tasks.workunit.client.0.smithi012.stdout:8/247: readlink d0/d18/d19/d1b/l48 0 2024-03-18T14:50:46.984 INFO:tasks.workunit.client.0.smithi012.stdout:4/190: read d6/f28 [38933,59395] 0 2024-03-18T14:50:46.984 INFO:tasks.workunit.client.0.smithi012.stdout:4/191: creat d6/d19/f42 x:0 0 0 2024-03-18T14:50:46.985 INFO:tasks.workunit.client.0.smithi012.stdout:3/134: creat dc/f2b x:0 0 0 2024-03-18T14:50:46.987 INFO:tasks.workunit.client.0.smithi012.stdout:0/188: dwrite d1/d20/d23/d3b/f2c [0,4194304] 0 2024-03-18T14:50:46.987 INFO:tasks.workunit.client.0.smithi012.stdout:0/189: truncate d1/d20/f2e 1255074 0 2024-03-18T14:50:46.988 INFO:tasks.workunit.client.0.smithi012.stdout:8/248: symlink d0/d15/l51 0 2024-03-18T14:50:46.990 INFO:tasks.workunit.client.0.smithi012.stdout:0/190: dread d1/d20/d23/d26/f36 [0,4194304] 0 2024-03-18T14:50:46.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/135: creat dc/d29/f2c x:0 0 0 2024-03-18T14:50:46.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/136: write dc/f26 [1272301,12553] 0 2024-03-18T14:50:47.001 INFO:tasks.workunit.client.0.smithi012.stdout:8/249: rename d0/l13 to d0/d15/l52 0 2024-03-18T14:50:47.001 INFO:tasks.workunit.client.0.smithi012.stdout:8/250: chown d0/d12/d2c/f37 1 1 2024-03-18T14:50:47.013 INFO:tasks.workunit.client.0.smithi012.stdout:7/202: dwrite fa [0,4194304] 0 2024-03-18T14:50:47.013 INFO:tasks.workunit.client.0.smithi012.stdout:7/203: stat dd/d13/d23/d33/d37/c3f 0 2024-03-18T14:50:47.025 INFO:tasks.workunit.client.0.smithi012.stdout:0/191: dwrite d1/d16/d31/f35 [0,4194304] 0 2024-03-18T14:50:47.025 INFO:tasks.workunit.client.0.smithi012.stdout:0/192: chown d1/d20/f2e 61697 1 2024-03-18T14:50:47.032 INFO:tasks.workunit.client.0.smithi012.stdout:8/251: dwrite d0/d18/d19/d1b/f27 [0,4194304] 0 2024-03-18T14:50:47.032 INFO:tasks.workunit.client.0.smithi012.stdout:8/252: creat d0/d12/d2c/f53 x:0 0 0 2024-03-18T14:50:47.032 INFO:tasks.workunit.client.0.smithi012.stdout:4/192: dwrite d6/ff [0,4194304] 0 2024-03-18T14:50:47.036 INFO:tasks.workunit.client.0.smithi012.stdout:8/253: getdents d0/d18/d19/d1b 0 2024-03-18T14:50:47.037 INFO:tasks.workunit.client.0.smithi012.stdout:0/193: dread d1/f3 [0,4194304] 0 2024-03-18T14:50:47.037 INFO:tasks.workunit.client.0.smithi012.stdout:7/204: dwrite dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:47.037 INFO:tasks.workunit.client.0.smithi012.stdout:0/194: stat d1/d16/d19/f2a 0 2024-03-18T14:50:47.038 INFO:tasks.workunit.client.0.smithi012.stdout:0/195: chown d1/f10 349880 1 2024-03-18T14:50:47.040 INFO:tasks.workunit.client.0.smithi012.stdout:7/205: rename dd/d13/d23/d33/d37/c3f to dd/c47 0 2024-03-18T14:50:47.043 INFO:tasks.workunit.client.0.smithi012.stdout:7/206: mkdir dd/d13/d23/d33/d48 0 2024-03-18T14:50:47.043 INFO:tasks.workunit.client.0.smithi012.stdout:7/207: truncate dd/f10 2250023 0 2024-03-18T14:50:47.045 INFO:tasks.workunit.client.0.smithi012.stdout:7/208: creat dd/d13/d23/d33/f49 x:0 0 0 2024-03-18T14:50:47.050 INFO:tasks.workunit.client.0.smithi012.stdout:0/196: dread d1/d16/d19/f37 [0,4194304] 0 2024-03-18T14:50:47.054 INFO:tasks.workunit.client.0.smithi012.stdout:7/209: write dd/d13/d23/f2b [3485373,83353] 0 2024-03-18T14:50:47.057 INFO:tasks.workunit.client.0.smithi012.stdout:7/210: rename dd/l16 to dd/d13/d23/d33/d37/l4a 0 2024-03-18T14:50:47.059 INFO:tasks.workunit.client.0.smithi012.stdout:7/211: symlink dd/d13/l4b 0 2024-03-18T14:50:47.067 INFO:tasks.workunit.client.0.smithi012.stdout:4/193: dwrite d6/d1a/f31 [4194304,4194304] 0 2024-03-18T14:50:47.067 INFO:tasks.workunit.client.0.smithi012.stdout:4/194: stat d6/d19/d39/f41 0 2024-03-18T14:50:47.067 INFO:tasks.workunit.client.0.smithi012.stdout:4/195: write d6/f33 [439410,61469] 0 2024-03-18T14:50:47.068 INFO:tasks.workunit.client.0.smithi012.stdout:4/196: getdents d6/da 0 2024-03-18T14:50:47.068 INFO:tasks.workunit.client.0.smithi012.stdout:4/197: write d6/d19/f3e [964860,5489] 0 2024-03-18T14:50:47.073 INFO:tasks.workunit.client.0.smithi012.stdout:0/197: dwrite d1/d20/d23/d26/f36 [0,4194304] 0 2024-03-18T14:50:47.078 INFO:tasks.workunit.client.0.smithi012.stdout:0/198: rename d1/d16/d31/c3c to d1/d20/c49 0 2024-03-18T14:50:47.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/199: write d1/f10 [727154,6101] 0 2024-03-18T14:50:47.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/200: fdatasync d1/d16/d19/f37 0 2024-03-18T14:50:47.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/201: chown d1/d20/d23/d26/f40 3803 1 2024-03-18T14:50:47.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/202: stat d1/d20/l43 0 2024-03-18T14:50:47.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/203: mknod d1/d16/d19/c4a 0 2024-03-18T14:50:47.080 INFO:tasks.workunit.client.0.smithi012.stdout:0/204: mknod d1/d16/d19/c4b 0 2024-03-18T14:50:47.080 INFO:tasks.workunit.client.0.smithi012.stdout:7/212: dwrite dd/d13/d23/d33/f49 [0,4194304] 0 2024-03-18T14:50:47.083 INFO:tasks.workunit.client.0.smithi012.stdout:0/205: rename d1/d2d/c44 to d1/d16/c4c 0 2024-03-18T14:50:47.083 INFO:tasks.workunit.client.0.smithi012.stdout:0/206: fsync d1/d16/d18/ff 0 2024-03-18T14:50:47.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/198: dwrite f1 [0,4194304] 0 2024-03-18T14:50:47.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/199: truncate d6/d1a/f2d 4996346 0 2024-03-18T14:50:47.091 INFO:tasks.workunit.client.0.smithi012.stdout:7/213: write dd/d13/d23/d33/f49 [323310,52635] 0 2024-03-18T14:50:47.105 INFO:tasks.workunit.client.0.smithi012.stdout:2/193: sync 2024-03-18T14:50:47.105 INFO:tasks.workunit.client.0.smithi012.stdout:2/194: fsync d0/d10/d18/f1f 0 2024-03-18T14:50:47.109 INFO:tasks.workunit.client.0.smithi012.stdout:0/207: dwrite d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:47.110 INFO:tasks.workunit.client.0.smithi012.stdout:0/208: mkdir d1/d20/d4d 0 2024-03-18T14:50:47.116 INFO:tasks.workunit.client.0.smithi012.stdout:4/200: dwrite d6/d19/f1c [0,4194304] 0 2024-03-18T14:50:47.116 INFO:tasks.workunit.client.0.smithi012.stdout:4/201: rename d6/d1a/d30 to d6/d1a/d30/d43 22 2024-03-18T14:50:47.118 INFO:tasks.workunit.client.0.smithi012.stdout:4/202: creat d6/f44 x:0 0 0 2024-03-18T14:50:47.120 INFO:tasks.workunit.client.0.smithi012.stdout:7/214: dwrite dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:50:47.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/195: dread d0/d10/f16 [0,4194304] 0 2024-03-18T14:50:47.121 INFO:tasks.workunit.client.0.smithi012.stdout:0/209: dread d1/f9 [0,4194304] 0 2024-03-18T14:50:47.122 INFO:tasks.workunit.client.0.smithi012.stdout:4/203: mknod d6/d19/c45 0 2024-03-18T14:50:47.127 INFO:tasks.workunit.client.0.smithi012.stdout:0/210: mkdir d1/d20/d4e 0 2024-03-18T14:50:47.129 INFO:tasks.workunit.client.0.smithi012.stdout:0/211: link d1/d16/d19/f47 d1/f4f 0 2024-03-18T14:50:47.129 INFO:tasks.workunit.client.0.smithi012.stdout:0/212: creat d1/d16/d31/f50 x:0 0 0 2024-03-18T14:50:47.146 INFO:tasks.workunit.client.0.smithi012.stdout:2/196: dwrite d0/d10/f2a [0,4194304] 0 2024-03-18T14:50:47.151 INFO:tasks.workunit.client.0.smithi012.stdout:4/204: dwrite d6/d19/f3e [0,4194304] 0 2024-03-18T14:50:47.152 INFO:tasks.workunit.client.0.smithi012.stdout:0/213: dwrite d1/fa [0,4194304] 0 2024-03-18T14:50:47.154 INFO:tasks.workunit.client.0.smithi012.stdout:4/205: mkdir d6/d1a/d30/d46 0 2024-03-18T14:50:47.166 INFO:tasks.workunit.client.0.smithi012.stdout:0/214: read d1/d16/d31/f35 [2600095,18699] 0 2024-03-18T14:50:47.167 INFO:tasks.workunit.client.0.smithi012.stdout:2/197: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:47.168 INFO:tasks.workunit.client.0.smithi012.stdout:2/198: mknod d0/d10/d18/d31/c39 0 2024-03-18T14:50:47.169 INFO:tasks.workunit.client.0.smithi012.stdout:4/206: dwrite d6/fe [0,4194304] 0 2024-03-18T14:50:47.170 INFO:tasks.workunit.client.0.smithi012.stdout:2/199: creat d0/d10/d18/f3a x:0 0 0 2024-03-18T14:50:47.173 INFO:tasks.workunit.client.0.smithi012.stdout:2/200: symlink d0/d10/l3b 0 2024-03-18T14:50:47.174 INFO:tasks.workunit.client.0.smithi012.stdout:4/207: write d6/d19/d39/f40 [341618,22631] 0 2024-03-18T14:50:47.176 INFO:tasks.workunit.client.0.smithi012.stdout:4/208: creat d6/d1a/d30/f47 x:0 0 0 2024-03-18T14:50:47.180 INFO:tasks.workunit.client.0.smithi012.stdout:4/209: rename d6/d27/f34 to d6/d1a/d30/d46/f48 0 2024-03-18T14:50:47.190 INFO:tasks.workunit.client.0.smithi012.stdout:2/201: dwrite d0/d10/d18/f24 [0,4194304] 0 2024-03-18T14:50:47.191 INFO:tasks.workunit.client.0.smithi012.stdout:2/202: fdatasync d0/f6 0 2024-03-18T14:50:47.193 INFO:tasks.workunit.client.0.smithi012.stdout:4/210: write d6/d19/f3e [1326219,30227] 0 2024-03-18T14:50:47.193 INFO:tasks.workunit.client.0.smithi012.stdout:0/215: dwrite d1/d20/f2e [0,4194304] 0 2024-03-18T14:50:47.193 INFO:tasks.workunit.client.0.smithi012.stdout:4/211: creat d6/d1a/d30/f49 x:0 0 0 2024-03-18T14:50:47.200 INFO:tasks.workunit.client.0.smithi012.stdout:0/216: truncate d1/d16/fe 467819 0 2024-03-18T14:50:47.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/212: rmdir d6/d19/d39 39 2024-03-18T14:50:47.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/213: write d6/d1a/f26 [3923592,71269] 0 2024-03-18T14:50:47.204 INFO:tasks.workunit.client.0.smithi012.stdout:0/217: mkdir d1/d20/d23/d3b/d33/d51 0 2024-03-18T14:50:47.206 INFO:tasks.workunit.client.0.smithi012.stdout:9/241: sync 2024-03-18T14:50:47.206 INFO:tasks.workunit.client.0.smithi012.stdout:1/189: sync 2024-03-18T14:50:47.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/230: sync 2024-03-18T14:50:47.206 INFO:tasks.workunit.client.0.smithi012.stdout:9/242: chown d3/f25 434 1 2024-03-18T14:50:47.206 INFO:tasks.workunit.client.0.smithi012.stdout:6/186: sync 2024-03-18T14:50:47.208 INFO:tasks.workunit.client.0.smithi012.stdout:5/231: dread d3/d15/f31 [0,4194304] 0 2024-03-18T14:50:47.208 INFO:tasks.workunit.client.0.smithi012.stdout:9/243: rename d3/f40 to d3/d1e/f45 0 2024-03-18T14:50:47.209 INFO:tasks.workunit.client.0.smithi012.stdout:6/187: rename d17/d18 to d17/d18/d2d 22 2024-03-18T14:50:47.210 INFO:tasks.workunit.client.0.smithi012.stdout:5/232: rmdir d3/d1e 39 2024-03-18T14:50:47.210 INFO:tasks.workunit.client.0.smithi012.stdout:2/203: dwrite d0/d10/f2a [0,4194304] 0 2024-03-18T14:50:47.211 INFO:tasks.workunit.client.0.smithi012.stdout:9/244: creat d3/d1e/d21/f46 x:0 0 0 2024-03-18T14:50:47.211 INFO:tasks.workunit.client.0.smithi012.stdout:9/245: chown d3/d20/f43 72 1 2024-03-18T14:50:47.214 INFO:tasks.workunit.client.0.smithi012.stdout:2/204: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:47.214 INFO:tasks.workunit.client.0.smithi012.stdout:5/233: creat d3/d15/d42/f4f x:0 0 0 2024-03-18T14:50:47.216 INFO:tasks.workunit.client.0.smithi012.stdout:6/188: dread f11 [0,4194304] 0 2024-03-18T14:50:47.218 INFO:tasks.workunit.client.0.smithi012.stdout:4/214: dread d6/d1a/f2d [0,4194304] 0 2024-03-18T14:50:47.218 INFO:tasks.workunit.client.0.smithi012.stdout:4/215: write d6/f44 [461406,119703] 0 2024-03-18T14:50:47.220 INFO:tasks.workunit.client.0.smithi012.stdout:9/246: rename d3/d1e/d21/f30 to d3/d20/f47 0 2024-03-18T14:50:47.223 INFO:tasks.workunit.client.0.smithi012.stdout:5/234: unlink d3/d29/c38 0 2024-03-18T14:50:47.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/205: mkdir d0/d3c 0 2024-03-18T14:50:47.225 INFO:tasks.workunit.client.0.smithi012.stdout:4/216: mknod d6/d27/c4a 0 2024-03-18T14:50:47.226 INFO:tasks.workunit.client.0.smithi012.stdout:5/235: mknod d3/d29/d36/c50 0 2024-03-18T14:50:47.228 INFO:tasks.workunit.client.0.smithi012.stdout:2/206: unlink d0/d10/d18/f1f 0 2024-03-18T14:50:47.229 INFO:tasks.workunit.client.0.smithi012.stdout:4/217: mknod d6/d19/c4b 0 2024-03-18T14:50:47.230 INFO:tasks.workunit.client.0.smithi012.stdout:4/218: creat d6/f4c x:0 0 0 2024-03-18T14:50:47.232 INFO:tasks.workunit.client.0.smithi012.stdout:4/219: symlink d6/d1a/d30/d46/l4d 0 2024-03-18T14:50:47.232 INFO:tasks.workunit.client.0.smithi012.stdout:4/220: chown d6/f12 2293432 1 2024-03-18T14:50:47.235 INFO:tasks.workunit.client.0.smithi012.stdout:4/221: mknod d6/c4e 0 2024-03-18T14:50:47.235 INFO:tasks.workunit.client.0.smithi012.stdout:1/190: dwrite d8/fa [0,4194304] 0 2024-03-18T14:50:47.237 INFO:tasks.workunit.client.0.smithi012.stdout:1/191: mknod d8/c3b 0 2024-03-18T14:50:47.237 INFO:tasks.workunit.client.0.smithi012.stdout:1/192: creat d8/d1e/d27/f3c x:0 0 0 2024-03-18T14:50:47.241 INFO:tasks.workunit.client.0.smithi012.stdout:2/207: write d0/d10/f2a [1355301,36144] 0 2024-03-18T14:50:47.241 INFO:tasks.workunit.client.0.smithi012.stdout:2/208: fdatasync d0/f2c 0 2024-03-18T14:50:47.241 INFO:tasks.workunit.client.0.smithi012.stdout:2/209: chown d0/c36 4 1 2024-03-18T14:50:47.241 INFO:tasks.workunit.client.0.smithi012.stdout:2/210: chown d0/f2c 166 1 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:50:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:46 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:50:47.245 INFO:tasks.workunit.client.0.smithi012.stdout:5/236: dwrite d3/f43 [0,4194304] 0 2024-03-18T14:50:47.246 INFO:tasks.workunit.client.0.smithi012.stdout:6/189: dwrite d17/f1a [0,4194304] 0 2024-03-18T14:50:47.248 INFO:tasks.workunit.client.0.smithi012.stdout:1/193: dread d8/db/d10/f11 [0,4194304] 0 2024-03-18T14:50:47.250 INFO:tasks.workunit.client.0.smithi012.stdout:5/237: mknod d3/d29/d36/d4e/c51 0 2024-03-18T14:50:47.255 INFO:tasks.workunit.client.0.smithi012.stdout:6/190: mknod d17/c2e 0 2024-03-18T14:50:47.255 INFO:tasks.workunit.client.0.smithi012.stdout:6/191: fsync ff 0 2024-03-18T14:50:47.256 INFO:tasks.workunit.client.0.smithi012.stdout:1/194: truncate f4 1491000 0 2024-03-18T14:50:47.257 INFO:tasks.workunit.client.0.smithi012.stdout:5/238: rename d3/d1e/l40 to d3/l52 0 2024-03-18T14:50:47.257 INFO:tasks.workunit.client.0.smithi012.stdout:5/239: write d3/d15/f1b [4721318,32701] 0 2024-03-18T14:50:47.258 INFO:tasks.workunit.client.0.smithi012.stdout:6/192: mkdir d17/d19/d27/d2f 0 2024-03-18T14:50:47.258 INFO:tasks.workunit.client.0.smithi012.stdout:6/193: chown f12 12 1 2024-03-18T14:50:47.260 INFO:tasks.workunit.client.0.smithi012.stdout:1/195: symlink d8/db/l3d 0 2024-03-18T14:50:47.262 INFO:tasks.workunit.client.0.smithi012.stdout:1/196: mknod d8/db/d10/d16/d1c/d2e/c3e 0 2024-03-18T14:50:47.263 INFO:tasks.workunit.client.0.smithi012.stdout:0/218: truncate d1/d16/fe 299315 0 2024-03-18T14:50:47.264 INFO:tasks.workunit.client.0.smithi012.stdout:6/194: dread f12 [0,4194304] 0 2024-03-18T14:50:47.264 INFO:tasks.workunit.client.0.smithi012.stdout:2/211: dwrite d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:47.264 INFO:tasks.workunit.client.0.smithi012.stdout:2/212: chown d0/d10/l2f 3278683 1 2024-03-18T14:50:47.264 INFO:tasks.workunit.client.0.smithi012.stdout:6/195: dread - d17/d26/f28 zero size 2024-03-18T14:50:47.265 INFO:tasks.workunit.client.0.smithi012.stdout:0/219: creat d1/d20/d4e/f52 x:0 0 0 2024-03-18T14:50:47.266 INFO:tasks.workunit.client.0.smithi012.stdout:4/222: dwrite d6/f28 [0,4194304] 0 2024-03-18T14:50:47.267 INFO:tasks.workunit.client.0.smithi012.stdout:4/223: readlink d6/d1a/d30/d46/l4d 0 2024-03-18T14:50:47.271 INFO:tasks.workunit.client.0.smithi012.stdout:4/224: mkdir d6/d19/d4f 0 2024-03-18T14:50:47.273 INFO:tasks.workunit.client.0.smithi012.stdout:1/197: read d8/db/d10/d16/d1c/f2c [3857561,36618] 0 2024-03-18T14:50:47.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/225: creat d6/d19/d39/f50 x:0 0 0 2024-03-18T14:50:47.274 INFO:tasks.workunit.client.0.smithi012.stdout:4/226: creat d6/d19/f51 x:0 0 0 2024-03-18T14:50:47.275 INFO:tasks.workunit.client.0.smithi012.stdout:1/198: creat d8/db/d10/d16/d1c/d2e/f3f x:0 0 0 2024-03-18T14:50:47.275 INFO:tasks.workunit.client.0.smithi012.stdout:1/199: fsync f4 0 2024-03-18T14:50:47.275 INFO:tasks.workunit.client.0.smithi012.stdout:1/200: write d8/db/d10/f33 [171573,64367] 0 2024-03-18T14:50:47.276 INFO:tasks.workunit.client.0.smithi012.stdout:1/201: write d8/db/ff [2046278,97511] 0 2024-03-18T14:50:47.276 INFO:tasks.workunit.client.0.smithi012.stdout:1/202: chown d8/db/d10/c3a 5071669 1 2024-03-18T14:50:47.278 INFO:tasks.workunit.client.0.smithi012.stdout:0/220: read d1/d16/d19/f2a [3338526,112733] 0 2024-03-18T14:50:47.278 INFO:tasks.workunit.client.0.smithi012.stdout:4/227: mknod d6/d1a/d30/c52 0 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:50:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:46 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:50:47.299 INFO:tasks.workunit.client.0.smithi012.stdout:4/228: write d6/d1a/f31 [4754797,16768] 0 2024-03-18T14:50:47.299 INFO:tasks.workunit.client.0.smithi012.stdout:4/229: dread - d6/f4c zero size 2024-03-18T14:50:47.299 INFO:tasks.workunit.client.0.smithi012.stdout:4/230: chown d6/d19/c1e 101376339 1 2024-03-18T14:50:47.300 INFO:tasks.workunit.client.0.smithi012.stdout:4/231: chown d6/f12 18 1 2024-03-18T14:50:47.312 INFO:tasks.workunit.client.0.smithi012.stdout:6/196: dwrite d17/f1a [0,4194304] 0 2024-03-18T14:50:47.312 INFO:tasks.workunit.client.0.smithi012.stdout:2/213: dwrite d0/f2c [0,4194304] 0 2024-03-18T14:50:47.314 INFO:tasks.workunit.client.0.smithi012.stdout:1/203: dwrite d8/db/f2d [0,4194304] 0 2024-03-18T14:50:47.315 INFO:tasks.workunit.client.0.smithi012.stdout:0/221: dwrite d1/d20/d23/d3b/f2c [0,4194304] 0 2024-03-18T14:50:47.315 INFO:tasks.workunit.client.0.smithi012.stdout:0/222: stat d1/d16/d31/f3f 0 2024-03-18T14:50:47.316 INFO:tasks.workunit.client.0.smithi012.stdout:4/232: dwrite d6/d1a/f29 [0,4194304] 0 2024-03-18T14:50:47.316 INFO:tasks.workunit.client.0.smithi012.stdout:4/233: write d6/d19/f1d [754107,43014] 0 2024-03-18T14:50:47.316 INFO:tasks.workunit.client.0.smithi012.stdout:6/197: link d17/l2a d17/d18/l30 0 2024-03-18T14:50:47.322 INFO:tasks.workunit.client.0.smithi012.stdout:1/204: creat d8/f40 x:0 0 0 2024-03-18T14:50:47.322 INFO:tasks.workunit.client.0.smithi012.stdout:1/205: chown d8/db/c14 1145 1 2024-03-18T14:50:47.327 INFO:tasks.workunit.client.0.smithi012.stdout:6/198: mknod d17/d26/c31 0 2024-03-18T14:50:47.327 INFO:tasks.workunit.client.0.smithi012.stdout:6/199: truncate f12 1042577 0 2024-03-18T14:50:47.329 INFO:tasks.workunit.client.0.smithi012.stdout:4/234: rename d6/l22 to d6/d1a/l53 0 2024-03-18T14:50:47.330 INFO:tasks.workunit.client.0.smithi012.stdout:4/235: creat d6/da/f54 x:0 0 0 2024-03-18T14:50:47.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/236: mknod d6/d1a/c55 0 2024-03-18T14:50:47.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/237: chown d6/da/f13 211 1 2024-03-18T14:50:47.333 INFO:tasks.workunit.client.0.smithi012.stdout:0/223: dwrite d1/d20/d23/f46 [0,4194304] 0 2024-03-18T14:50:47.333 INFO:tasks.workunit.client.0.smithi012.stdout:2/214: dwrite d0/f6 [0,4194304] 0 2024-03-18T14:50:47.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/206: fdatasync d8/db/d10/f33 0 2024-03-18T14:50:47.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/207: read d8/db/d10/d16/d1c/f2b [706337,88742] 0 2024-03-18T14:50:47.336 INFO:tasks.workunit.client.0.smithi012.stdout:1/208: read - d8/db/d10/f34 zero size 2024-03-18T14:50:47.336 INFO:tasks.workunit.client.0.smithi012.stdout:1/209: truncate d8/d1e/d27/f28 442289 0 2024-03-18T14:50:47.338 INFO:tasks.workunit.client.0.smithi012.stdout:1/210: rename d8/d1e/f35 to d8/db/f41 0 2024-03-18T14:50:47.338 INFO:tasks.workunit.client.0.smithi012.stdout:1/211: fdatasync f5 0 2024-03-18T14:50:47.338 INFO:tasks.workunit.client.0.smithi012.stdout:1/212: readlink d8/db/d10/l29 0 2024-03-18T14:50:47.338 INFO:tasks.workunit.client.0.smithi012.stdout:1/213: write d8/db/d10/d16/d1c/d2e/f3f [283666,67562] 0 2024-03-18T14:50:47.339 INFO:tasks.workunit.client.0.smithi012.stdout:8/254: sync 2024-03-18T14:50:47.340 INFO:tasks.workunit.client.0.smithi012.stdout:3/137: sync 2024-03-18T14:50:47.340 INFO:tasks.workunit.client.0.smithi012.stdout:2/215: dread d0/d10/f2a [0,4194304] 0 2024-03-18T14:50:47.341 INFO:tasks.workunit.client.0.smithi012.stdout:8/255: mkdir d0/d18/d19/d1b/d54 0 2024-03-18T14:50:47.343 INFO:tasks.workunit.client.0.smithi012.stdout:2/216: truncate d0/d10/d18/f24 1182279 0 2024-03-18T14:50:47.345 INFO:tasks.workunit.client.0.smithi012.stdout:8/256: dread d0/d12/f20 [0,4194304] 0 2024-03-18T14:50:47.350 INFO:tasks.workunit.client.0.smithi012.stdout:6/200: dwrite fe [4194304,4194304] 0 2024-03-18T14:50:47.350 INFO:tasks.workunit.client.0.smithi012.stdout:8/257: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:47.350 INFO:tasks.workunit.client.0.smithi012.stdout:6/201: truncate f8 3594312 0 2024-03-18T14:50:47.351 INFO:tasks.workunit.client.0.smithi012.stdout:6/202: symlink d17/d19/d27/d2f/l32 0 2024-03-18T14:50:47.351 INFO:tasks.workunit.client.0.smithi012.stdout:6/203: write d17/f1b [966247,28881] 0 2024-03-18T14:50:47.351 INFO:tasks.workunit.client.0.smithi012.stdout:0/224: dread d1/d20/d23/d3b/f2c [0,4194304] 0 2024-03-18T14:50:47.353 INFO:tasks.workunit.client.0.smithi012.stdout:6/204: mkdir d17/d33 0 2024-03-18T14:50:47.355 INFO:tasks.workunit.client.0.smithi012.stdout:0/225: link d1/f9 d1/d20/d23/d26/f53 0 2024-03-18T14:50:47.356 INFO:tasks.workunit.client.0.smithi012.stdout:2/217: dread d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:47.358 INFO:tasks.workunit.client.0.smithi012.stdout:6/205: mkdir d17/d34 0 2024-03-18T14:50:47.358 INFO:tasks.workunit.client.0.smithi012.stdout:2/218: symlink d0/d10/d18/l3d 0 2024-03-18T14:50:47.360 INFO:tasks.workunit.client.0.smithi012.stdout:2/219: symlink d0/d10/d18/l3e 0 2024-03-18T14:50:47.360 INFO:tasks.workunit.client.0.smithi012.stdout:2/220: fsync d0/d10/d18/d31/f35 0 2024-03-18T14:50:47.360 INFO:tasks.workunit.client.0.smithi012.stdout:2/221: fdatasync d0/d10/d18/f1b 0 2024-03-18T14:50:47.361 INFO:tasks.workunit.client.0.smithi012.stdout:8/258: write d0/d18/f25 [1049637,57205] 0 2024-03-18T14:50:47.361 INFO:tasks.workunit.client.0.smithi012.stdout:8/259: chown d0/d18 210329 1 2024-03-18T14:50:47.362 INFO:tasks.workunit.client.0.smithi012.stdout:4/238: dwrite d6/f4c [0,4194304] 0 2024-03-18T14:50:47.365 INFO:tasks.workunit.client.0.smithi012.stdout:8/260: dread d0/d12/f3c [0,4194304] 0 2024-03-18T14:50:47.365 INFO:tasks.workunit.client.0.smithi012.stdout:8/261: stat d0/fa 0 2024-03-18T14:50:47.366 INFO:tasks.workunit.client.0.smithi012.stdout:1/214: dwrite d8/db/d10/f11 [4194304,4194304] 0 2024-03-18T14:50:47.367 INFO:tasks.workunit.client.0.smithi012.stdout:3/138: dwrite dc/f12 [0,4194304] 0 2024-03-18T14:50:47.367 INFO:tasks.workunit.client.0.smithi012.stdout:3/139: truncate dc/f27 4198995 0 2024-03-18T14:50:47.374 INFO:tasks.workunit.client.0.smithi012.stdout:8/262: rename d0/d12/d2c to d0/d18/d55 0 2024-03-18T14:50:47.374 INFO:tasks.workunit.client.0.smithi012.stdout:8/263: write d0/d12/f3c [899228,89274] 0 2024-03-18T14:50:47.374 INFO:tasks.workunit.client.0.smithi012.stdout:8/264: write d0/f10 [495072,57] 0 2024-03-18T14:50:47.374 INFO:tasks.workunit.client.0.smithi012.stdout:8/265: truncate d0/d18/d19/d1b/f30 1449882 0 2024-03-18T14:50:47.375 INFO:tasks.workunit.client.0.smithi012.stdout:3/140: rmdir dc 39 2024-03-18T14:50:47.378 INFO:tasks.workunit.client.0.smithi012.stdout:8/266: link d0/d15/l51 d0/d18/d19/l56 0 2024-03-18T14:50:47.380 INFO:tasks.workunit.client.0.smithi012.stdout:3/141: creat dc/d29/f2d x:0 0 0 2024-03-18T14:50:47.381 INFO:tasks.workunit.client.0.smithi012.stdout:8/267: mkdir d0/d18/d19/d57 0 2024-03-18T14:50:47.393 INFO:tasks.workunit.client.0.smithi012.stdout:0/226: dwrite d1/d16/d18/f13 [0,4194304] 0 2024-03-18T14:50:47.396 INFO:tasks.workunit.client.0.smithi012.stdout:6/206: write f8 [2776269,109720] 0 2024-03-18T14:50:47.398 INFO:tasks.workunit.client.0.smithi012.stdout:1/215: dwrite f4 [0,4194304] 0 2024-03-18T14:50:47.399 INFO:tasks.workunit.client.0.smithi012.stdout:8/268: dwrite d0/d15/f28 [4194304,4194304] 0 2024-03-18T14:50:47.399 INFO:tasks.workunit.client.0.smithi012.stdout:2/222: dwrite d0/d10/d18/f1b [0,4194304] 0 2024-03-18T14:50:47.399 INFO:tasks.workunit.client.0.smithi012.stdout:2/223: write d0/d10/f13 [2502480,7495] 0 2024-03-18T14:50:47.400 INFO:tasks.workunit.client.0.smithi012.stdout:1/216: symlink d8/db/l42 0 2024-03-18T14:50:47.413 INFO:tasks.workunit.client.0.smithi012.stdout:3/142: fsync dc/d29/f2d 0 2024-03-18T14:50:47.416 INFO:tasks.workunit.client.0.smithi012.stdout:3/143: dread dc/ff [0,4194304] 0 2024-03-18T14:50:47.416 INFO:tasks.workunit.client.0.smithi012.stdout:3/144: chown dc/f24 1 1 2024-03-18T14:50:47.416 INFO:tasks.workunit.client.0.smithi012.stdout:3/145: creat dc/d29/f2e x:0 0 0 2024-03-18T14:50:47.416 INFO:tasks.workunit.client.0.smithi012.stdout:3/146: write dc/f12 [9229617,82068] 0 2024-03-18T14:50:47.422 INFO:tasks.workunit.client.0.smithi012.stdout:3/147: mknod dc/d29/c2f 0 2024-03-18T14:50:47.433 INFO:tasks.workunit.client.0.smithi012.stdout:0/227: dwrite d1/d16/d31/f35 [0,4194304] 0 2024-03-18T14:50:47.433 INFO:tasks.workunit.client.0.smithi012.stdout:0/228: readlink d1/d16/d31/l28 0 2024-03-18T14:50:47.434 INFO:tasks.workunit.client.0.smithi012.stdout:8/269: dwrite d0/d18/d19/d1b/d2a/d3e/f4a [0,4194304] 0 2024-03-18T14:50:47.434 INFO:tasks.workunit.client.0.smithi012.stdout:8/270: dread - d0/d12/d33/f44 zero size 2024-03-18T14:50:47.434 INFO:tasks.workunit.client.0.smithi012.stdout:8/271: write d0/d18/d19/d1b/f30 [19705,105390] 0 2024-03-18T14:50:47.434 INFO:tasks.workunit.client.0.smithi012.stdout:8/272: chown d0/d12 478867134 1 2024-03-18T14:50:47.435 INFO:tasks.workunit.client.0.smithi012.stdout:0/229: mkdir d1/d20/d4e/d54 0 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:1/217: dwrite f4 [4194304,4194304] 0 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:1/218: read d8/db/d10/f13 [1033933,116756] 0 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:1/219: chown d8/db/d10/f34 6448 1 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:1/220: read d8/fc [4133397,128731] 0 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:1/221: chown d8/db/l3d 1620463 1 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:2/224: dwrite d0/f8 [0,4194304] 0 2024-03-18T14:50:47.436 INFO:tasks.workunit.client.0.smithi012.stdout:2/225: chown d0/c22 4172782 1 2024-03-18T14:50:47.439 INFO:tasks.workunit.client.0.smithi012.stdout:0/230: mknod d1/d20/c55 0 2024-03-18T14:50:47.442 INFO:tasks.workunit.client.0.smithi012.stdout:1/222: rename d8/db/d10/d16/d1c/d2e/d39 to d8/db/d10/d31/d43 0 2024-03-18T14:50:47.445 INFO:tasks.workunit.client.0.smithi012.stdout:2/226: getdents d0/d10/d18/d33 0 2024-03-18T14:50:47.446 INFO:tasks.workunit.client.0.smithi012.stdout:3/148: dread dc/f12 [4194304,4194304] 0 2024-03-18T14:50:47.447 INFO:tasks.workunit.client.0.smithi012.stdout:2/227: symlink d0/d10/d18/d31/l3f 0 2024-03-18T14:50:47.448 INFO:tasks.workunit.client.0.smithi012.stdout:8/273: dwrite d0/d18/d19/f3a [0,4194304] 0 2024-03-18T14:50:47.453 INFO:tasks.workunit.client.0.smithi012.stdout:2/228: dread d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:47.456 INFO:tasks.workunit.client.0.smithi012.stdout:3/149: read dc/f11 [2862027,45930] 0 2024-03-18T14:50:47.457 INFO:tasks.workunit.client.0.smithi012.stdout:2/229: mkdir d0/d3c/d40 0 2024-03-18T14:50:47.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/150: mkdir dc/d30 0 2024-03-18T14:50:47.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/151: write dc/f24 [472835,118133] 0 2024-03-18T14:50:47.462 INFO:tasks.workunit.client.0.smithi012.stdout:3/152: mknod dc/c31 0 2024-03-18T14:50:47.462 INFO:tasks.workunit.client.0.smithi012.stdout:1/223: dwrite d8/db/f2d [0,4194304] 0 2024-03-18T14:50:47.462 INFO:tasks.workunit.client.0.smithi012.stdout:1/224: truncate d8/db/d10/d16/d1c/f2b 1534077 0 2024-03-18T14:50:47.464 INFO:tasks.workunit.client.0.smithi012.stdout:3/153: getdents dc/d29 0 2024-03-18T14:50:47.464 INFO:tasks.workunit.client.0.smithi012.stdout:3/154: write dc/f14 [804393,81244] 0 2024-03-18T14:50:47.465 INFO:tasks.workunit.client.0.smithi012.stdout:3/155: creat dc/f32 x:0 0 0 2024-03-18T14:50:47.466 INFO:tasks.workunit.client.0.smithi012.stdout:3/156: mknod dc/d29/c33 0 2024-03-18T14:50:47.466 INFO:tasks.workunit.client.0.smithi012.stdout:3/157: dread - dc/d29/f2e zero size 2024-03-18T14:50:47.467 INFO:tasks.workunit.client.0.smithi012.stdout:3/158: symlink dc/d29/l34 0 2024-03-18T14:50:47.467 INFO:tasks.workunit.client.0.smithi012.stdout:3/159: dread - dc/d29/f2c zero size 2024-03-18T14:50:47.467 INFO:tasks.workunit.client.0.smithi012.stdout:7/215: sync 2024-03-18T14:50:47.468 INFO:tasks.workunit.client.0.smithi012.stdout:8/274: dwrite d0/d12/f3c [0,4194304] 0 2024-03-18T14:50:47.468 INFO:tasks.workunit.client.0.smithi012.stdout:8/275: stat d0/l3b 0 2024-03-18T14:50:47.468 INFO:tasks.workunit.client.0.smithi012.stdout:8/276: dread - d0/d18/d55/f53 zero size 2024-03-18T14:50:47.468 INFO:tasks.workunit.client.0.smithi012.stdout:3/160: truncate f6 4517748 0 2024-03-18T14:50:47.474 INFO:tasks.workunit.client.0.smithi012.stdout:2/230: dwrite d0/fa [4194304,4194304] 0 2024-03-18T14:50:47.550 INFO:tasks.workunit.client.0.smithi012.stdout:7/216: creat dd/d13/d23/d2f/f4c x:0 0 0 2024-03-18T14:50:47.552 INFO:tasks.workunit.client.0.smithi012.stdout:8/277: creat d0/d18/d19/d1b/f58 x:0 0 0 2024-03-18T14:50:47.557 INFO:tasks.workunit.client.0.smithi012.stdout:3/161: rename dc/d29 to dc/d30/d35 0 2024-03-18T14:50:47.557 INFO:tasks.workunit.client.0.smithi012.stdout:3/162: write dc/f27 [3708034,92845] 0 2024-03-18T14:50:47.558 INFO:tasks.workunit.client.0.smithi012.stdout:2/231: creat d0/d3c/f41 x:0 0 0 2024-03-18T14:50:47.558 INFO:tasks.workunit.client.0.smithi012.stdout:2/232: fsync d0/d10/d18/f27 0 2024-03-18T14:50:47.559 INFO:tasks.workunit.client.0.smithi012.stdout:2/233: fsync d0/f5 0 2024-03-18T14:50:47.559 INFO:tasks.workunit.client.0.smithi012.stdout:3/163: dread dc/ff [0,4194304] 0 2024-03-18T14:50:47.559 INFO:tasks.workunit.client.0.smithi012.stdout:8/278: mkdir d0/d47/d59 0 2024-03-18T14:50:47.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/234: truncate d0/f21 2957507 0 2024-03-18T14:50:47.562 INFO:tasks.workunit.client.0.smithi012.stdout:3/164: rename dc/c31 to dc/d30/d35/c36 0 2024-03-18T14:50:47.565 INFO:tasks.workunit.client.0.smithi012.stdout:2/235: mknod d0/d10/c42 0 2024-03-18T14:50:47.566 INFO:tasks.workunit.client.0.smithi012.stdout:8/279: write d0/fa [134393,30002] 0 2024-03-18T14:50:47.566 INFO:tasks.workunit.client.0.smithi012.stdout:8/280: chown d0/d18/d19/d1b/f58 5232635 1 2024-03-18T14:50:47.567 INFO:tasks.workunit.client.0.smithi012.stdout:8/281: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:47.567 INFO:tasks.workunit.client.0.smithi012.stdout:2/236: symlink d0/l43 0 2024-03-18T14:50:47.569 INFO:tasks.workunit.client.0.smithi012.stdout:8/282: getdents d0/d47 0 2024-03-18T14:50:47.570 INFO:tasks.workunit.client.0.smithi012.stdout:2/237: truncate d0/f6 3896967 0 2024-03-18T14:50:47.570 INFO:tasks.workunit.client.0.smithi012.stdout:2/238: chown d0/d10/f13 4547 1 2024-03-18T14:50:47.571 INFO:tasks.workunit.client.0.smithi012.stdout:2/239: symlink d0/d3c/l44 0 2024-03-18T14:50:47.571 INFO:tasks.workunit.client.0.smithi012.stdout:8/283: dread d0/d18/d19/d1b/f30 [0,4194304] 0 2024-03-18T14:50:47.573 INFO:tasks.workunit.client.0.smithi012.stdout:2/240: truncate d0/f2c 364418 0 2024-03-18T14:50:47.573 INFO:tasks.workunit.client.0.smithi012.stdout:3/165: getdents dc/d30/d35 0 2024-03-18T14:50:47.576 INFO:tasks.workunit.client.0.smithi012.stdout:3/166: creat dc/d30/f37 x:0 0 0 2024-03-18T14:50:47.579 INFO:tasks.workunit.client.0.smithi012.stdout:3/167: mknod dc/d30/d35/c38 0 2024-03-18T14:50:47.581 INFO:tasks.workunit.client.0.smithi012.stdout:8/284: rename d0/d18 to d0/d5a 0 2024-03-18T14:50:47.582 INFO:tasks.workunit.client.0.smithi012.stdout:2/241: dread d0/d10/d18/f1b [0,4194304] 0 2024-03-18T14:50:47.604 INFO:tasks.workunit.client.0.smithi012.stdout:3/168: dwrite f2 [0,4194304] 0 2024-03-18T14:50:47.604 INFO:tasks.workunit.client.0.smithi012.stdout:8/285: dwrite d0/d5a/f25 [0,4194304] 0 2024-03-18T14:50:47.606 INFO:tasks.workunit.client.0.smithi012.stdout:3/169: truncate dc/f11 3557893 0 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:9/247: sync 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:5/240: sync 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:9/248: chown d3/d1e/d21/d35/l3d 23058 1 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:9/249: fsync d3/d9/f13 0 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:5/241: write d3/d1e/d33/f34 [901423,110673] 0 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:8/286: symlink d0/d5a/d19/d1b/l5b 0 2024-03-18T14:50:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:8/287: chown d0/d5a/d19/d1b/c2b 3515 1 2024-03-18T14:50:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/288: rename d0/d5a/d19/f3f to d0/d12/d33/f5c 0 2024-03-18T14:50:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/289: readlink d0/d15/l52 0 2024-03-18T14:50:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:8/290: rename d0/fa to d0/d5a/d19/d1b/d2a/f5d 0 2024-03-18T14:50:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:8/291: dread - d0/d12/f46 zero size 2024-03-18T14:50:47.636 INFO:tasks.workunit.client.0.smithi012.stdout:3/170: dwrite dc/f27 [0,4194304] 0 2024-03-18T14:50:47.636 INFO:tasks.workunit.client.0.smithi012.stdout:9/250: dwrite d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:47.638 INFO:tasks.workunit.client.0.smithi012.stdout:5/242: dwrite d3/d29/f41 [0,4194304] 0 2024-03-18T14:50:47.638 INFO:tasks.workunit.client.0.smithi012.stdout:8/292: dwrite d0/f16 [0,4194304] 0 2024-03-18T14:50:47.639 INFO:tasks.workunit.client.0.smithi012.stdout:3/171: chown dc/c1a 65 1 2024-03-18T14:50:47.644 INFO:tasks.workunit.client.0.smithi012.stdout:3/172: dread dc/f11 [0,4194304] 0 2024-03-18T14:50:47.645 INFO:tasks.workunit.client.0.smithi012.stdout:9/251: rename d3/d9/d2c to d3/d1e/d21/d35/d48 0 2024-03-18T14:50:47.645 INFO:tasks.workunit.client.0.smithi012.stdout:9/252: write d3/d9/f1c [739924,75223] 0 2024-03-18T14:50:47.648 INFO:tasks.workunit.client.0.smithi012.stdout:5/243: symlink d3/d29/d35/l53 0 2024-03-18T14:50:47.652 INFO:tasks.workunit.client.0.smithi012.stdout:8/293: mkdir d0/d5a/d5e 0 2024-03-18T14:50:47.652 INFO:tasks.workunit.client.0.smithi012.stdout:8/294: readlink d0/d5a/l2e 0 2024-03-18T14:50:47.655 INFO:tasks.workunit.client.0.smithi012.stdout:8/295: creat d0/d5a/d19/d1b/d2a/d3e/f5f x:0 0 0 2024-03-18T14:50:47.656 INFO:tasks.workunit.client.0.smithi012.stdout:3/173: rmdir dc/d30 39 2024-03-18T14:50:47.657 INFO:tasks.workunit.client.0.smithi012.stdout:3/174: truncate f2 3336411 0 2024-03-18T14:50:47.660 INFO:tasks.workunit.client.0.smithi012.stdout:3/175: mknod dc/d30/d35/c39 0 2024-03-18T14:50:47.660 INFO:tasks.workunit.client.0.smithi012.stdout:3/176: write dc/f1f [1468843,76538] 0 2024-03-18T14:50:47.670 INFO:tasks.workunit.client.1.smithi100.stderr:++ readlink -f fsstress 2024-03-18T14:50:47.671 INFO:tasks.workunit.client.1.smithi100.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-18T14:50:47.671 INFO:tasks.workunit.client.1.smithi100.stderr:+ popd 2024-03-18T14:50:47.671 INFO:tasks.workunit.client.1.smithi100.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-18T14:50:47.671 INFO:tasks.workunit.client.1.smithi100.stderr:+ popd 2024-03-18T14:50:47.672 INFO:tasks.workunit.client.1.smithi100.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-18T14:50:47.672 INFO:tasks.workunit.client.1.smithi100.stderr:++ mktemp -d -p . 2024-03-18T14:50:47.673 INFO:tasks.workunit.client.1.smithi100.stderr:+ T=./tmp.uLlJmv3TXW 2024-03-18T14:50:47.673 INFO:tasks.workunit.client.1.smithi100.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.uLlJmv3TXW -l 1 -n 1000 -p 10 -v 2024-03-18T14:50:47.674 INFO:tasks.workunit.client.1.smithi100.stdout:seed = 1710364904 2024-03-18T14:50:47.677 INFO:tasks.workunit.client.1.smithi100.stdout:0/0: mknod c0 0 2024-03-18T14:50:47.678 INFO:tasks.workunit.client.1.smithi100.stdout:2/0: fsync - no filename 2024-03-18T14:50:47.678 INFO:tasks.workunit.client.1.smithi100.stdout:2/1: write - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:2/2: truncate - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/0: mkdir d0 0 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/1: readlink - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/2: dwrite - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/3: dread - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/4: read - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/5: dread - no filename 2024-03-18T14:50:47.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/6: read - no filename 2024-03-18T14:50:47.680 INFO:tasks.workunit.client.1.smithi100.stdout:0/1: creat f1 x:0 0 0 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/2: stat f1 0 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/3: stat c0 0 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/4: dread - f1 zero size 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/5: dread - f1 zero size 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/6: dread - f1 zero size 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/7: creat f2 x:0 0 0 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:0/8: creat f3 x:0 0 0 2024-03-18T14:50:47.681 INFO:tasks.workunit.client.1.smithi100.stdout:3/0: write - no filename 2024-03-18T14:50:47.682 INFO:tasks.workunit.client.0.smithi012.stdout:5/244: dwrite d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:47.682 INFO:tasks.workunit.client.0.smithi012.stdout:8/296: dwrite d0/d12/f3c [0,4194304] 0 2024-03-18T14:50:47.682 INFO:tasks.workunit.client.0.smithi012.stdout:3/177: dwrite dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:2/3: creat f0 x:0 0 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:2/4: write f0 [403672,71005] 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:2/5: write f0 [1080259,68732] 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/7: mkdir d0/d1 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/8: unlink - no file 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/9: dwrite - no filename 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/10: read - no filename 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/11: getdents d0/d1 0 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/12: chown d0/d1 495616691 1 2024-03-18T14:50:47.683 INFO:tasks.workunit.client.1.smithi100.stdout:1/13: write - no filename 2024-03-18T14:50:47.686 INFO:tasks.workunit.client.0.smithi012.stdout:5/245: rename d3/d29/d36/c50 to d3/d29/d36/d4e/c54 0 2024-03-18T14:50:47.686 INFO:tasks.workunit.client.0.smithi012.stdout:5/246: chown d3/d1e/l10 2 1 2024-03-18T14:50:47.689 INFO:tasks.workunit.client.1.smithi100.stdout:0/9: mknod c4 0 2024-03-18T14:50:47.690 INFO:tasks.workunit.client.1.smithi100.stdout:0/10: creat f5 x:0 0 0 2024-03-18T14:50:47.690 INFO:tasks.workunit.client.1.smithi100.stdout:0/11: chown f3 191760166 1 2024-03-18T14:50:47.703 INFO:tasks.workunit.client.1.smithi100.stdout:3/1: symlink l0 0 2024-03-18T14:50:47.704 INFO:tasks.workunit.client.1.smithi100.stdout:3/2: truncate - no filename 2024-03-18T14:50:47.704 INFO:tasks.workunit.client.1.smithi100.stdout:3/3: chown l0 737650 1 2024-03-18T14:50:47.704 INFO:tasks.workunit.client.1.smithi100.stdout:3/4: chown l0 800704 1 2024-03-18T14:50:47.704 INFO:tasks.workunit.client.1.smithi100.stdout:3/5: dread - no filename 2024-03-18T14:50:47.704 INFO:tasks.workunit.client.1.smithi100.stdout:3/6: read - no filename 2024-03-18T14:50:47.705 INFO:tasks.workunit.client.0.smithi012.stdout:3/178: stat dc/c13 0 2024-03-18T14:50:47.705 INFO:tasks.workunit.client.0.smithi012.stdout:5/247: mkdir d3/d1e/d33/d55 0 2024-03-18T14:50:47.705 INFO:tasks.workunit.client.0.smithi012.stdout:5/248: chown d3/l25 4 1 2024-03-18T14:50:47.705 INFO:tasks.workunit.client.0.smithi012.stdout:8/297: rename d0/d5a/d19/d1b/f58 to d0/d5a/f60 0 2024-03-18T14:50:47.705 INFO:tasks.workunit.client.0.smithi012.stdout:8/298: dread - d0/d12/d33/f44 zero size 2024-03-18T14:50:47.707 INFO:tasks.workunit.client.0.smithi012.stdout:5/249: mknod d3/d29/c56 0 2024-03-18T14:50:47.707 INFO:tasks.workunit.client.0.smithi012.stdout:5/250: creat d3/d29/f57 x:0 0 0 2024-03-18T14:50:47.710 INFO:tasks.workunit.client.0.smithi012.stdout:8/299: mknod d0/d12/d33/c61 0 2024-03-18T14:50:47.715 INFO:tasks.workunit.client.0.smithi012.stdout:5/251: creat d3/d1e/d33/d55/f58 x:0 0 0 2024-03-18T14:50:47.728 INFO:tasks.workunit.client.1.smithi100.stdout:3/7: unlink l0 0 2024-03-18T14:50:47.728 INFO:tasks.workunit.client.1.smithi100.stdout:3/8: fsync - no filename 2024-03-18T14:50:47.728 INFO:tasks.workunit.client.1.smithi100.stdout:3/9: dread - no filename 2024-03-18T14:50:47.728 INFO:tasks.workunit.client.1.smithi100.stdout:3/10: dwrite - no filename 2024-03-18T14:50:47.729 INFO:tasks.workunit.client.1.smithi100.stdout:1/14: rmdir d0/d1 0 2024-03-18T14:50:47.732 INFO:tasks.workunit.client.1.smithi100.stdout:5/0: creat f0 x:0 0 0 2024-03-18T14:50:47.735 INFO:tasks.workunit.client.1.smithi100.stdout:6/0: mknod c0 0 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/1: symlink l1 0 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/2: chown l1 1331 1 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/3: readlink l1 0 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/4: dread - f0 zero size 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/5: creat f2 x:0 0 0 2024-03-18T14:50:47.737 INFO:tasks.workunit.client.1.smithi100.stdout:6/1: creat f1 x:0 0 0 2024-03-18T14:50:47.738 INFO:tasks.workunit.client.1.smithi100.stdout:6/2: read - f1 zero size 2024-03-18T14:50:47.738 INFO:tasks.workunit.client.1.smithi100.stdout:6/3: truncate f1 807596 0 2024-03-18T14:50:47.739 INFO:tasks.workunit.client.1.smithi100.stdout:7/0: mknod c0 0 2024-03-18T14:50:47.739 INFO:tasks.workunit.client.1.smithi100.stdout:8/0: fsync - no filename 2024-03-18T14:50:47.740 INFO:tasks.workunit.client.1.smithi100.stdout:8/1: dwrite - no filename 2024-03-18T14:50:47.742 INFO:tasks.workunit.client.1.smithi100.stdout:5/6: rename l1 to l3 0 2024-03-18T14:50:47.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/4: mknod c2 0 2024-03-18T14:50:47.751 INFO:tasks.workunit.client.1.smithi100.stdout:7/1: creat f1 x:0 0 0 2024-03-18T14:50:47.751 INFO:tasks.workunit.client.1.smithi100.stdout:8/2: mknod c0 0 2024-03-18T14:50:47.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/12: dwrite f3 [0,4194304] 0 2024-03-18T14:50:47.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/13: creat f6 x:0 0 0 2024-03-18T14:50:47.932 INFO:tasks.workunit.client.1.smithi100.stdout:0/14: link f3 f7 0 2024-03-18T14:50:47.933 INFO:tasks.workunit.client.1.smithi100.stdout:0/15: dread - f5 zero size 2024-03-18T14:50:47.974 INFO:tasks.workunit.client.0.smithi012.stdout:4/239: sync 2024-03-18T14:50:47.974 INFO:tasks.workunit.client.0.smithi012.stdout:4/240: readlink d6/da/l3f 0 2024-03-18T14:50:47.978 INFO:tasks.workunit.client.0.smithi012.stdout:4/241: link d6/d19/f42 d6/d27/f56 0 2024-03-18T14:50:47.978 INFO:tasks.workunit.client.0.smithi012.stdout:4/242: dread - d6/d1a/d30/d46/f48 zero size 2024-03-18T14:50:47.982 INFO:tasks.workunit.client.0.smithi012.stdout:4/243: rename c4 to d6/d19/c57 0 2024-03-18T14:50:47.982 INFO:tasks.workunit.client.0.smithi012.stdout:4/244: dread - d6/da/f16 zero size 2024-03-18T14:50:47.982 INFO:tasks.workunit.client.0.smithi012.stdout:4/245: readlink d6/l3a 0 2024-03-18T14:50:47.987 INFO:tasks.workunit.client.0.smithi012.stdout:4/246: mkdir d6/da/d58 0 2024-03-18T14:50:47.987 INFO:tasks.workunit.client.0.smithi012.stdout:4/247: stat d6/d19/d39 0 2024-03-18T14:50:47.989 INFO:tasks.workunit.client.0.smithi012.stdout:8/300: dwrite d0/d5a/d19/f3a [0,4194304] 0 2024-03-18T14:50:47.990 INFO:tasks.workunit.client.0.smithi012.stdout:4/248: creat d6/da/d58/f59 x:0 0 0 2024-03-18T14:50:47.994 INFO:tasks.workunit.client.0.smithi012.stdout:8/301: mkdir d0/d5a/d19/d1b/d54/d62 0 2024-03-18T14:50:47.996 INFO:tasks.workunit.client.0.smithi012.stdout:3/179: dread dc/f14 [0,4194304] 0 2024-03-18T14:50:47.999 INFO:tasks.workunit.client.0.smithi012.stdout:4/249: mknod d6/d1a/d30/d46/c5a 0 2024-03-18T14:50:47.999 INFO:tasks.workunit.client.0.smithi012.stdout:4/250: fdatasync d6/d1a/f35 0 2024-03-18T14:50:47.999 INFO:tasks.workunit.client.0.smithi012.stdout:8/302: mkdir d0/d5a/d19/d63 0 2024-03-18T14:50:47.999 INFO:tasks.workunit.client.0.smithi012.stdout:8/303: chown d0/d12/d33/f5c 3 1 2024-03-18T14:50:47.999 INFO:tasks.workunit.client.0.smithi012.stdout:8/304: chown d0/f10 289250 1 2024-03-18T14:50:48.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/180: rmdir dc/d30/d35 39 2024-03-18T14:50:48.004 INFO:tasks.workunit.client.0.smithi012.stdout:4/251: mkdir d6/d19/d4f/d5b 0 2024-03-18T14:50:48.004 INFO:tasks.workunit.client.0.smithi012.stdout:4/252: readlink d6/l3a 0 2024-03-18T14:50:48.005 INFO:tasks.workunit.client.0.smithi012.stdout:8/305: mknod d0/d5a/d55/c64 0 2024-03-18T14:50:48.007 INFO:tasks.workunit.client.0.smithi012.stdout:4/253: creat d6/f5c x:0 0 0 2024-03-18T14:50:48.008 INFO:tasks.workunit.client.0.smithi012.stdout:4/254: truncate d6/f2e 195232 0 2024-03-18T14:50:48.032 INFO:tasks.workunit.client.0.smithi012.stdout:4/255: dread d6/d1a/f35 [0,4194304] 0 2024-03-18T14:50:48.035 INFO:tasks.workunit.client.0.smithi012.stdout:4/256: creat d6/d19/d4f/f5d x:0 0 0 2024-03-18T14:50:48.037 INFO:tasks.workunit.client.0.smithi012.stdout:8/306: dread d0/d15/f43 [0,4194304] 0 2024-03-18T14:50:48.078 INFO:tasks.workunit.client.1.smithi100.stdout:0/16: mknod c8 0 2024-03-18T14:50:48.078 INFO:tasks.workunit.client.1.smithi100.stdout:0/17: creat f9 x:0 0 0 2024-03-18T14:50:48.094 INFO:tasks.workunit.client.0.smithi012.stdout:3/181: dwrite dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:48.101 INFO:tasks.workunit.client.0.smithi012.stdout:3/182: truncate f2 91252 0 2024-03-18T14:50:48.101 INFO:tasks.workunit.client.0.smithi012.stdout:3/183: dread - dc/f22 zero size 2024-03-18T14:50:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:47 smithi012 ceph-mon[29182]: pgmap v241: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 17 MiB/s wr, 512 op/s 2024-03-18T14:50:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:47 smithi012 ceph-mon[29182]: mgrmap e20: smithi012.tmscgt(active, since 6m), standbys: smithi100.rwnrcn 2024-03-18T14:50:48.259 INFO:tasks.workunit.client.0.smithi012.stdout:5/252: dwrite d3/d29/d36/f45 [4194304,4194304] 0 2024-03-18T14:50:48.267 INFO:tasks.workunit.client.0.smithi012.stdout:5/253: mkdir d3/d15/d21/d59 0 2024-03-18T14:50:48.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:47 smithi100 ceph-mon[37807]: pgmap v241: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 17 MiB/s wr, 512 op/s 2024-03-18T14:50:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:47 smithi100 ceph-mon[37807]: mgrmap e20: smithi012.tmscgt(active, since 6m), standbys: smithi100.rwnrcn 2024-03-18T14:50:48.369 INFO:tasks.workunit.client.0.smithi012.stdout:4/257: dread d6/d19/f1c [0,4194304] 0 2024-03-18T14:50:48.369 INFO:tasks.workunit.client.0.smithi012.stdout:4/258: readlink d6/l7 0 2024-03-18T14:50:48.375 INFO:tasks.workunit.client.0.smithi012.stdout:4/259: rename d6/d1a/f26 to d6/d19/d4f/f5e 0 2024-03-18T14:50:48.404 INFO:tasks.workunit.client.0.smithi012.stdout:4/260: write d6/d19/f1c [2669796,70598] 0 2024-03-18T14:50:48.453 INFO:tasks.workunit.client.0.smithi012.stdout:4/261: write d6/ff [1225557,7068] 0 2024-03-18T14:50:48.458 INFO:tasks.workunit.client.0.smithi012.stdout:4/262: mknod d6/da/d58/c5f 0 2024-03-18T14:50:48.462 INFO:tasks.workunit.client.0.smithi012.stdout:4/263: mknod d6/d1a/d30/c60 0 2024-03-18T14:50:48.466 INFO:tasks.workunit.client.0.smithi012.stdout:4/264: mknod d6/d1a/d30/c61 0 2024-03-18T14:50:48.466 INFO:tasks.workunit.client.0.smithi012.stdout:4/265: truncate d6/da/f3b 92888 0 2024-03-18T14:50:48.466 INFO:tasks.workunit.client.0.smithi012.stdout:4/266: creat d6/d1a/f62 x:0 0 0 2024-03-18T14:50:48.504 INFO:tasks.workunit.client.1.smithi100.stdout:7/2: stat f1 0 2024-03-18T14:50:48.504 INFO:tasks.workunit.client.1.smithi100.stdout:7/3: truncate f1 501981 0 2024-03-18T14:50:48.504 INFO:tasks.workunit.client.1.smithi100.stdout:7/4: truncate f1 950951 0 2024-03-18T14:50:48.504 INFO:tasks.workunit.client.1.smithi100.stdout:7/5: write f1 [1573580,50820] 0 2024-03-18T14:50:48.691 INFO:tasks.workunit.client.1.smithi100.stdout:6/5: write f1 [495082,98896] 0 2024-03-18T14:50:48.704 INFO:tasks.workunit.client.1.smithi100.stdout:6/6: dread f1 [0,4194304] 0 2024-03-18T14:50:48.704 INFO:tasks.workunit.client.1.smithi100.stdout:6/7: chown c2 25712517 1 2024-03-18T14:50:48.704 INFO:tasks.workunit.client.1.smithi100.stdout:6/8: fsync f1 0 2024-03-18T14:50:48.705 INFO:tasks.workunit.client.1.smithi100.stdout:6/9: dread f1 [0,4194304] 0 2024-03-18T14:50:48.706 INFO:tasks.workunit.client.1.smithi100.stdout:6/10: write f1 [477429,96902] 0 2024-03-18T14:50:48.761 INFO:tasks.workunit.client.1.smithi100.stdout:0/18: rename f1 to fa 0 2024-03-18T14:50:48.815 INFO:tasks.workunit.client.1.smithi100.stdout:7/6: dwrite f1 [0,4194304] 0 2024-03-18T14:50:49.035 INFO:tasks.workunit.client.0.smithi012.stdout:8/307: rmdir d0 39 2024-03-18T14:50:49.035 INFO:tasks.workunit.client.0.smithi012.stdout:8/308: dread - d0/d5a/f60 zero size 2024-03-18T14:50:49.035 INFO:tasks.workunit.client.0.smithi012.stdout:8/309: chown d0/d12 994878858 1 2024-03-18T14:50:49.036 INFO:tasks.workunit.client.0.smithi012.stdout:8/310: rmdir d0/d5a/d19 39 2024-03-18T14:50:49.036 INFO:tasks.workunit.client.0.smithi012.stdout:8/311: write d0/fc [2586304,87803] 0 2024-03-18T14:50:49.036 INFO:tasks.workunit.client.0.smithi012.stdout:8/312: write d0/d15/f1a [5186939,59864] 0 2024-03-18T14:50:49.040 INFO:tasks.workunit.client.0.smithi012.stdout:8/313: mknod d0/d15/d4e/c65 0 2024-03-18T14:50:49.043 INFO:tasks.workunit.client.0.smithi012.stdout:8/314: creat d0/d5a/d19/d57/f66 x:0 0 0 2024-03-18T14:50:49.043 INFO:tasks.workunit.client.0.smithi012.stdout:8/315: write d0/d5a/f60 [717505,26270] 0 2024-03-18T14:50:49.043 INFO:tasks.workunit.client.0.smithi012.stdout:8/316: fdatasync d0/d5a/d19/d1b/d2a/d3e/f4a 0 2024-03-18T14:50:49.058 INFO:tasks.workunit.client.0.smithi012.stdout:8/317: dwrite d0/d12/f20 [0,4194304] 0 2024-03-18T14:50:49.061 INFO:tasks.workunit.client.0.smithi012.stdout:8/318: dread d0/d5a/d19/f29 [0,4194304] 0 2024-03-18T14:50:49.061 INFO:tasks.workunit.client.0.smithi012.stdout:8/319: chown d0/d12/d33/f5c 101678575 1 2024-03-18T14:50:49.062 INFO:tasks.workunit.client.0.smithi012.stdout:8/320: chown d0/d5a/d19/d1b 0 1 2024-03-18T14:50:49.078 INFO:tasks.workunit.client.0.smithi012.stdout:8/321: dwrite d0/d5a/d19/d1b/f30 [0,4194304] 0 2024-03-18T14:50:49.086 INFO:tasks.workunit.client.0.smithi012.stdout:8/322: write d0/d12/f20 [1116554,104090] 0 2024-03-18T14:50:49.089 INFO:tasks.workunit.client.0.smithi012.stdout:8/323: mkdir d0/d47/d59/d67 0 2024-03-18T14:50:49.095 INFO:tasks.workunit.client.0.smithi012.stdout:8/324: dread d0/d12/f20 [0,4194304] 0 2024-03-18T14:50:49.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/325: dread d0/d5a/d19/d1b/f30 [0,4194304] 0 2024-03-18T14:50:49.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/326: creat d0/d5a/d19/d57/f68 x:0 0 0 2024-03-18T14:50:49.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/327: readlink d0/l2 0 2024-03-18T14:50:49.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/328: creat d0/d5a/d19/d57/f69 x:0 0 0 2024-03-18T14:50:49.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/329: fsync d0/d12/d33/f5c 0 2024-03-18T14:50:49.109 INFO:tasks.workunit.client.0.smithi012.stdout:8/330: truncate d0/d12/f4b 4066075 0 2024-03-18T14:50:49.112 INFO:tasks.workunit.client.0.smithi012.stdout:8/331: dread d0/d12/f2d [0,4194304] 0 2024-03-18T14:50:49.115 INFO:tasks.workunit.client.0.smithi012.stdout:8/332: mkdir d0/d12/d33/d50/d6a 0 2024-03-18T14:50:49.117 INFO:tasks.workunit.client.0.smithi012.stdout:9/253: sync 2024-03-18T14:50:49.117 INFO:tasks.workunit.client.0.smithi012.stdout:9/254: chown d3/f2e 431860 1 2024-03-18T14:50:49.118 INFO:tasks.workunit.client.0.smithi012.stdout:8/333: creat d0/d5a/d19/d1b/f6b x:0 0 0 2024-03-18T14:50:49.121 INFO:tasks.workunit.client.0.smithi012.stdout:9/255: mkdir d3/d49 0 2024-03-18T14:50:49.126 INFO:tasks.workunit.client.0.smithi012.stdout:9/256: mkdir d3/d1e/d21/d4a 0 2024-03-18T14:50:49.132 INFO:tasks.workunit.client.0.smithi012.stdout:9/257: rmdir d3/d9/d19 39 2024-03-18T14:50:49.133 INFO:tasks.workunit.client.0.smithi012.stdout:9/258: write d3/d1e/d21/f26 [608136,59613] 0 2024-03-18T14:50:49.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/334: dwrite d0/f6 [0,4194304] 0 2024-03-18T14:50:49.135 INFO:tasks.workunit.client.0.smithi012.stdout:9/259: truncate d3/f25 1327996 0 2024-03-18T14:50:49.141 INFO:tasks.workunit.client.0.smithi012.stdout:8/335: rmdir d0/d15 39 2024-03-18T14:50:49.147 INFO:tasks.workunit.client.0.smithi012.stdout:8/336: mknod d0/d5a/d55/c6c 0 2024-03-18T14:50:49.258 INFO:tasks.workunit.client.0.smithi012.stdout:6/207: sync 2024-03-18T14:50:49.258 INFO:tasks.workunit.client.0.smithi012.stdout:1/225: sync 2024-03-18T14:50:49.258 INFO:tasks.workunit.client.0.smithi012.stdout:7/217: sync 2024-03-18T14:50:49.258 INFO:tasks.workunit.client.0.smithi012.stdout:2/242: sync 2024-03-18T14:50:49.258 INFO:tasks.workunit.client.0.smithi012.stdout:0/231: sync 2024-03-18T14:50:49.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/208: chown d17/d19/d27/d2f/l32 2 1 2024-03-18T14:50:49.259 INFO:tasks.workunit.client.0.smithi012.stdout:7/218: fsync dd/d13/f15 0 2024-03-18T14:50:49.259 INFO:tasks.workunit.client.0.smithi012.stdout:6/209: write f4 [5008278,50697] 0 2024-03-18T14:50:49.259 INFO:tasks.workunit.client.0.smithi012.stdout:0/232: creat d1/d16/d19/f56 x:0 0 0 2024-03-18T14:50:49.261 INFO:tasks.workunit.client.0.smithi012.stdout:2/243: rename d0/cd to d0/c45 0 2024-03-18T14:50:49.261 INFO:tasks.workunit.client.0.smithi012.stdout:2/244: creat d0/d3c/f46 x:0 0 0 2024-03-18T14:50:49.261 INFO:tasks.workunit.client.0.smithi012.stdout:2/245: fsync d0/f6 0 2024-03-18T14:50:49.263 INFO:tasks.workunit.client.0.smithi012.stdout:1/226: mkdir d8/d1e/d27/d44 0 2024-03-18T14:50:49.263 INFO:tasks.workunit.client.0.smithi012.stdout:1/227: chown d8/db/d10/f33 12028 1 2024-03-18T14:50:49.263 INFO:tasks.workunit.client.0.smithi012.stdout:1/228: stat d8/db 0 2024-03-18T14:50:49.263 INFO:tasks.workunit.client.0.smithi012.stdout:6/210: dread f5 [0,4194304] 0 2024-03-18T14:50:49.271 INFO:tasks.workunit.client.0.smithi012.stdout:2/246: dread d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:49.274 INFO:tasks.workunit.client.0.smithi012.stdout:6/211: rename d17/d19/f24 to d17/f35 0 2024-03-18T14:50:49.274 INFO:tasks.workunit.client.0.smithi012.stdout:1/229: dread d8/fc [0,4194304] 0 2024-03-18T14:50:49.275 INFO:tasks.workunit.client.0.smithi012.stdout:7/219: getdents dd/d13/d23/d2f 0 2024-03-18T14:50:49.277 INFO:tasks.workunit.client.0.smithi012.stdout:2/247: mkdir d0/d3c/d40/d47 0 2024-03-18T14:50:49.278 INFO:tasks.workunit.client.0.smithi012.stdout:0/233: dwrite d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:49.278 INFO:tasks.workunit.client.0.smithi012.stdout:0/234: readlink d1/l8 0 2024-03-18T14:50:49.279 INFO:tasks.workunit.client.0.smithi012.stdout:6/212: mkdir d17/d26/d36 0 2024-03-18T14:50:49.281 INFO:tasks.workunit.client.0.smithi012.stdout:1/230: mkdir d8/db/d10/d31/d45 0 2024-03-18T14:50:49.282 INFO:tasks.workunit.client.0.smithi012.stdout:7/220: rmdir dd/d13/d23/d2f 39 2024-03-18T14:50:49.282 INFO:tasks.workunit.client.0.smithi012.stdout:7/221: creat dd/d13/d23/d33/f4d x:0 0 0 2024-03-18T14:50:49.286 INFO:tasks.workunit.client.0.smithi012.stdout:0/235: creat d1/d20/d4e/d54/f57 x:0 0 0 2024-03-18T14:50:49.286 INFO:tasks.workunit.client.0.smithi012.stdout:0/236: stat d1/fa 0 2024-03-18T14:50:49.286 INFO:tasks.workunit.client.0.smithi012.stdout:0/237: creat d1/d16/d31/f58 x:0 0 0 2024-03-18T14:50:49.287 INFO:tasks.workunit.client.0.smithi012.stdout:0/238: chown d1/c48 0 1 2024-03-18T14:50:49.287 INFO:tasks.workunit.client.0.smithi012.stdout:2/248: dread d0/f21 [0,4194304] 0 2024-03-18T14:50:49.287 INFO:tasks.workunit.client.0.smithi012.stdout:2/249: chown d0/d3c/d40 176766 1 2024-03-18T14:50:49.289 INFO:tasks.workunit.client.0.smithi012.stdout:6/213: symlink d17/d19/d27/l37 0 2024-03-18T14:50:49.289 INFO:tasks.workunit.client.0.smithi012.stdout:6/214: chown f4 941601741 1 2024-03-18T14:50:49.292 INFO:tasks.workunit.client.0.smithi012.stdout:6/215: dread f12 [0,4194304] 0 2024-03-18T14:50:49.292 INFO:tasks.workunit.client.0.smithi012.stdout:6/216: write d17/f35 [635988,56272] 0 2024-03-18T14:50:49.292 INFO:tasks.workunit.client.0.smithi012.stdout:0/239: dread d1/d16/d19/f37 [0,4194304] 0 2024-03-18T14:50:49.293 INFO:tasks.workunit.client.0.smithi012.stdout:1/231: unlink d8/db/d10/f13 0 2024-03-18T14:50:49.298 INFO:tasks.workunit.client.0.smithi012.stdout:7/222: dwrite dd/f1e [4194304,4194304] 0 2024-03-18T14:50:49.300 INFO:tasks.workunit.client.0.smithi012.stdout:2/250: creat d0/f48 x:0 0 0 2024-03-18T14:50:49.300 INFO:tasks.workunit.client.0.smithi012.stdout:2/251: dread - d0/d10/f32 zero size 2024-03-18T14:50:49.301 INFO:tasks.workunit.client.0.smithi012.stdout:6/217: mkdir d17/d38 0 2024-03-18T14:50:49.303 INFO:tasks.workunit.client.0.smithi012.stdout:0/240: rmdir d1/d20/d23/d3b/d33 39 2024-03-18T14:50:49.303 INFO:tasks.workunit.client.0.smithi012.stdout:0/241: dread - d1/d16/d19/f56 zero size 2024-03-18T14:50:49.308 INFO:tasks.workunit.client.0.smithi012.stdout:7/223: mknod dd/d13/c4e 0 2024-03-18T14:50:49.311 INFO:tasks.workunit.client.0.smithi012.stdout:2/252: rename d0/f48 to d0/d3c/d40/d47/f49 0 2024-03-18T14:50:49.313 INFO:tasks.workunit.client.0.smithi012.stdout:0/242: creat d1/f59 x:0 0 0 2024-03-18T14:50:49.313 INFO:tasks.workunit.client.0.smithi012.stdout:0/243: truncate d1/d20/d23/f46 4864286 0 2024-03-18T14:50:49.313 INFO:tasks.workunit.client.0.smithi012.stdout:0/244: read - d1/d16/d31/f50 zero size 2024-03-18T14:50:49.315 INFO:tasks.workunit.client.0.smithi012.stdout:7/224: mkdir dd/d4f 0 2024-03-18T14:50:49.317 INFO:tasks.workunit.client.0.smithi012.stdout:1/232: dwrite d8/db/f2d [0,4194304] 0 2024-03-18T14:50:49.317 INFO:tasks.workunit.client.0.smithi012.stdout:6/218: dwrite d17/d19/f2b [0,4194304] 0 2024-03-18T14:50:49.318 INFO:tasks.workunit.client.0.smithi012.stdout:2/253: symlink d0/l4a 0 2024-03-18T14:50:49.321 INFO:tasks.workunit.client.0.smithi012.stdout:7/225: symlink dd/d13/d23/d33/d48/l50 0 2024-03-18T14:50:49.321 INFO:tasks.workunit.client.0.smithi012.stdout:7/226: creat dd/d13/d23/d33/f51 x:0 0 0 2024-03-18T14:50:49.325 INFO:tasks.workunit.client.0.smithi012.stdout:1/233: mknod d8/db/d10/d31/c46 0 2024-03-18T14:50:49.325 INFO:tasks.workunit.client.0.smithi012.stdout:1/234: dread - d8/db/d10/d16/d1c/f23 zero size 2024-03-18T14:50:49.326 INFO:tasks.workunit.client.0.smithi012.stdout:6/219: mknod d17/d19/c39 0 2024-03-18T14:50:49.327 INFO:tasks.workunit.client.0.smithi012.stdout:6/220: write d17/f1b [455643,30119] 0 2024-03-18T14:50:49.328 INFO:tasks.workunit.client.0.smithi012.stdout:2/254: creat d0/d3c/d40/f4b x:0 0 0 2024-03-18T14:50:49.333 INFO:tasks.workunit.client.0.smithi012.stdout:7/227: link dd/d13/d23/d2f/f3d dd/d13/d23/d33/d37/f52 0 2024-03-18T14:50:49.334 INFO:tasks.workunit.client.0.smithi012.stdout:6/221: write d17/d19/f2b [2183159,37360] 0 2024-03-18T14:50:49.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/235: mknod d8/d1e/d27/d44/c47 0 2024-03-18T14:50:49.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/236: creat d8/d1e/d27/f48 x:0 0 0 2024-03-18T14:50:49.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/237: write d8/d1e/f2f [387257,49534] 0 2024-03-18T14:50:49.338 INFO:tasks.workunit.client.0.smithi012.stdout:2/255: mkdir d0/d10/d18/d31/d4c 0 2024-03-18T14:50:49.342 INFO:tasks.workunit.client.0.smithi012.stdout:6/222: rmdir d17/d34 0 2024-03-18T14:50:49.347 INFO:tasks.workunit.client.0.smithi012.stdout:1/238: symlink d8/db/d10/d16/l49 0 2024-03-18T14:50:49.350 INFO:tasks.workunit.client.0.smithi012.stdout:7/228: dwrite dd/d13/d23/d2a/f3e [0,4194304] 0 2024-03-18T14:50:49.358 INFO:tasks.workunit.client.0.smithi012.stdout:6/223: dwrite f8 [0,4194304] 0 2024-03-18T14:50:49.361 INFO:tasks.workunit.client.0.smithi012.stdout:7/229: mkdir dd/d13/d23/d2f/d53 0 2024-03-18T14:50:49.373 INFO:tasks.workunit.client.0.smithi012.stdout:6/224: dwrite d17/d18/f1f [0,4194304] 0 2024-03-18T14:50:49.379 INFO:tasks.workunit.client.0.smithi012.stdout:7/230: unlink dd/d13/d23/c30 0 2024-03-18T14:50:49.380 INFO:tasks.workunit.client.0.smithi012.stdout:7/231: write dd/f10 [947282,54288] 0 2024-03-18T14:50:49.383 INFO:tasks.workunit.client.0.smithi012.stdout:6/225: symlink d17/d33/l3a 0 2024-03-18T14:50:49.386 INFO:tasks.workunit.client.0.smithi012.stdout:7/232: link dd/c20 dd/d13/d23/d33/c54 0 2024-03-18T14:50:49.386 INFO:tasks.workunit.client.0.smithi012.stdout:7/233: fsync dd/d13/d23/d2f/f32 0 2024-03-18T14:50:49.386 INFO:tasks.workunit.client.0.smithi012.stdout:7/234: dread - dd/f45 zero size 2024-03-18T14:50:49.386 INFO:tasks.workunit.client.0.smithi012.stdout:7/235: chown dd/c2d 29 1 2024-03-18T14:50:49.389 INFO:tasks.workunit.client.0.smithi012.stdout:6/226: truncate d17/f1a 2844172 0 2024-03-18T14:50:49.389 INFO:tasks.workunit.client.0.smithi012.stdout:6/227: creat d17/f3b x:0 0 0 2024-03-18T14:50:49.389 INFO:tasks.workunit.client.0.smithi012.stdout:6/228: chown d17 14587 1 2024-03-18T14:50:49.389 INFO:tasks.workunit.client.0.smithi012.stdout:6/229: write ff [2969698,71685] 0 2024-03-18T14:50:49.389 INFO:tasks.workunit.client.0.smithi012.stdout:6/230: fsync d17/d18/f1f 0 2024-03-18T14:50:49.391 INFO:tasks.workunit.client.0.smithi012.stdout:7/236: creat dd/d13/d23/f55 x:0 0 0 2024-03-18T14:50:49.391 INFO:tasks.workunit.client.0.smithi012.stdout:7/237: truncate dd/f31 383932 0 2024-03-18T14:50:49.398 INFO:tasks.workunit.client.0.smithi012.stdout:6/231: unlink d17/l2a 0 2024-03-18T14:50:49.401 INFO:tasks.workunit.client.0.smithi012.stdout:7/238: mknod dd/d13/d23/c56 0 2024-03-18T14:50:49.404 INFO:tasks.workunit.client.0.smithi012.stdout:6/232: dread f14 [0,4194304] 0 2024-03-18T14:50:49.406 INFO:tasks.workunit.client.0.smithi012.stdout:7/239: creat dd/d13/f57 x:0 0 0 2024-03-18T14:50:49.408 INFO:tasks.workunit.client.0.smithi012.stdout:6/233: creat d17/d19/f3c x:0 0 0 2024-03-18T14:50:49.410 INFO:tasks.workunit.client.0.smithi012.stdout:6/234: mknod d17/d19/c3d 0 2024-03-18T14:50:49.410 INFO:tasks.workunit.client.0.smithi012.stdout:6/235: creat d17/d19/f3e x:0 0 0 2024-03-18T14:50:49.411 INFO:tasks.workunit.client.0.smithi012.stdout:7/240: dread dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:49.411 INFO:tasks.workunit.client.0.smithi012.stdout:7/241: fdatasync dd/d13/d23/d33/f4d 0 2024-03-18T14:50:49.411 INFO:tasks.workunit.client.0.smithi012.stdout:7/242: fsync dd/f42 0 2024-03-18T14:50:49.426 INFO:tasks.workunit.client.0.smithi012.stdout:6/236: dwrite d17/d19/f20 [0,4194304] 0 2024-03-18T14:50:49.436 INFO:tasks.workunit.client.0.smithi012.stdout:7/243: mknod dd/d13/c58 0 2024-03-18T14:50:49.459 INFO:tasks.workunit.client.0.smithi012.stdout:7/244: dwrite f1 [0,4194304] 0 2024-03-18T14:50:49.479 INFO:tasks.workunit.client.0.smithi012.stdout:6/237: mkdir d17/d38/d3f 0 2024-03-18T14:50:49.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/245: rmdir dd/d13/d23/d33/d48 39 2024-03-18T14:50:49.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/246: creat dd/f59 x:0 0 0 2024-03-18T14:50:49.493 INFO:tasks.workunit.client.0.smithi012.stdout:6/238: symlink d17/d26/d36/l40 0 2024-03-18T14:50:49.496 INFO:tasks.workunit.client.0.smithi012.stdout:7/247: rename dd/f1e to dd/f5a 0 2024-03-18T14:50:49.497 INFO:tasks.workunit.client.0.smithi012.stdout:7/248: dread - dd/d13/d23/d33/f51 zero size 2024-03-18T14:50:49.497 INFO:tasks.workunit.client.0.smithi012.stdout:7/249: creat dd/f5b x:0 0 0 2024-03-18T14:50:49.499 INFO:tasks.workunit.client.0.smithi012.stdout:6/239: write ff [1382155,128168] 0 2024-03-18T14:50:49.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/250: link dd/d13/d23/d33/d37/f52 dd/d13/d23/d33/d48/f5c 0 2024-03-18T14:50:49.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/251: write dd/d13/d23/d33/f49 [3882420,79990] 0 2024-03-18T14:50:49.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/252: dread - dd/f5b zero size 2024-03-18T14:50:49.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/253: chown dd/l1b 5513 1 2024-03-18T14:50:49.504 INFO:tasks.workunit.client.0.smithi012.stdout:6/240: link d17/d19/c39 d17/d38/d3f/c41 0 2024-03-18T14:50:49.512 INFO:tasks.workunit.client.0.smithi012.stdout:7/254: write dd/d13/f15 [2600444,46073] 0 2024-03-18T14:50:49.518 INFO:tasks.workunit.client.0.smithi012.stdout:6/241: dwrite f9 [4194304,4194304] 0 2024-03-18T14:50:49.529 INFO:tasks.workunit.client.0.smithi012.stdout:4/267: rmdir d6/d1a/d30 39 2024-03-18T14:50:49.532 INFO:tasks.workunit.client.0.smithi012.stdout:4/268: mknod d6/d1a/d30/d46/c63 0 2024-03-18T14:50:49.535 INFO:tasks.workunit.client.0.smithi012.stdout:6/242: dwrite f16 [0,4194304] 0 2024-03-18T14:50:49.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/254: sync 2024-03-18T14:50:49.742 INFO:tasks.workunit.client.0.smithi012.stdout:3/184: sync 2024-03-18T14:50:49.742 INFO:tasks.workunit.client.0.smithi012.stdout:3/185: chown dc/l20 9 1 2024-03-18T14:50:49.743 INFO:tasks.workunit.client.0.smithi012.stdout:3/186: read - dc/f2b zero size 2024-03-18T14:50:49.745 INFO:tasks.workunit.client.0.smithi012.stdout:5/255: creat d3/d29/d35/f5a x:0 0 0 2024-03-18T14:50:49.748 INFO:tasks.workunit.client.0.smithi012.stdout:3/187: mkdir dc/d30/d3a 0 2024-03-18T14:50:49.754 INFO:tasks.workunit.client.0.smithi012.stdout:5/256: write d3/f1f [7678410,117975] 0 2024-03-18T14:50:49.847 INFO:tasks.workunit.client.0.smithi012.stdout:5/257: mkdir d3/d29/d35/d5b 0 2024-03-18T14:50:49.847 INFO:tasks.workunit.client.0.smithi012.stdout:5/258: creat d3/d29/d36/d3d/f5c x:0 0 0 2024-03-18T14:50:49.852 INFO:tasks.workunit.client.0.smithi012.stdout:5/259: mknod d3/d29/d35/d5b/c5d 0 2024-03-18T14:50:49.868 INFO:tasks.workunit.client.0.smithi012.stdout:5/260: dwrite d3/d15/f1c [4194304,4194304] 0 2024-03-18T14:50:49.868 INFO:tasks.workunit.client.0.smithi012.stdout:5/261: chown d3/d1e/c9 377 1 2024-03-18T14:50:49.889 INFO:tasks.workunit.client.0.smithi012.stdout:5/262: dwrite d3/d1e/d33/d55/f58 [0,4194304] 0 2024-03-18T14:50:49.890 INFO:tasks.workunit.client.0.smithi012.stdout:5/263: chown d3/f22 1655724 1 2024-03-18T14:50:49.901 INFO:tasks.workunit.client.0.smithi012.stdout:5/264: rename d3/d29/d36/d4e to d3/d15/d21/d5e 0 2024-03-18T14:50:49.905 INFO:tasks.workunit.client.0.smithi012.stdout:5/265: unlink d3/d15/c2f 0 2024-03-18T14:50:49.906 INFO:tasks.workunit.client.0.smithi012.stdout:5/266: stat d3/d29/d36/d3d/f3e 0 2024-03-18T14:50:50.080 INFO:tasks.workunit.client.0.smithi012.stdout:8/337: dread d0/d12/f4b [0,4194304] 0 2024-03-18T14:50:50.082 INFO:tasks.workunit.client.0.smithi012.stdout:8/338: creat d0/d5a/f6d x:0 0 0 2024-03-18T14:50:50.085 INFO:tasks.workunit.client.0.smithi012.stdout:8/339: mkdir d0/d15/d6e 0 2024-03-18T14:50:50.088 INFO:tasks.workunit.client.0.smithi012.stdout:8/340: getdents d0/d5a/d19/d1b/d2a 0 2024-03-18T14:50:50.088 INFO:tasks.workunit.client.0.smithi012.stdout:8/341: readlink d0/d5a/l2e 0 2024-03-18T14:50:50.096 INFO:tasks.workunit.client.0.smithi012.stdout:8/342: write d0/f6 [1524900,59254] 0 2024-03-18T14:50:50.105 INFO:tasks.workunit.client.0.smithi012.stdout:8/343: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:50.126 INFO:tasks.workunit.client.0.smithi012.stdout:8/344: dwrite d0/d12/f46 [0,4194304] 0 2024-03-18T14:50:50.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/345: write d0/d5a/d19/f3a [3898260,120030] 0 2024-03-18T14:50:50.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/346: stat d0/d5a/d55/c64 0 2024-03-18T14:50:50.133 INFO:tasks.workunit.client.0.smithi012.stdout:8/347: write d0/d12/f20 [1452416,116903] 0 2024-03-18T14:50:50.151 INFO:tasks.workunit.client.0.smithi012.stdout:8/348: dwrite d0/d5a/d19/d1b/d2a/d3e/f4a [0,4194304] 0 2024-03-18T14:50:50.168 INFO:tasks.workunit.client.0.smithi012.stdout:8/349: dwrite d0/d5a/f60 [0,4194304] 0 2024-03-18T14:50:50.174 INFO:tasks.workunit.client.0.smithi012.stdout:8/350: symlink d0/d47/l6f 0 2024-03-18T14:50:50.178 INFO:tasks.workunit.client.0.smithi012.stdout:8/351: symlink d0/d5a/d19/d1b/d54/d62/l70 0 2024-03-18T14:50:50.178 INFO:tasks.workunit.client.0.smithi012.stdout:8/352: stat d0/d5a/d19/f3a 0 2024-03-18T14:50:50.185 INFO:tasks.workunit.client.0.smithi012.stdout:8/353: dread d0/d5a/f1d [0,4194304] 0 2024-03-18T14:50:50.191 INFO:tasks.workunit.client.0.smithi012.stdout:8/354: write d0/d5a/d19/d1b/f30 [1664666,111839] 0 2024-03-18T14:50:50.191 INFO:tasks.workunit.client.0.smithi012.stdout:8/355: chown d0/d12/d33/f44 1015173 1 2024-03-18T14:50:50.192 INFO:tasks.workunit.client.0.smithi012.stdout:8/356: fsync d0/d15/f28 0 2024-03-18T14:50:50.192 INFO:tasks.workunit.client.0.smithi012.stdout:8/357: write d0/d5a/d19/d57/f68 [690620,107171] 0 2024-03-18T14:50:50.196 INFO:tasks.workunit.client.0.smithi012.stdout:8/358: creat d0/d12/f71 x:0 0 0 2024-03-18T14:50:50.199 INFO:tasks.workunit.client.0.smithi012.stdout:8/359: dread d0/d12/f2d [0,4194304] 0 2024-03-18T14:50:50.206 INFO:tasks.workunit.client.0.smithi012.stdout:8/360: dread d0/d12/f46 [0,4194304] 0 2024-03-18T14:50:50.208 INFO:tasks.workunit.client.0.smithi012.stdout:8/361: creat d0/d5a/d19/d1b/d54/f72 x:0 0 0 2024-03-18T14:50:50.213 INFO:tasks.workunit.client.0.smithi012.stdout:8/362: dread d0/d5a/d19/d1b/d2a/f5d [0,4194304] 0 2024-03-18T14:50:50.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:49 smithi012 ceph-mon[29182]: pgmap v242: 65 pgs: 65 active+clean; 591 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 63 MiB/s wr, 538 op/s 2024-03-18T14:50:50.282 INFO:tasks.workunit.client.0.smithi012.stdout:0/245: getdents d1 0 2024-03-18T14:50:50.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:49 smithi100 ceph-mon[37807]: pgmap v242: 65 pgs: 65 active+clean; 591 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 63 MiB/s wr, 538 op/s 2024-03-18T14:50:50.300 INFO:tasks.workunit.client.0.smithi012.stdout:0/246: dwrite d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:50.304 INFO:tasks.workunit.client.0.smithi012.stdout:0/247: unlink d1/d16/d18/l1f 0 2024-03-18T14:50:50.306 INFO:tasks.workunit.client.0.smithi012.stdout:0/248: link d1/d16/d31/f50 d1/d20/f5a 0 2024-03-18T14:50:50.309 INFO:tasks.workunit.client.0.smithi012.stdout:0/249: mknod d1/d16/d31/c5b 0 2024-03-18T14:50:50.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/239: rmdir d8 39 2024-03-18T14:50:50.312 INFO:tasks.workunit.client.0.smithi012.stdout:1/240: fsync d8/d1e/d27/f3c 0 2024-03-18T14:50:50.316 INFO:tasks.workunit.client.0.smithi012.stdout:0/250: write d1/d20/d23/d26/f36 [2522865,126631] 0 2024-03-18T14:50:50.321 INFO:tasks.workunit.client.0.smithi012.stdout:1/241: dread f5 [4194304,4194304] 0 2024-03-18T14:50:50.321 INFO:tasks.workunit.client.0.smithi012.stdout:1/242: chown d8 6764710 1 2024-03-18T14:50:50.341 INFO:tasks.workunit.client.0.smithi012.stdout:1/243: dwrite d8/f40 [0,4194304] 0 2024-03-18T14:50:50.347 INFO:tasks.workunit.client.0.smithi012.stdout:1/244: creat d8/d1e/f4a x:0 0 0 2024-03-18T14:50:50.350 INFO:tasks.workunit.client.0.smithi012.stdout:1/245: creat d8/db/d10/d16/d1c/d2e/f4b x:0 0 0 2024-03-18T14:50:50.366 INFO:tasks.workunit.client.0.smithi012.stdout:1/246: dwrite d8/db/f41 [0,4194304] 0 2024-03-18T14:50:50.371 INFO:tasks.workunit.client.0.smithi012.stdout:1/247: creat d8/db/d10/d16/d1c/f4c x:0 0 0 2024-03-18T14:50:50.374 INFO:tasks.workunit.client.0.smithi012.stdout:1/248: unlink d8/db/d10/d16/d1c/d2e/f3f 0 2024-03-18T14:50:50.374 INFO:tasks.workunit.client.0.smithi012.stdout:1/249: chown d8/d1e/f4a 69282 1 2024-03-18T14:50:50.388 INFO:tasks.workunit.client.0.smithi012.stdout:7/255: fsync f5 0 2024-03-18T14:50:50.397 INFO:tasks.workunit.client.0.smithi012.stdout:1/250: dwrite d8/db/ff [0,4194304] 0 2024-03-18T14:50:50.413 INFO:tasks.workunit.client.0.smithi012.stdout:1/251: dwrite d8/db/d10/d16/d1c/f2c [0,4194304] 0 2024-03-18T14:50:50.413 INFO:tasks.workunit.client.0.smithi012.stdout:1/252: write d8/db/f2d [4324792,46268] 0 2024-03-18T14:50:50.417 INFO:tasks.workunit.client.0.smithi012.stdout:1/253: creat d8/db/d10/d31/f4d x:0 0 0 2024-03-18T14:50:50.417 INFO:tasks.workunit.client.0.smithi012.stdout:1/254: chown d8/d1e/d27 235814209 1 2024-03-18T14:50:50.417 INFO:tasks.workunit.client.0.smithi012.stdout:1/255: chown d8/db/d10/d31/d43 853439 1 2024-03-18T14:50:50.419 INFO:tasks.workunit.client.0.smithi012.stdout:1/256: mkdir d8/db/d10/d4e 0 2024-03-18T14:50:50.422 INFO:tasks.workunit.client.0.smithi012.stdout:1/257: creat d8/d1e/d27/d44/f4f x:0 0 0 2024-03-18T14:50:50.424 INFO:tasks.workunit.client.0.smithi012.stdout:1/258: mknod d8/db/d10/c50 0 2024-03-18T14:50:50.507 INFO:tasks.workunit.client.0.smithi012.stdout:9/260: sync 2024-03-18T14:50:50.507 INFO:tasks.workunit.client.0.smithi012.stdout:2/256: sync 2024-03-18T14:50:50.507 INFO:tasks.workunit.client.0.smithi012.stdout:4/269: sync 2024-03-18T14:50:50.507 INFO:tasks.workunit.client.0.smithi012.stdout:4/270: dread - d6/f5c zero size 2024-03-18T14:50:50.509 INFO:tasks.workunit.client.0.smithi012.stdout:4/271: mkdir d6/d1a/d64 0 2024-03-18T14:50:50.512 INFO:tasks.workunit.client.0.smithi012.stdout:4/272: mknod d6/d1a/d64/c65 0 2024-03-18T14:50:50.528 INFO:tasks.workunit.client.0.smithi012.stdout:9/261: dwrite d3/d9/d19/f3a [0,4194304] 0 2024-03-18T14:50:50.529 INFO:tasks.workunit.client.0.smithi012.stdout:4/273: dwrite d6/d19/d39/f50 [0,4194304] 0 2024-03-18T14:50:50.535 INFO:tasks.workunit.client.0.smithi012.stdout:9/262: symlink d3/d1e/l4b 0 2024-03-18T14:50:50.542 INFO:tasks.workunit.client.0.smithi012.stdout:4/274: dread d6/da/fd [0,4194304] 0 2024-03-18T14:50:50.545 INFO:tasks.workunit.client.0.smithi012.stdout:9/263: mknod d3/d9/c4c 0 2024-03-18T14:50:50.571 INFO:tasks.workunit.client.0.smithi012.stdout:9/264: dwrite d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:50.576 INFO:tasks.workunit.client.0.smithi012.stdout:4/275: dwrite d6/f5c [0,4194304] 0 2024-03-18T14:50:50.582 INFO:tasks.workunit.client.0.smithi012.stdout:4/276: mkdir d6/d1a/d30/d66 0 2024-03-18T14:50:50.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/265: dwrite d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:50.593 INFO:tasks.workunit.client.0.smithi012.stdout:9/266: creat d3/d9/d19/f4d x:0 0 0 2024-03-18T14:50:50.593 INFO:tasks.workunit.client.0.smithi012.stdout:9/267: fdatasync d3/d9/f17 0 2024-03-18T14:50:50.595 INFO:tasks.workunit.client.0.smithi012.stdout:9/268: stat d3/d1e/d21/d35/l3d 0 2024-03-18T14:50:50.596 INFO:tasks.workunit.client.0.smithi012.stdout:9/269: write d3/d9/f1c [72223,95964] 0 2024-03-18T14:50:50.604 INFO:tasks.workunit.client.0.smithi012.stdout:9/270: write d3/d20/f43 [5349118,82009] 0 2024-03-18T14:50:50.610 INFO:tasks.workunit.client.0.smithi012.stdout:4/277: dwrite d6/d19/f2f [0,4194304] 0 2024-03-18T14:50:50.610 INFO:tasks.workunit.client.0.smithi012.stdout:4/278: fsync d6/d1a/f2d 0 2024-03-18T14:50:50.610 INFO:tasks.workunit.client.0.smithi012.stdout:4/279: readlink d6/d1a/l53 0 2024-03-18T14:50:50.610 INFO:tasks.workunit.client.0.smithi012.stdout:4/280: chown d6/d19/f1d 471 1 2024-03-18T14:50:50.610 INFO:tasks.workunit.client.0.smithi012.stdout:4/281: dread - d6/d1a/f62 zero size 2024-03-18T14:50:50.612 INFO:tasks.workunit.client.0.smithi012.stdout:9/271: dread d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:50.621 INFO:tasks.workunit.client.0.smithi012.stdout:4/282: write d6/f28 [3052685,55539] 0 2024-03-18T14:50:50.625 INFO:tasks.workunit.client.0.smithi012.stdout:4/283: rmdir d6/d27 39 2024-03-18T14:50:50.625 INFO:tasks.workunit.client.0.smithi012.stdout:4/284: write d6/d1a/f29 [4462664,73126] 0 2024-03-18T14:50:50.627 INFO:tasks.workunit.client.0.smithi012.stdout:4/285: dread d6/d19/d39/f40 [0,4194304] 0 2024-03-18T14:50:50.628 INFO:tasks.workunit.client.0.smithi012.stdout:9/272: dwrite d3/f3e [0,4194304] 0 2024-03-18T14:50:50.630 INFO:tasks.workunit.client.0.smithi012.stdout:9/273: symlink d3/d20/l4e 0 2024-03-18T14:50:50.644 INFO:tasks.workunit.client.0.smithi012.stdout:4/286: dwrite d6/d19/d39/f50 [0,4194304] 0 2024-03-18T14:50:50.644 INFO:tasks.workunit.client.0.smithi012.stdout:4/287: chown d6/l7 12 1 2024-03-18T14:50:50.648 INFO:tasks.workunit.client.0.smithi012.stdout:4/288: rmdir d6/d19/d4f/d5b 0 2024-03-18T14:50:50.651 INFO:tasks.workunit.client.0.smithi012.stdout:4/289: creat d6/d27/f67 x:0 0 0 2024-03-18T14:50:50.654 INFO:tasks.workunit.client.0.smithi012.stdout:4/290: mknod d6/d1a/d30/d66/c68 0 2024-03-18T14:50:50.655 INFO:tasks.workunit.client.0.smithi012.stdout:4/291: fsync d6/f44 0 2024-03-18T14:50:50.659 INFO:tasks.workunit.client.0.smithi012.stdout:4/292: unlink f1 0 2024-03-18T14:50:50.659 INFO:tasks.workunit.client.0.smithi012.stdout:4/293: chown d6/d1a/d30/d46 1951464394 1 2024-03-18T14:50:50.662 INFO:tasks.workunit.client.0.smithi012.stdout:4/294: rmdir d6/d1a/d30/d66 39 2024-03-18T14:50:50.664 INFO:tasks.workunit.client.0.smithi012.stdout:4/295: symlink d6/da/l69 0 2024-03-18T14:50:50.682 INFO:tasks.workunit.client.0.smithi012.stdout:6/243: sync 2024-03-18T14:50:50.697 INFO:tasks.workunit.client.0.smithi012.stdout:6/244: dwrite f8 [0,4194304] 0 2024-03-18T14:50:50.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/188: sync 2024-03-18T14:50:50.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/189: chown dc/f27 248 1 2024-03-18T14:50:50.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/190: dread - dc/f32 zero size 2024-03-18T14:50:50.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/191: fsync f3 0 2024-03-18T14:50:50.723 INFO:tasks.workunit.client.0.smithi012.stdout:3/192: readlink dc/l1d 0 2024-03-18T14:50:50.728 INFO:tasks.workunit.client.0.smithi012.stdout:5/267: rename d3/d15/d21/d5e to d3/d15/d5f 0 2024-03-18T14:50:50.728 INFO:tasks.workunit.client.0.smithi012.stdout:5/268: fdatasync d3/d15/f37 0 2024-03-18T14:50:50.732 INFO:tasks.workunit.client.0.smithi012.stdout:7/256: rename dd/d13/d23/d33/f49 to dd/d13/d23/d2f/d53/f5d 0 2024-03-18T14:50:50.732 INFO:tasks.workunit.client.0.smithi012.stdout:7/257: chown dd/d13/d23/d33/d37/l40 1 1 2024-03-18T14:50:50.737 INFO:tasks.workunit.client.0.smithi012.stdout:7/258: truncate f1 3720011 0 2024-03-18T14:50:50.739 INFO:tasks.workunit.client.0.smithi012.stdout:2/257: rename d0/d10/d18/l1e to d0/d10/l4d 0 2024-03-18T14:50:50.741 INFO:tasks.workunit.client.0.smithi012.stdout:5/269: dwrite d3/d1e/d33/f34 [0,4194304] 0 2024-03-18T14:50:50.741 INFO:tasks.workunit.client.0.smithi012.stdout:7/259: creat dd/d13/d26/f5e x:0 0 0 2024-03-18T14:50:50.742 INFO:tasks.workunit.client.0.smithi012.stdout:7/260: write dd/d13/d23/d33/d48/f5c [1489754,57927] 0 2024-03-18T14:50:50.742 INFO:tasks.workunit.client.0.smithi012.stdout:7/261: creat dd/f5f x:0 0 0 2024-03-18T14:50:50.743 INFO:tasks.workunit.client.0.smithi012.stdout:6/245: rename d17/d19/c39 to d17/d33/c42 0 2024-03-18T14:50:50.748 INFO:tasks.workunit.client.0.smithi012.stdout:5/270: symlink d3/d1e/l60 0 2024-03-18T14:50:50.749 INFO:tasks.workunit.client.0.smithi012.stdout:2/258: dread d0/f5 [0,4194304] 0 2024-03-18T14:50:50.751 INFO:tasks.workunit.client.0.smithi012.stdout:7/262: mkdir dd/d13/d23/d60 0 2024-03-18T14:50:50.753 INFO:tasks.workunit.client.0.smithi012.stdout:6/246: creat d17/d19/d27/d2f/f43 x:0 0 0 2024-03-18T14:50:50.753 INFO:tasks.workunit.client.0.smithi012.stdout:6/247: creat d17/f44 x:0 0 0 2024-03-18T14:50:50.758 INFO:tasks.workunit.client.0.smithi012.stdout:6/248: creat d17/d18/f45 x:0 0 0 2024-03-18T14:50:50.759 INFO:tasks.workunit.client.0.smithi012.stdout:5/271: dread d3/d15/f1b [0,4194304] 0 2024-03-18T14:50:50.762 INFO:tasks.workunit.client.0.smithi012.stdout:6/249: rename d17/c1d to d17/d26/c46 0 2024-03-18T14:50:50.763 INFO:tasks.workunit.client.0.smithi012.stdout:2/259: write d0/d10/f16 [698869,105166] 0 2024-03-18T14:50:50.763 INFO:tasks.workunit.client.0.smithi012.stdout:2/260: fsync d0/fa 0 2024-03-18T14:50:50.763 INFO:tasks.workunit.client.0.smithi012.stdout:2/261: write d0/d10/d18/f1b [4431314,125515] 0 2024-03-18T14:50:50.764 INFO:tasks.workunit.client.0.smithi012.stdout:7/263: read dd/d13/d23/d2a/f3e [831993,96658] 0 2024-03-18T14:50:50.765 INFO:tasks.workunit.client.0.smithi012.stdout:7/264: stat dd/f5a 0 2024-03-18T14:50:50.765 INFO:tasks.workunit.client.0.smithi012.stdout:6/250: symlink d17/d38/d3f/l47 0 2024-03-18T14:50:50.766 INFO:tasks.workunit.client.0.smithi012.stdout:6/251: chown d17/d19/f3e 1628 1 2024-03-18T14:50:50.766 INFO:tasks.workunit.client.0.smithi012.stdout:6/252: dread - d17/f2c zero size 2024-03-18T14:50:50.768 INFO:tasks.workunit.client.0.smithi012.stdout:5/272: dread d3/f3a [0,4194304] 0 2024-03-18T14:50:50.768 INFO:tasks.workunit.client.0.smithi012.stdout:5/273: fdatasync d3/f22 0 2024-03-18T14:50:50.768 INFO:tasks.workunit.client.0.smithi012.stdout:5/274: read d3/f1f [6621571,6898] 0 2024-03-18T14:50:50.768 INFO:tasks.workunit.client.0.smithi012.stdout:5/275: creat d3/d29/f61 x:0 0 0 2024-03-18T14:50:50.770 INFO:tasks.workunit.client.0.smithi012.stdout:7/265: unlink dd/f42 0 2024-03-18T14:50:50.777 INFO:tasks.workunit.client.0.smithi012.stdout:5/276: unlink d3/d29/f41 0 2024-03-18T14:50:50.784 INFO:tasks.workunit.client.0.smithi012.stdout:2/262: dwrite d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:50.784 INFO:tasks.workunit.client.0.smithi012.stdout:6/253: dwrite f4 [0,4194304] 0 2024-03-18T14:50:50.788 INFO:tasks.workunit.client.0.smithi012.stdout:5/277: dread d3/d1e/f13 [0,4194304] 0 2024-03-18T14:50:50.788 INFO:tasks.workunit.client.0.smithi012.stdout:5/278: stat d3/d15/f37 0 2024-03-18T14:50:50.790 INFO:tasks.workunit.client.0.smithi012.stdout:2/263: rename d0/f2c to d0/d3c/d40/d47/f4e 0 2024-03-18T14:50:50.790 INFO:tasks.workunit.client.0.smithi012.stdout:2/264: write d0/d10/f32 [740527,81724] 0 2024-03-18T14:50:50.793 INFO:tasks.workunit.client.0.smithi012.stdout:6/254: creat d17/d33/f48 x:0 0 0 2024-03-18T14:50:50.793 INFO:tasks.workunit.client.0.smithi012.stdout:6/255: truncate d17/f35 1108935 0 2024-03-18T14:50:50.796 INFO:tasks.workunit.client.0.smithi012.stdout:6/256: symlink d17/d26/d36/l49 0 2024-03-18T14:50:50.796 INFO:tasks.workunit.client.0.smithi012.stdout:6/257: stat d17/f1a 0 2024-03-18T14:50:50.818 INFO:tasks.workunit.client.0.smithi012.stdout:5/279: dwrite d3/d15/f1c [4194304,4194304] 0 2024-03-18T14:50:50.821 INFO:tasks.workunit.client.0.smithi012.stdout:2/265: dwrite d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:50.822 INFO:tasks.workunit.client.0.smithi012.stdout:6/258: dwrite f16 [0,4194304] 0 2024-03-18T14:50:50.822 INFO:tasks.workunit.client.0.smithi012.stdout:6/259: chown d17/d26/d36 71394580 1 2024-03-18T14:50:50.823 INFO:tasks.workunit.client.0.smithi012.stdout:2/266: mkdir d0/d10/d18/d4f 0 2024-03-18T14:50:50.825 INFO:tasks.workunit.client.0.smithi012.stdout:6/260: link d17/f2c d17/d26/f4a 0 2024-03-18T14:50:50.825 INFO:tasks.workunit.client.0.smithi012.stdout:6/261: chown d17/d19/f20 294 1 2024-03-18T14:50:50.825 INFO:tasks.workunit.client.0.smithi012.stdout:6/262: read - d17/f3b zero size 2024-03-18T14:50:50.825 INFO:tasks.workunit.client.0.smithi012.stdout:6/263: write d17/d19/f3e [777948,24436] 0 2024-03-18T14:50:50.826 INFO:tasks.workunit.client.0.smithi012.stdout:5/280: write d3/f43 [2254939,12339] 0 2024-03-18T14:50:50.833 INFO:tasks.workunit.client.0.smithi012.stdout:5/281: truncate d3/d15/f31 69806 0 2024-03-18T14:50:50.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/267: dwrite d0/d10/f32 [0,4194304] 0 2024-03-18T14:50:50.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/268: chown d0/d3c/d40/d47 106 1 2024-03-18T14:50:50.841 INFO:tasks.workunit.client.0.smithi012.stdout:5/282: rename d3/d1e/d33/d55 to d3/d1e/d33/d62 0 2024-03-18T14:50:50.842 INFO:tasks.workunit.client.0.smithi012.stdout:5/283: fsync d3/d15/f2c 0 2024-03-18T14:50:50.842 INFO:tasks.workunit.client.0.smithi012.stdout:5/284: dread - d3/d29/f57 zero size 2024-03-18T14:50:50.842 INFO:tasks.workunit.client.0.smithi012.stdout:5/285: chown d3/d15/d5f/c51 8093714 1 2024-03-18T14:50:50.846 INFO:tasks.workunit.client.0.smithi012.stdout:2/269: rename d0/c22 to d0/d3c/c50 0 2024-03-18T14:50:50.846 INFO:tasks.workunit.client.0.smithi012.stdout:2/270: chown d0/d10/d18/d31 2 1 2024-03-18T14:50:50.848 INFO:tasks.workunit.client.0.smithi012.stdout:5/286: truncate d3/d29/d36/d3d/f3e 428353 0 2024-03-18T14:50:50.850 INFO:tasks.workunit.client.0.smithi012.stdout:2/271: creat d0/d10/d18/d31/d4c/f51 x:0 0 0 2024-03-18T14:50:50.853 INFO:tasks.workunit.client.0.smithi012.stdout:5/287: link d3/d1e/l17 d3/d1e/l63 0 2024-03-18T14:50:50.855 INFO:tasks.workunit.client.0.smithi012.stdout:2/272: symlink d0/d10/d18/d31/l52 0 2024-03-18T14:50:50.858 INFO:tasks.workunit.client.0.smithi012.stdout:2/273: link d0/d10/d18/d31/f35 d0/f53 0 2024-03-18T14:50:50.865 INFO:tasks.workunit.client.0.smithi012.stdout:5/288: write d3/d1e/f28 [1210590,116049] 0 2024-03-18T14:50:50.868 INFO:tasks.workunit.client.0.smithi012.stdout:5/289: mknod d3/d15/d42/c64 0 2024-03-18T14:50:50.870 INFO:tasks.workunit.client.0.smithi012.stdout:5/290: mkdir d3/d29/d36/d3d/d65 0 2024-03-18T14:50:50.877 INFO:tasks.workunit.client.0.smithi012.stdout:5/291: dread d3/d1e/d33/f34 [0,4194304] 0 2024-03-18T14:50:50.879 INFO:tasks.workunit.client.0.smithi012.stdout:5/292: truncate f2 438935 0 2024-03-18T14:50:50.880 INFO:tasks.workunit.client.0.smithi012.stdout:5/293: readlink d3/l52 0 2024-03-18T14:50:50.882 INFO:tasks.workunit.client.0.smithi012.stdout:5/294: symlink d3/d29/d35/d5b/l66 0 2024-03-18T14:50:50.885 INFO:tasks.workunit.client.0.smithi012.stdout:5/295: symlink d3/d1e/d33/d62/l67 0 2024-03-18T14:50:50.889 INFO:tasks.workunit.client.0.smithi012.stdout:5/296: rename d3/l30 to d3/d1e/d33/l68 0 2024-03-18T14:50:50.896 INFO:tasks.workunit.client.0.smithi012.stdout:5/297: dread d3/d29/d36/f45 [0,4194304] 0 2024-03-18T14:50:50.912 INFO:tasks.workunit.client.0.smithi012.stdout:5/298: dwrite d3/d15/d21/f49 [0,4194304] 0 2024-03-18T14:50:50.912 INFO:tasks.workunit.client.0.smithi012.stdout:5/299: readlink d3/d15/l16 0 2024-03-18T14:50:50.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/3: creat f1 x:0 0 0 2024-03-18T14:50:50.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/4: write f1 [402895,67029] 0 2024-03-18T14:50:50.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/5: rmdir - no directory 2024-03-18T14:50:50.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/6: read f1 [258063,114904] 0 2024-03-18T14:50:50.935 INFO:tasks.workunit.client.1.smithi100.stdout:9/0: creat f0 x:0 0 0 2024-03-18T14:50:50.948 INFO:tasks.workunit.client.1.smithi100.stdout:8/7: dwrite f1 [0,4194304] 0 2024-03-18T14:50:50.957 INFO:tasks.workunit.client.1.smithi100.stdout:8/8: read f1 [1357659,104963] 0 2024-03-18T14:50:50.960 INFO:tasks.workunit.client.1.smithi100.stdout:8/9: dread f1 [0,4194304] 0 2024-03-18T14:50:51.016 INFO:tasks.workunit.client.0.smithi012.stdout:8/363: rmdir d0/d5a/d19/d1b/d54 39 2024-03-18T14:50:51.016 INFO:tasks.workunit.client.0.smithi012.stdout:8/364: truncate d0/f16 4208407 0 2024-03-18T14:50:51.016 INFO:tasks.workunit.client.0.smithi012.stdout:8/365: fsync d0/d5a/f6d 0 2024-03-18T14:50:51.027 INFO:tasks.workunit.client.0.smithi012.stdout:8/366: write d0/d5a/f60 [2694040,127703] 0 2024-03-18T14:50:51.045 INFO:tasks.workunit.client.0.smithi012.stdout:8/367: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:51.045 INFO:tasks.workunit.client.0.smithi012.stdout:8/368: dread - d0/d5a/d19/d1b/d54/f72 zero size 2024-03-18T14:50:51.045 INFO:tasks.workunit.client.0.smithi012.stdout:8/369: write d0/d5a/d19/d1b/d2a/f5d [655068,73806] 0 2024-03-18T14:50:51.049 INFO:tasks.workunit.client.0.smithi012.stdout:8/370: link d0/d5a/c23 d0/d5a/d19/d63/c73 0 2024-03-18T14:50:51.053 INFO:tasks.workunit.client.0.smithi012.stdout:8/371: mknod d0/d12/d33/d50/c74 0 2024-03-18T14:50:51.057 INFO:tasks.workunit.client.0.smithi012.stdout:8/372: creat d0/d5a/d19/f75 x:0 0 0 2024-03-18T14:50:51.062 INFO:tasks.workunit.client.0.smithi012.stdout:8/373: dread d0/f6 [0,4194304] 0 2024-03-18T14:50:51.155 INFO:tasks.workunit.client.0.smithi012.stdout:0/251: getdents d1/d20 0 2024-03-18T14:50:51.155 INFO:tasks.workunit.client.0.smithi012.stdout:0/252: dread - d1/d16/d18/f30 zero size 2024-03-18T14:50:51.158 INFO:tasks.workunit.client.0.smithi012.stdout:0/253: symlink d1/d20/d23/d3b/d33/d51/l5c 0 2024-03-18T14:50:51.158 INFO:tasks.workunit.client.0.smithi012.stdout:0/254: fsync d1/d16/d18/f13 0 2024-03-18T14:50:51.166 INFO:tasks.workunit.client.0.smithi012.stdout:0/255: read d1/d20/d23/d26/f40 [1957866,10785] 0 2024-03-18T14:50:51.168 INFO:tasks.workunit.client.0.smithi012.stdout:0/256: mknod d1/c5d 0 2024-03-18T14:50:51.196 INFO:tasks.workunit.client.0.smithi012.stdout:1/259: write d8/db/f41 [4461602,74790] 0 2024-03-18T14:50:51.196 INFO:tasks.workunit.client.0.smithi012.stdout:1/260: fsync d8/d1e/d27/f38 0 2024-03-18T14:50:51.212 INFO:tasks.workunit.client.0.smithi012.stdout:1/261: dwrite d8/db/d10/d16/d1c/f2c [0,4194304] 0 2024-03-18T14:50:51.216 INFO:tasks.workunit.client.0.smithi012.stdout:1/262: unlink d8/d1e/d27/d44/c47 0 2024-03-18T14:50:51.216 INFO:tasks.workunit.client.0.smithi012.stdout:1/263: write d8/d1e/f4a [813166,15187] 0 2024-03-18T14:50:51.218 INFO:tasks.workunit.client.0.smithi012.stdout:1/264: creat d8/db/d10/d31/d43/f51 x:0 0 0 2024-03-18T14:50:51.218 INFO:tasks.workunit.client.0.smithi012.stdout:1/265: chown d8/db/d10/d31/d43/f51 28097258 1 2024-03-18T14:50:51.220 INFO:tasks.workunit.client.0.smithi012.stdout:1/266: mknod d8/db/d10/d31/c52 0 2024-03-18T14:50:51.239 INFO:tasks.workunit.client.0.smithi012.stdout:1/267: dwrite d8/d1e/f2f [0,4194304] 0 2024-03-18T14:50:51.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:50 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:51.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:50 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:51.337 INFO:tasks.workunit.client.0.smithi012.stdout:9/274: truncate d3/d9/d19/f3a 3863823 0 2024-03-18T14:50:51.572 INFO:tasks.workunit.client.0.smithi012.stdout:6/264: unlink d17/d18/f1f 0 2024-03-18T14:50:51.573 INFO:tasks.workunit.client.0.smithi012.stdout:6/265: write d17/f35 [535375,8607] 0 2024-03-18T14:50:51.573 INFO:tasks.workunit.client.0.smithi012.stdout:6/266: dread - d17/d18/f45 zero size 2024-03-18T14:50:51.574 INFO:tasks.workunit.client.0.smithi012.stdout:6/267: write f12 [978991,17633] 0 2024-03-18T14:50:51.576 INFO:tasks.workunit.client.0.smithi012.stdout:6/268: creat d17/d26/d36/f4b x:0 0 0 2024-03-18T14:50:51.700 INFO:tasks.workunit.client.1.smithi100.stdout:9/1: fsync f0 0 2024-03-18T14:50:51.719 INFO:tasks.workunit.client.1.smithi100.stdout:9/2: dwrite f0 [0,4194304] 0 2024-03-18T14:50:51.719 INFO:tasks.workunit.client.1.smithi100.stdout:9/3: fsync f0 0 2024-03-18T14:50:51.887 INFO:tasks.workunit.client.0.smithi012.stdout:9/275: creat d3/d1e/d21/f4f x:0 0 0 2024-03-18T14:50:51.887 INFO:tasks.workunit.client.0.smithi012.stdout:9/276: readlink d3/d1e/l37 0 2024-03-18T14:50:51.890 INFO:tasks.workunit.client.0.smithi012.stdout:9/277: symlink d3/d9/d19/l50 0 2024-03-18T14:50:51.892 INFO:tasks.workunit.client.0.smithi012.stdout:9/278: creat d3/d1e/f51 x:0 0 0 2024-03-18T14:50:51.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/279: mkdir d3/d1e/d21/d4a/d52 0 2024-03-18T14:50:51.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/280: write d3/f4 [409744,118212] 0 2024-03-18T14:50:51.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/281: read - d3/d1e/f45 zero size 2024-03-18T14:50:51.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/282: readlink d3/d1e/d21/d35/d48/l2d 0 2024-03-18T14:50:51.910 INFO:tasks.workunit.client.0.smithi012.stdout:9/283: dwrite d3/d1e/f44 [0,4194304] 0 2024-03-18T14:50:51.941 INFO:tasks.workunit.client.1.smithi100.stdout:0/19: symlink lb 0 2024-03-18T14:50:52.015 INFO:tasks.workunit.client.1.smithi100.stdout:7/7: symlink l2 0 2024-03-18T14:50:52.089 INFO:tasks.workunit.client.1.smithi100.stdout:6/11: getdents . 0 2024-03-18T14:50:52.089 INFO:tasks.workunit.client.1.smithi100.stdout:0/20: mknod cc 0 2024-03-18T14:50:52.091 INFO:tasks.workunit.client.1.smithi100.stdout:7/8: rename f1 to f3 0 2024-03-18T14:50:52.092 INFO:tasks.workunit.client.1.smithi100.stdout:0/21: mkdir dd 0 2024-03-18T14:50:52.092 INFO:tasks.workunit.client.1.smithi100.stdout:0/22: stat lb 0 2024-03-18T14:50:52.115 INFO:tasks.workunit.client.1.smithi100.stdout:7/9: dwrite f3 [0,4194304] 0 2024-03-18T14:50:52.115 INFO:tasks.workunit.client.1.smithi100.stdout:0/23: dwrite f2 [0,4194304] 0 2024-03-18T14:50:52.115 INFO:tasks.workunit.client.1.smithi100.stdout:0/24: stat f3 0 2024-03-18T14:50:52.115 INFO:tasks.workunit.client.1.smithi100.stdout:7/10: chown f3 817370226 1 2024-03-18T14:50:52.115 INFO:tasks.workunit.client.1.smithi100.stdout:7/11: creat f4 x:0 0 0 2024-03-18T14:50:52.119 INFO:tasks.workunit.client.1.smithi100.stdout:7/12: mkdir d5 0 2024-03-18T14:50:52.120 INFO:tasks.workunit.client.1.smithi100.stdout:7/13: creat d5/f6 x:0 0 0 2024-03-18T14:50:52.120 INFO:tasks.workunit.client.1.smithi100.stdout:7/14: rename d5 to d5/d7 22 2024-03-18T14:50:52.121 INFO:tasks.workunit.client.1.smithi100.stdout:7/15: symlink d5/l8 0 2024-03-18T14:50:52.123 INFO:tasks.workunit.client.1.smithi100.stdout:7/16: rename c0 to d5/c9 0 2024-03-18T14:50:52.124 INFO:tasks.workunit.client.1.smithi100.stdout:0/25: dread f3 [0,4194304] 0 2024-03-18T14:50:52.125 INFO:tasks.workunit.client.1.smithi100.stdout:0/26: creat dd/fe x:0 0 0 2024-03-18T14:50:52.125 INFO:tasks.workunit.client.1.smithi100.stdout:0/27: creat dd/ff x:0 0 0 2024-03-18T14:50:52.125 INFO:tasks.workunit.client.1.smithi100.stdout:0/28: truncate f6 631566 0 2024-03-18T14:50:52.129 INFO:tasks.workunit.client.1.smithi100.stdout:0/29: dread f7 [0,4194304] 0 2024-03-18T14:50:52.130 INFO:tasks.workunit.client.1.smithi100.stdout:0/30: mknod dd/c10 0 2024-03-18T14:50:52.130 INFO:tasks.workunit.client.1.smithi100.stdout:0/31: creat dd/f11 x:0 0 0 2024-03-18T14:50:52.131 INFO:tasks.workunit.client.1.smithi100.stdout:7/17: dread f3 [0,4194304] 0 2024-03-18T14:50:52.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:51 smithi012 ceph-mon[29182]: pgmap v243: 65 pgs: 65 active+clean; 693 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 53 MiB/s rd, 73 MiB/s wr, 232 op/s 2024-03-18T14:50:52.270 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:51 smithi100 ceph-mon[37807]: pgmap v243: 65 pgs: 65 active+clean; 693 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 53 MiB/s rd, 73 MiB/s wr, 232 op/s 2024-03-18T14:50:52.287 INFO:tasks.workunit.client.0.smithi012.stdout:0/257: rmdir d1/d20/d23 39 2024-03-18T14:50:52.288 INFO:tasks.workunit.client.0.smithi012.stdout:0/258: mknod d1/d16/d18/c5e 0 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:3/193: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:8/374: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:4/296: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:2/274: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:2/275: fdatasync d0/f53 0 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:5/300: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:6/269: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:1/268: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:6/270: write d17/d33/f48 [520901,124762] 0 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:7/266: sync 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:7/267: chown dd/f59 6630853 1 2024-03-18T14:50:52.291 INFO:tasks.workunit.client.0.smithi012.stdout:0/259: creat d1/f5f x:0 0 0 2024-03-18T14:50:52.292 INFO:tasks.workunit.client.0.smithi012.stdout:0/260: write d1/f9 [5226849,70141] 0 2024-03-18T14:50:52.292 INFO:tasks.workunit.client.0.smithi012.stdout:0/261: fdatasync d1/d16/f17 0 2024-03-18T14:50:52.292 INFO:tasks.workunit.client.0.smithi012.stdout:0/262: read - d1/d20/d4e/d54/f57 zero size 2024-03-18T14:50:52.292 INFO:tasks.workunit.client.0.smithi012.stdout:3/194: symlink dc/d30/l3b 0 2024-03-18T14:50:52.292 INFO:tasks.workunit.client.0.smithi012.stdout:3/195: truncate dc/ff 768229 0 2024-03-18T14:50:52.294 INFO:tasks.workunit.client.0.smithi012.stdout:2/276: truncate d0/f21 1653332 0 2024-03-18T14:50:52.294 INFO:tasks.workunit.client.0.smithi012.stdout:9/284: sync 2024-03-18T14:50:52.296 INFO:tasks.workunit.client.0.smithi012.stdout:9/285: read d3/d1e/f2b [495940,4378] 0 2024-03-18T14:50:52.296 INFO:tasks.workunit.client.0.smithi012.stdout:5/301: link d3/d29/d35/l3c d3/d1e/d33/l69 0 2024-03-18T14:50:52.297 INFO:tasks.workunit.client.0.smithi012.stdout:1/269: truncate d8/f40 2190890 0 2024-03-18T14:50:52.300 INFO:tasks.workunit.client.0.smithi012.stdout:8/375: dread d0/d5a/d19/d1b/f27 [0,4194304] 0 2024-03-18T14:50:52.300 INFO:tasks.workunit.client.0.smithi012.stdout:6/271: dread fe [4194304,4194304] 0 2024-03-18T14:50:52.300 INFO:tasks.workunit.client.0.smithi012.stdout:6/272: dread - d17/d26/f4a zero size 2024-03-18T14:50:52.300 INFO:tasks.workunit.client.0.smithi012.stdout:7/268: symlink dd/d13/d23/d33/l61 0 2024-03-18T14:50:52.304 INFO:tasks.workunit.client.0.smithi012.stdout:0/263: link d1/f4f d1/d2d/f60 0 2024-03-18T14:50:52.308 INFO:tasks.workunit.client.0.smithi012.stdout:4/297: link d6/d19/c1e d6/d1a/d30/c6a 0 2024-03-18T14:50:52.313 INFO:tasks.workunit.client.0.smithi012.stdout:2/277: symlink d0/d3c/d40/l54 0 2024-03-18T14:50:52.315 INFO:tasks.workunit.client.0.smithi012.stdout:9/286: symlink d3/d1e/d21/l53 0 2024-03-18T14:50:52.315 INFO:tasks.workunit.client.0.smithi012.stdout:0/264: read d1/d20/d23/f46 [2173811,55324] 0 2024-03-18T14:50:52.316 INFO:tasks.workunit.client.0.smithi012.stdout:5/302: mknod d3/d15/d21/d59/c6a 0 2024-03-18T14:50:52.318 INFO:tasks.workunit.client.0.smithi012.stdout:3/196: mkdir dc/d3c 0 2024-03-18T14:50:52.321 INFO:tasks.workunit.client.0.smithi012.stdout:4/298: link d6/f33 d6/d1a/d30/d46/f6b 0 2024-03-18T14:50:52.323 INFO:tasks.workunit.client.0.smithi012.stdout:9/287: mkdir d3/d1e/d21/d35/d48/d54 0 2024-03-18T14:50:52.324 INFO:tasks.workunit.client.0.smithi012.stdout:0/265: mknod d1/d20/d23/d3b/d33/c61 0 2024-03-18T14:50:52.324 INFO:tasks.workunit.client.0.smithi012.stdout:4/299: dread d6/d19/f1d [0,4194304] 0 2024-03-18T14:50:52.325 INFO:tasks.workunit.client.0.smithi012.stdout:3/197: dread dc/f26 [0,4194304] 0 2024-03-18T14:50:52.326 INFO:tasks.workunit.client.0.smithi012.stdout:9/288: rename d3/d20/l4e to d3/d1e/d21/d4a/d52/l55 0 2024-03-18T14:50:52.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/270: dwrite d8/db/d10/d16/d1c/f23 [0,4194304] 0 2024-03-18T14:50:52.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/271: write d8/db/d10/d16/d1c/f4c [834718,91025] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:3/198: dread dc/f26 [0,4194304] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:3/199: write f3 [852097,111986] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:8/376: dwrite d0/d5a/d19/d1b/d54/f72 [0,4194304] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:0/266: creat d1/d2d/f62 x:0 0 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:7/269: dwrite dd/d13/d23/d33/f4d [0,4194304] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:5/303: dread d3/d1e/f28 [4194304,4194304] 0 2024-03-18T14:50:52.330 INFO:tasks.workunit.client.0.smithi012.stdout:7/270: chown dd/d13/l18 6 1 2024-03-18T14:50:52.331 INFO:tasks.workunit.client.0.smithi012.stdout:9/289: creat d3/d1e/d21/d4a/f56 x:0 0 0 2024-03-18T14:50:52.336 INFO:tasks.workunit.client.0.smithi012.stdout:1/272: creat d8/db/d10/d4e/f53 x:0 0 0 2024-03-18T14:50:52.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/200: rename dc/l18 to dc/d30/d3a/l3d 0 2024-03-18T14:50:52.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/201: chown dc/f27 4028 1 2024-03-18T14:50:52.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/202: dread - dc/d30/d35/f2c zero size 2024-03-18T14:50:52.343 INFO:tasks.workunit.client.0.smithi012.stdout:8/377: truncate d0/f6 1985640 0 2024-03-18T14:50:52.343 INFO:tasks.workunit.client.0.smithi012.stdout:8/378: creat d0/d5a/d19/d57/f76 x:0 0 0 2024-03-18T14:50:52.343 INFO:tasks.workunit.client.0.smithi012.stdout:8/379: creat d0/d5a/d19/d1b/f77 x:0 0 0 2024-03-18T14:50:52.343 INFO:tasks.workunit.client.0.smithi012.stdout:8/380: fdatasync d0/d12/f4d 0 2024-03-18T14:50:52.344 INFO:tasks.workunit.client.0.smithi012.stdout:4/300: dwrite d6/f33 [0,4194304] 0 2024-03-18T14:50:52.344 INFO:tasks.workunit.client.0.smithi012.stdout:4/301: chown d6/da/f16 89297647 1 2024-03-18T14:50:52.345 INFO:tasks.workunit.client.0.smithi012.stdout:0/267: rename d1/d20/c49 to d1/d20/c63 0 2024-03-18T14:50:52.349 INFO:tasks.workunit.client.0.smithi012.stdout:7/271: creat dd/d13/d23/d60/f62 x:0 0 0 2024-03-18T14:50:52.353 INFO:tasks.workunit.client.0.smithi012.stdout:8/381: truncate d0/f7 735879 0 2024-03-18T14:50:52.356 INFO:tasks.workunit.client.0.smithi012.stdout:7/272: symlink dd/d13/d23/d33/l63 0 2024-03-18T14:50:52.357 INFO:tasks.workunit.client.0.smithi012.stdout:7/273: dread - dd/d13/d23/d60/f62 zero size 2024-03-18T14:50:52.357 INFO:tasks.workunit.client.0.smithi012.stdout:7/274: chown dd/d13/d23/d2a/f3e 1 1 2024-03-18T14:50:52.358 INFO:tasks.workunit.client.0.smithi012.stdout:4/302: write d6/d1a/d30/d46/f6b [3959280,119743] 0 2024-03-18T14:50:52.358 INFO:tasks.workunit.client.0.smithi012.stdout:8/382: getdents d0/d12/d33/d50 0 2024-03-18T14:50:52.358 INFO:tasks.workunit.client.0.smithi012.stdout:4/303: readlink d6/da/l3f 0 2024-03-18T14:50:52.369 INFO:tasks.workunit.client.0.smithi012.stdout:8/383: dread d0/d15/f43 [0,4194304] 0 2024-03-18T14:50:52.372 INFO:tasks.workunit.client.0.smithi012.stdout:4/304: dread d6/d19/f1c [0,4194304] 0 2024-03-18T14:50:52.373 INFO:tasks.workunit.client.0.smithi012.stdout:4/305: fdatasync d6/d1a/f2a 0 2024-03-18T14:50:52.374 INFO:tasks.workunit.client.0.smithi012.stdout:4/306: rmdir d6/d1a/d30 39 2024-03-18T14:50:52.374 INFO:tasks.workunit.client.0.smithi012.stdout:4/307: rmdir d6/d1a/d30/d46 39 2024-03-18T14:50:52.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/308: mkdir d6/d19/d6c 0 2024-03-18T14:50:52.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/309: stat d6/d1a 0 2024-03-18T14:50:52.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/310: chown d6/ff 15837160 1 2024-03-18T14:50:52.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/311: chown d6/da/d58/c5f 979 1 2024-03-18T14:50:52.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/312: truncate d6/f5c 5167632 0 2024-03-18T14:50:52.380 INFO:tasks.workunit.client.0.smithi012.stdout:5/304: dwrite d3/d15/f19 [0,4194304] 0 2024-03-18T14:50:52.380 INFO:tasks.workunit.client.0.smithi012.stdout:9/290: dwrite d3/d9/fd [0,4194304] 0 2024-03-18T14:50:52.381 INFO:tasks.workunit.client.0.smithi012.stdout:9/291: creat d3/d49/f57 x:0 0 0 2024-03-18T14:50:52.383 INFO:tasks.workunit.client.0.smithi012.stdout:9/292: creat d3/d1e/d21/d35/d48/d54/f58 x:0 0 0 2024-03-18T14:50:52.383 INFO:tasks.workunit.client.0.smithi012.stdout:9/293: write d3/f2e [344764,30414] 0 2024-03-18T14:50:52.384 INFO:tasks.workunit.client.0.smithi012.stdout:9/294: creat d3/d1e/d21/f59 x:0 0 0 2024-03-18T14:50:52.387 INFO:tasks.workunit.client.0.smithi012.stdout:1/273: mkdir d8/db/d10/d16/d54 0 2024-03-18T14:50:52.388 INFO:tasks.workunit.client.0.smithi012.stdout:9/295: dread d3/d20/f43 [4194304,4194304] 0 2024-03-18T14:50:52.389 INFO:tasks.workunit.client.0.smithi012.stdout:1/274: rename d8/db/d10/d16/d1c/d2e/c37 to d8/db/d10/d31/c55 0 2024-03-18T14:50:52.390 INFO:tasks.workunit.client.0.smithi012.stdout:1/275: truncate d8/db/f15 2575474 0 2024-03-18T14:50:52.392 INFO:tasks.workunit.client.0.smithi012.stdout:9/296: dread d3/d20/f47 [0,4194304] 0 2024-03-18T14:50:52.392 INFO:tasks.workunit.client.0.smithi012.stdout:9/297: stat d3/d1e/d21/d35 0 2024-03-18T14:50:52.392 INFO:tasks.workunit.client.0.smithi012.stdout:0/268: dwrite d1/d20/d23/d26/f53 [0,4194304] 0 2024-03-18T14:50:52.392 INFO:tasks.workunit.client.0.smithi012.stdout:3/203: dwrite dc/f26 [0,4194304] 0 2024-03-18T14:50:52.394 INFO:tasks.workunit.client.0.smithi012.stdout:0/269: rename d1/d20/d4d to d1/d20/d23/d3b/d33/d64 0 2024-03-18T14:50:52.396 INFO:tasks.workunit.client.0.smithi012.stdout:8/384: dwrite d0/d12/f4b [4194304,4194304] 0 2024-03-18T14:50:52.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/275: dwrite dd/f5a [4194304,4194304] 0 2024-03-18T14:50:52.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/276: dread - dd/f5f zero size 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:5/305: dwrite d3/d1e/d33/f34 [0,4194304] 0 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:0/270: link d1/d16/d18/f30 d1/d20/d23/f65 0 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:0/271: dread - d1/d20/d4e/f52 zero size 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:0/272: stat d1/d20/d23/d26/f36 0 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:5/306: write d3/d15/f37 [70788,105134] 0 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:5/307: chown d3/d15/d21 4116238 1 2024-03-18T14:50:52.399 INFO:tasks.workunit.client.0.smithi012.stdout:5/308: stat d3/d29/d36/f45 0 2024-03-18T14:50:52.400 INFO:tasks.workunit.client.0.smithi012.stdout:1/276: mkdir d8/db/d10/d56 0 2024-03-18T14:50:52.400 INFO:tasks.workunit.client.0.smithi012.stdout:8/385: stat d0/d5a/c23 0 2024-03-18T14:50:52.401 INFO:tasks.workunit.client.0.smithi012.stdout:7/277: mknod dd/d13/d23/d2f/d53/c64 0 2024-03-18T14:50:52.401 INFO:tasks.workunit.client.0.smithi012.stdout:7/278: chown dd/d13/d23/d33/c54 57861996 1 2024-03-18T14:50:52.403 INFO:tasks.workunit.client.0.smithi012.stdout:9/298: dread d3/f3e [0,4194304] 0 2024-03-18T14:50:52.404 INFO:tasks.workunit.client.0.smithi012.stdout:0/273: rmdir d1/d20 39 2024-03-18T14:50:52.406 INFO:tasks.workunit.client.0.smithi012.stdout:5/309: mkdir d3/d1e/d33/d62/d6b 0 2024-03-18T14:50:52.408 INFO:tasks.workunit.client.0.smithi012.stdout:1/277: link d8/db/l1b d8/db/d10/d31/d45/l57 0 2024-03-18T14:50:52.411 INFO:tasks.workunit.client.0.smithi012.stdout:0/274: symlink d1/d16/d31/l66 0 2024-03-18T14:50:52.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/386: dread d0/d5a/d19/d1b/d2a/d3e/f4a [0,4194304] 0 2024-03-18T14:50:52.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/387: truncate d0/f10 859689 0 2024-03-18T14:50:52.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/388: chown d0/d5a/d19/d1b/d2a/d3e/f5f 1152 1 2024-03-18T14:50:52.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/389: chown d0/d5a/d55/c6c 304413 1 2024-03-18T14:50:52.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/390: chown d0/d15/c22 949 1 2024-03-18T14:50:52.413 INFO:tasks.workunit.client.0.smithi012.stdout:7/279: dread dd/d13/d23/d33/f4d [0,4194304] 0 2024-03-18T14:50:52.415 INFO:tasks.workunit.client.0.smithi012.stdout:0/275: mkdir d1/d20/d4e/d54/d67 0 2024-03-18T14:50:52.415 INFO:tasks.workunit.client.0.smithi012.stdout:5/310: dread d3/f3a [0,4194304] 0 2024-03-18T14:50:52.418 INFO:tasks.workunit.client.0.smithi012.stdout:8/391: rmdir d0/d5a/d19/d1b/d2a 39 2024-03-18T14:50:52.418 INFO:tasks.workunit.client.0.smithi012.stdout:8/392: chown d0/d5a/d19/d57 253923727 1 2024-03-18T14:50:52.420 INFO:tasks.workunit.client.0.smithi012.stdout:1/278: dread d8/f1a [0,4194304] 0 2024-03-18T14:50:52.420 INFO:tasks.workunit.client.0.smithi012.stdout:5/311: read d3/d15/f1c [6898016,20578] 0 2024-03-18T14:50:52.421 INFO:tasks.workunit.client.0.smithi012.stdout:7/280: rename dd/d13/d23/d33/f4d to dd/d13/d23/d2f/f65 0 2024-03-18T14:50:52.423 INFO:tasks.workunit.client.0.smithi012.stdout:9/299: dwrite d3/f25 [0,4194304] 0 2024-03-18T14:50:52.424 INFO:tasks.workunit.client.0.smithi012.stdout:0/276: truncate d1/f3 1029674 0 2024-03-18T14:50:52.425 INFO:tasks.workunit.client.0.smithi012.stdout:8/393: mkdir d0/d15/d78 0 2024-03-18T14:50:52.427 INFO:tasks.workunit.client.0.smithi012.stdout:1/279: symlink d8/db/d10/d16/d54/l58 0 2024-03-18T14:50:52.430 INFO:tasks.workunit.client.0.smithi012.stdout:5/312: truncate d3/d29/d36/f45 5763733 0 2024-03-18T14:50:52.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/313: read - d3/d29/d36/d3d/f4a zero size 2024-03-18T14:50:52.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/314: write d3/d15/f31 [1021597,121701] 0 2024-03-18T14:50:52.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/315: readlink d3/d29/d35/l3c 0 2024-03-18T14:50:52.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/316: dread - d3/d29/f61 zero size 2024-03-18T14:50:52.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/317: write d3/d29/d36/d3d/f47 [1103655,43238] 0 2024-03-18T14:50:52.438 INFO:tasks.workunit.client.0.smithi012.stdout:9/300: mknod d3/d20/c5a 0 2024-03-18T14:50:52.443 INFO:tasks.workunit.client.0.smithi012.stdout:0/277: link d1/d16/d19/f21 d1/d16/d19/f68 0 2024-03-18T14:50:52.443 INFO:tasks.workunit.client.0.smithi012.stdout:0/278: chown d1/d16/c4c 7271628 1 2024-03-18T14:50:52.448 INFO:tasks.workunit.client.0.smithi012.stdout:7/281: dwrite dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:52.448 INFO:tasks.workunit.client.0.smithi012.stdout:8/394: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:52.448 INFO:tasks.workunit.client.0.smithi012.stdout:8/395: dread - d0/d5a/d19/d57/f76 zero size 2024-03-18T14:50:52.448 INFO:tasks.workunit.client.0.smithi012.stdout:8/396: readlink d0/l34 0 2024-03-18T14:50:52.449 INFO:tasks.workunit.client.0.smithi012.stdout:8/397: dread d0/d15/f1a [4194304,4194304] 0 2024-03-18T14:50:52.461 INFO:tasks.workunit.client.0.smithi012.stdout:9/301: mknod d3/d49/c5b 0 2024-03-18T14:50:52.466 INFO:tasks.workunit.client.0.smithi012.stdout:0/279: link d1/d20/d23/l27 d1/d20/d23/l69 0 2024-03-18T14:50:52.468 INFO:tasks.workunit.client.0.smithi012.stdout:9/302: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:52.469 INFO:tasks.workunit.client.0.smithi012.stdout:7/282: symlink dd/d13/d23/d60/l66 0 2024-03-18T14:50:52.473 INFO:tasks.workunit.client.0.smithi012.stdout:8/398: link d0/d5a/d19/d1b/c36 d0/d5a/d19/d63/c79 0 2024-03-18T14:50:52.491 INFO:tasks.workunit.client.0.smithi012.stdout:0/280: creat d1/d20/f6a x:0 0 0 2024-03-18T14:50:52.491 INFO:tasks.workunit.client.0.smithi012.stdout:0/281: chown d1/d20/d23/l3a 19 1 2024-03-18T14:50:52.494 INFO:tasks.workunit.client.0.smithi012.stdout:7/283: mknod dd/d13/d23/d33/d37/c67 0 2024-03-18T14:50:52.495 INFO:tasks.workunit.client.0.smithi012.stdout:8/399: symlink d0/d5a/d19/d63/l7a 0 2024-03-18T14:50:52.497 INFO:tasks.workunit.client.0.smithi012.stdout:0/282: mknod d1/d20/d23/d3b/d33/d64/c6b 0 2024-03-18T14:50:52.498 INFO:tasks.workunit.client.0.smithi012.stdout:0/283: write d1/d16/f17 [22334,44396] 0 2024-03-18T14:50:52.498 INFO:tasks.workunit.client.0.smithi012.stdout:0/284: truncate d1/d16/d31/f58 934190 0 2024-03-18T14:50:52.498 INFO:tasks.workunit.client.0.smithi012.stdout:0/285: truncate d1/d16/d19/f47 612793 0 2024-03-18T14:50:52.498 INFO:tasks.workunit.client.0.smithi012.stdout:0/286: readlink d1/d20/d23/l69 0 2024-03-18T14:50:52.498 INFO:tasks.workunit.client.0.smithi012.stdout:9/303: mkdir d3/d9/d5c 0 2024-03-18T14:50:52.501 INFO:tasks.workunit.client.0.smithi012.stdout:8/400: creat d0/d15/d6e/f7b x:0 0 0 2024-03-18T14:50:52.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/284: dread dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:50:52.503 INFO:tasks.workunit.client.0.smithi012.stdout:9/304: symlink d3/d49/l5d 0 2024-03-18T14:50:52.506 INFO:tasks.workunit.client.0.smithi012.stdout:8/401: creat d0/d12/f7c x:0 0 0 2024-03-18T14:50:52.507 INFO:tasks.workunit.client.0.smithi012.stdout:0/287: dread d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:50:52.507 INFO:tasks.workunit.client.0.smithi012.stdout:0/288: chown d1/f10 135943439 1 2024-03-18T14:50:52.507 INFO:tasks.workunit.client.0.smithi012.stdout:0/289: truncate d1/f59 235581 0 2024-03-18T14:50:52.508 INFO:tasks.workunit.client.0.smithi012.stdout:7/285: creat dd/d13/d23/d2a/f68 x:0 0 0 2024-03-18T14:50:52.510 INFO:tasks.workunit.client.0.smithi012.stdout:9/305: rmdir d3/d20 39 2024-03-18T14:50:52.510 INFO:tasks.workunit.client.0.smithi012.stdout:9/306: chown d3/d1e/d21/f26 55871669 1 2024-03-18T14:50:52.514 INFO:tasks.workunit.client.0.smithi012.stdout:0/290: link d1/d20/d23/d3b/d33/d64/c6b d1/d20/d4e/c6c 0 2024-03-18T14:50:52.514 INFO:tasks.workunit.client.0.smithi012.stdout:0/291: fdatasync d1/d20/f6a 0 2024-03-18T14:50:52.516 INFO:tasks.workunit.client.0.smithi012.stdout:9/307: symlink d3/d1e/l5e 0 2024-03-18T14:50:52.517 INFO:tasks.workunit.client.0.smithi012.stdout:7/286: write fa [3577663,54399] 0 2024-03-18T14:50:52.517 INFO:tasks.workunit.client.0.smithi012.stdout:7/287: truncate dd/d13/d23/d2f/f32 786677 0 2024-03-18T14:50:52.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/292: rename d1/d20/d23/f65 to d1/d20/d23/d3b/d33/f6d 0 2024-03-18T14:50:52.525 INFO:tasks.workunit.client.0.smithi012.stdout:7/288: creat dd/d4f/f69 x:0 0 0 2024-03-18T14:50:52.529 INFO:tasks.workunit.client.0.smithi012.stdout:0/293: mkdir d1/d16/d6e 0 2024-03-18T14:50:52.529 INFO:tasks.workunit.client.0.smithi012.stdout:0/294: stat d1/d20/d23/d3b/d33/c42 0 2024-03-18T14:50:52.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/308: dwrite d3/d9/f39 [0,4194304] 0 2024-03-18T14:50:52.534 INFO:tasks.workunit.client.1.smithi100.stdout:9/4: write f0 [4493356,6886] 0 2024-03-18T14:50:52.534 INFO:tasks.workunit.client.0.smithi012.stdout:0/295: truncate d1/d20/d23/d26/f40 1184879 0 2024-03-18T14:50:52.538 INFO:tasks.workunit.client.1.smithi100.stdout:7/18: rename f3 to d5/fa 0 2024-03-18T14:50:52.538 INFO:tasks.workunit.client.0.smithi012.stdout:7/289: dread fa [0,4194304] 0 2024-03-18T14:50:52.538 INFO:tasks.workunit.client.0.smithi012.stdout:7/290: chown dd/f10 223536 1 2024-03-18T14:50:52.542 INFO:tasks.workunit.client.1.smithi100.stdout:0/32: write f2 [4728688,18050] 0 2024-03-18T14:50:52.544 INFO:tasks.workunit.client.1.smithi100.stdout:9/5: write f0 [2964688,55435] 0 2024-03-18T14:50:52.553 INFO:tasks.workunit.client.0.smithi012.stdout:2/278: creat d0/f55 x:0 0 0 2024-03-18T14:50:52.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/309: link d3/d1e/d21/f4f d3/d20/f5f 0 2024-03-18T14:50:52.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/310: stat d3/d1e/f51 0 2024-03-18T14:50:52.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/311: stat d3/f25 0 2024-03-18T14:50:52.557 INFO:tasks.workunit.client.0.smithi012.stdout:9/312: truncate d3/d1e/d21/f46 728620 0 2024-03-18T14:50:52.559 INFO:tasks.workunit.client.0.smithi012.stdout:0/296: mkdir d1/d20/d23/d6f 0 2024-03-18T14:50:52.559 INFO:tasks.workunit.client.0.smithi012.stdout:0/297: chown d1/d16/d19/c3d 1085884 1 2024-03-18T14:50:52.562 INFO:tasks.workunit.client.0.smithi012.stdout:7/291: rename dd/d13/d23/d33/c41 to dd/d13/d23/d2a/c6a 0 2024-03-18T14:50:52.564 INFO:tasks.workunit.client.0.smithi012.stdout:9/313: creat d3/d1e/d21/d4a/d52/f60 x:0 0 0 2024-03-18T14:50:52.565 INFO:tasks.workunit.client.0.smithi012.stdout:0/298: creat d1/d20/d23/d3b/d33/f70 x:0 0 0 2024-03-18T14:50:52.566 INFO:tasks.workunit.client.0.smithi012.stdout:7/292: stat dd/d13/d23/d33/d37/l4a 0 2024-03-18T14:50:52.568 INFO:tasks.workunit.client.0.smithi012.stdout:2/279: dwrite d0/f55 [0,4194304] 0 2024-03-18T14:50:52.568 INFO:tasks.workunit.client.0.smithi012.stdout:2/280: chown d0/cb 11 1 2024-03-18T14:50:52.569 INFO:tasks.workunit.client.0.smithi012.stdout:9/314: link d3/le d3/d1e/d21/l61 0 2024-03-18T14:50:52.570 INFO:tasks.workunit.client.0.smithi012.stdout:0/299: unlink d1/d16/d31/c5b 0 2024-03-18T14:50:52.570 INFO:tasks.workunit.client.0.smithi012.stdout:0/300: chown d1/d20 1694015 1 2024-03-18T14:50:52.574 INFO:tasks.workunit.client.0.smithi012.stdout:2/281: dread d0/d10/d18/f1b [0,4194304] 0 2024-03-18T14:50:52.574 INFO:tasks.workunit.client.0.smithi012.stdout:2/282: truncate d0/d3c/d40/d47/f4e 857336 0 2024-03-18T14:50:52.574 INFO:tasks.workunit.client.0.smithi012.stdout:2/283: write d0/d10/d18/d33/f37 [147496,108620] 0 2024-03-18T14:50:52.574 INFO:tasks.workunit.client.0.smithi012.stdout:2/284: chown d0/fa 5187 1 2024-03-18T14:50:52.574 INFO:tasks.workunit.client.0.smithi012.stdout:2/285: getdents d0/d10/d18/d33 0 2024-03-18T14:50:52.575 INFO:tasks.workunit.client.0.smithi012.stdout:2/286: readlink d0/lc 0 2024-03-18T14:50:52.577 INFO:tasks.workunit.client.0.smithi012.stdout:7/293: dread f1 [0,4194304] 0 2024-03-18T14:50:52.580 INFO:tasks.workunit.client.0.smithi012.stdout:7/294: mkdir dd/d13/d23/d2a/d6b 0 2024-03-18T14:50:52.580 INFO:tasks.workunit.client.0.smithi012.stdout:7/295: chown dd/d13/d23/d33/d37/c67 23 1 2024-03-18T14:50:52.580 INFO:tasks.workunit.client.0.smithi012.stdout:7/296: write dd/f5b [17134,94235] 0 2024-03-18T14:50:52.581 INFO:tasks.workunit.client.0.smithi012.stdout:9/315: dread d3/f25 [0,4194304] 0 2024-03-18T14:50:52.584 INFO:tasks.workunit.client.0.smithi012.stdout:9/316: unlink d3/d1e/d21/f4f 0 2024-03-18T14:50:52.592 INFO:tasks.workunit.client.0.smithi012.stdout:8/402: creat d0/d5a/f7d x:0 0 0 2024-03-18T14:50:52.593 INFO:tasks.workunit.client.0.smithi012.stdout:8/403: mknod d0/d5a/d19/d1b/d2a/c7e 0 2024-03-18T14:50:52.596 INFO:tasks.workunit.client.0.smithi012.stdout:0/301: dwrite d1/d20/d4e/f52 [0,4194304] 0 2024-03-18T14:50:52.596 INFO:tasks.workunit.client.0.smithi012.stdout:0/302: readlink d1/d16/d19/l34 0 2024-03-18T14:50:52.596 INFO:tasks.workunit.client.0.smithi012.stdout:8/404: symlink d0/d5a/l7f 0 2024-03-18T14:50:52.596 INFO:tasks.workunit.client.0.smithi012.stdout:8/405: truncate d0/d15/d6e/f7b 418426 0 2024-03-18T14:50:52.596 INFO:tasks.workunit.client.0.smithi012.stdout:8/406: dread - d0/d12/d33/f44 zero size 2024-03-18T14:50:52.601 INFO:tasks.workunit.client.0.smithi012.stdout:9/317: read d3/d9/f29 [1893194,125219] 0 2024-03-18T14:50:52.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/297: dwrite dd/d13/d23/d33/f51 [0,4194304] 0 2024-03-18T14:50:52.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/298: dread - dd/d13/d23/f55 zero size 2024-03-18T14:50:52.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/299: creat dd/d13/d23/d60/f6c x:0 0 0 2024-03-18T14:50:52.603 INFO:tasks.workunit.client.0.smithi012.stdout:2/287: dwrite d0/f55 [0,4194304] 0 2024-03-18T14:50:52.604 INFO:tasks.workunit.client.0.smithi012.stdout:0/303: mknod d1/d20/d4e/d54/d67/c71 0 2024-03-18T14:50:52.606 INFO:tasks.workunit.client.0.smithi012.stdout:9/318: creat d3/d9/d5c/f62 x:0 0 0 2024-03-18T14:50:52.606 INFO:tasks.workunit.client.0.smithi012.stdout:9/319: readlink d3/d1e/l37 0 2024-03-18T14:50:52.606 INFO:tasks.workunit.client.0.smithi012.stdout:9/320: write d3/d1e/d21/f59 [257456,57830] 0 2024-03-18T14:50:52.614 INFO:tasks.workunit.client.0.smithi012.stdout:7/300: dread dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:52.618 INFO:tasks.workunit.client.0.smithi012.stdout:2/288: unlink d0/d3c/d40/d47/f4e 0 2024-03-18T14:50:52.621 INFO:tasks.workunit.client.0.smithi012.stdout:3/204: rmdir dc 39 2024-03-18T14:50:52.621 INFO:tasks.workunit.client.0.smithi012.stdout:3/205: write dc/f11 [1405378,129898] 0 2024-03-18T14:50:52.626 INFO:tasks.workunit.client.0.smithi012.stdout:9/321: creat d3/f63 x:0 0 0 2024-03-18T14:50:52.627 INFO:tasks.workunit.client.0.smithi012.stdout:9/322: chown d3/f3e 1949310 1 2024-03-18T14:50:52.627 INFO:tasks.workunit.client.0.smithi012.stdout:9/323: truncate d3/d1e/f51 637047 0 2024-03-18T14:50:52.628 INFO:tasks.workunit.client.0.smithi012.stdout:2/289: link d0/d3c/l44 d0/d10/d18/d33/l56 0 2024-03-18T14:50:52.629 INFO:tasks.workunit.client.0.smithi012.stdout:7/301: dwrite dd/f5b [0,4194304] 0 2024-03-18T14:50:52.634 INFO:tasks.workunit.client.0.smithi012.stdout:4/313: truncate d6/f5c 771432 0 2024-03-18T14:50:52.634 INFO:tasks.workunit.client.0.smithi012.stdout:4/314: creat d6/d19/d39/f6d x:0 0 0 2024-03-18T14:50:52.635 INFO:tasks.workunit.client.0.smithi012.stdout:9/324: creat d3/f64 x:0 0 0 2024-03-18T14:50:52.638 INFO:tasks.workunit.client.0.smithi012.stdout:2/290: mkdir d0/d10/d18/d57 0 2024-03-18T14:50:52.638 INFO:tasks.workunit.client.0.smithi012.stdout:2/291: dread - d0/d3c/f41 zero size 2024-03-18T14:50:52.641 INFO:tasks.workunit.client.0.smithi012.stdout:7/302: read dd/d13/f15 [2479697,11963] 0 2024-03-18T14:50:52.641 INFO:tasks.workunit.client.0.smithi012.stdout:7/303: write dd/d4f/f69 [584218,35887] 0 2024-03-18T14:50:52.643 INFO:tasks.workunit.client.0.smithi012.stdout:1/280: fsync d8/db/f15 0 2024-03-18T14:50:52.644 INFO:tasks.workunit.client.0.smithi012.stdout:2/292: dread d0/d10/d18/d31/f35 [0,4194304] 0 2024-03-18T14:50:52.645 INFO:tasks.workunit.client.0.smithi012.stdout:3/206: mknod dc/c3e 0 2024-03-18T14:50:52.645 INFO:tasks.workunit.client.0.smithi012.stdout:3/207: stat dc/d30/d35/f2e 0 2024-03-18T14:50:52.650 INFO:tasks.workunit.client.0.smithi012.stdout:7/304: symlink dd/d13/d23/d2f/d53/l6d 0 2024-03-18T14:50:52.652 INFO:tasks.workunit.client.0.smithi012.stdout:1/281: truncate f4 1348297 0 2024-03-18T14:50:52.652 INFO:tasks.workunit.client.0.smithi012.stdout:9/325: dwrite d3/d9/d5c/f62 [0,4194304] 0 2024-03-18T14:50:52.652 INFO:tasks.workunit.client.0.smithi012.stdout:1/282: creat d8/d1e/d27/d44/f59 x:0 0 0 2024-03-18T14:50:52.654 INFO:tasks.workunit.client.0.smithi012.stdout:2/293: creat d0/d10/f58 x:0 0 0 2024-03-18T14:50:52.655 INFO:tasks.workunit.client.0.smithi012.stdout:7/305: dread dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:52.658 INFO:tasks.workunit.client.0.smithi012.stdout:3/208: rename dc/d30/d35/f2d to dc/d30/f3f 0 2024-03-18T14:50:52.660 INFO:tasks.workunit.client.0.smithi012.stdout:4/315: getdents d6 0 2024-03-18T14:50:52.660 INFO:tasks.workunit.client.0.smithi012.stdout:4/316: chown d6/f5c 6737871 1 2024-03-18T14:50:52.662 INFO:tasks.workunit.client.0.smithi012.stdout:2/294: read d0/f8 [872360,113571] 0 2024-03-18T14:50:52.668 INFO:tasks.workunit.client.0.smithi012.stdout:1/283: unlink d8/db/d10/f11 0 2024-03-18T14:50:52.669 INFO:tasks.workunit.client.0.smithi012.stdout:9/326: mkdir d3/d1e/d21/d4a/d52/d65 0 2024-03-18T14:50:52.672 INFO:tasks.workunit.client.0.smithi012.stdout:7/306: mknod dd/d13/d23/d60/c6e 0 2024-03-18T14:50:52.674 INFO:tasks.workunit.client.0.smithi012.stdout:9/327: dread d3/d9/f29 [0,4194304] 0 2024-03-18T14:50:52.677 INFO:tasks.workunit.client.0.smithi012.stdout:4/317: creat d6/d1a/d30/f6e x:0 0 0 2024-03-18T14:50:52.677 INFO:tasks.workunit.client.0.smithi012.stdout:4/318: getdents d6/d19/d6c 0 2024-03-18T14:50:52.677 INFO:tasks.workunit.client.0.smithi012.stdout:4/319: write d6/d19/d4f/f5e [1232573,128854] 0 2024-03-18T14:50:52.678 INFO:tasks.workunit.client.0.smithi012.stdout:2/295: mkdir d0/d10/d18/d31/d4c/d59 0 2024-03-18T14:50:52.678 INFO:tasks.workunit.client.0.smithi012.stdout:2/296: chown d0 230786904 1 2024-03-18T14:50:52.678 INFO:tasks.workunit.client.0.smithi012.stdout:2/297: stat d0/d10/d18/l26 0 2024-03-18T14:50:52.684 INFO:tasks.workunit.client.0.smithi012.stdout:3/209: symlink dc/l40 0 2024-03-18T14:50:52.685 INFO:tasks.workunit.client.0.smithi012.stdout:9/328: read d3/f6 [1568250,119928] 0 2024-03-18T14:50:52.685 INFO:tasks.workunit.client.0.smithi012.stdout:9/329: read d3/d1e/d21/f46 [665144,80069] 0 2024-03-18T14:50:52.686 INFO:tasks.workunit.client.0.smithi012.stdout:1/284: dwrite d8/db/d10/d31/d43/f51 [0,4194304] 0 2024-03-18T14:50:52.689 INFO:tasks.workunit.client.0.smithi012.stdout:4/320: chown d6/d1a/f31 48144 1 2024-03-18T14:50:52.691 INFO:tasks.workunit.client.0.smithi012.stdout:2/298: rename d0/d10/d18/d31/f35 to d0/d3c/d40/f5a 0 2024-03-18T14:50:52.691 INFO:tasks.workunit.client.0.smithi012.stdout:2/299: chown d0/d3c/d40/f4b 1956081406 1 2024-03-18T14:50:52.692 INFO:tasks.workunit.client.0.smithi012.stdout:2/300: stat d0/d10/d18/f3a 0 2024-03-18T14:50:52.692 INFO:tasks.workunit.client.0.smithi012.stdout:2/301: creat d0/d10/d18/f5b x:0 0 0 2024-03-18T14:50:52.692 INFO:tasks.workunit.client.0.smithi012.stdout:2/302: creat d0/d10/d18/d31/d4c/f5c x:0 0 0 2024-03-18T14:50:52.693 INFO:tasks.workunit.client.0.smithi012.stdout:1/285: write d8/db/d10/d16/d1c/f23 [904088,45574] 0 2024-03-18T14:50:52.694 INFO:tasks.workunit.client.0.smithi012.stdout:1/286: fsync d8/db/ff 0 2024-03-18T14:50:52.696 INFO:tasks.workunit.client.0.smithi012.stdout:9/330: truncate d3/d9/f1c 44913 0 2024-03-18T14:50:52.700 INFO:tasks.workunit.client.0.smithi012.stdout:4/321: mkdir d6/d1a/d30/d46/d6f 0 2024-03-18T14:50:52.701 INFO:tasks.workunit.client.0.smithi012.stdout:4/322: truncate d6/d19/d39/f6d 938122 0 2024-03-18T14:50:52.706 INFO:tasks.workunit.client.0.smithi012.stdout:2/303: symlink d0/d10/d18/d57/l5d 0 2024-03-18T14:50:52.710 INFO:tasks.workunit.client.0.smithi012.stdout:1/287: creat d8/f5a x:0 0 0 2024-03-18T14:50:52.710 INFO:tasks.workunit.client.0.smithi012.stdout:1/288: read - d8/d1e/d27/d44/f59 zero size 2024-03-18T14:50:52.719 INFO:tasks.workunit.client.0.smithi012.stdout:2/304: truncate d0/d10/d18/f1b 652614 0 2024-03-18T14:50:52.719 INFO:tasks.workunit.client.0.smithi012.stdout:2/305: creat d0/d3c/d40/f5e x:0 0 0 2024-03-18T14:50:52.724 INFO:tasks.workunit.client.0.smithi012.stdout:1/289: dwrite d8/d1e/d27/d44/f59 [0,4194304] 0 2024-03-18T14:50:52.727 INFO:tasks.workunit.client.0.smithi012.stdout:4/323: dread d6/d1a/f29 [0,4194304] 0 2024-03-18T14:50:52.731 INFO:tasks.workunit.client.0.smithi012.stdout:4/324: dread - d6/d1a/f62 zero size 2024-03-18T14:50:52.738 INFO:tasks.workunit.client.0.smithi012.stdout:9/331: dwrite d3/d20/f5f [0,4194304] 0 2024-03-18T14:50:52.738 INFO:tasks.workunit.client.0.smithi012.stdout:2/306: dwrite d0/d3c/d40/f4b [0,4194304] 0 2024-03-18T14:50:52.746 INFO:tasks.workunit.client.0.smithi012.stdout:0/304: getdents d1/d20/d4e 0 2024-03-18T14:50:52.746 INFO:tasks.workunit.client.0.smithi012.stdout:0/305: write d1/f59 [493785,1750] 0 2024-03-18T14:50:52.750 INFO:tasks.workunit.client.0.smithi012.stdout:9/332: getdents d3/d9/d19 0 2024-03-18T14:50:52.750 INFO:tasks.workunit.client.0.smithi012.stdout:9/333: truncate d3/d20/f43 9214944 0 2024-03-18T14:50:52.753 INFO:tasks.workunit.client.0.smithi012.stdout:2/307: dwrite d0/f8 [0,4194304] 0 2024-03-18T14:50:52.753 INFO:tasks.workunit.client.0.smithi012.stdout:2/308: write d0/d10/d18/d31/d4c/f5c [237749,72579] 0 2024-03-18T14:50:52.753 INFO:tasks.workunit.client.0.smithi012.stdout:2/309: getdents d0/d10/d18/d4f 0 2024-03-18T14:50:52.755 INFO:tasks.workunit.client.0.smithi012.stdout:0/306: dread d1/d16/d19/f47 [0,4194304] 0 2024-03-18T14:50:52.758 INFO:tasks.workunit.client.0.smithi012.stdout:2/310: fdatasync d0/d10/d18/d31/d4c/f5c 0 2024-03-18T14:50:52.776 INFO:tasks.workunit.client.0.smithi012.stdout:9/334: dwrite d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:52.776 INFO:tasks.workunit.client.0.smithi012.stdout:0/307: dwrite d1/f5f [0,4194304] 0 2024-03-18T14:50:52.795 INFO:tasks.workunit.client.0.smithi012.stdout:0/308: creat d1/d20/d4e/d54/d67/f72 x:0 0 0 2024-03-18T14:50:52.805 INFO:tasks.workunit.client.0.smithi012.stdout:0/309: symlink d1/d20/d23/d3b/d33/l73 0 2024-03-18T14:50:52.807 INFO:tasks.workunit.client.0.smithi012.stdout:0/310: creat d1/d20/d23/d3b/d33/d64/f74 x:0 0 0 2024-03-18T14:50:52.809 INFO:tasks.workunit.client.0.smithi012.stdout:0/311: creat d1/d20/d23/d6f/f75 x:0 0 0 2024-03-18T14:50:52.826 INFO:tasks.workunit.client.0.smithi012.stdout:0/312: dwrite d1/d16/d19/f68 [0,4194304] 0 2024-03-18T14:50:52.826 INFO:tasks.workunit.client.0.smithi012.stdout:0/313: stat d1/l2 0 2024-03-18T14:50:52.827 INFO:tasks.workunit.client.0.smithi012.stdout:0/314: fdatasync d1/d16/d19/f21 0 2024-03-18T14:50:52.830 INFO:tasks.workunit.client.0.smithi012.stdout:0/315: mknod d1/d16/d31/c76 0 2024-03-18T14:50:52.830 INFO:tasks.workunit.client.0.smithi012.stdout:0/316: chown d1/d20/d4e/d54 47996092 1 2024-03-18T14:50:52.831 INFO:tasks.workunit.client.0.smithi012.stdout:0/317: truncate d1/d20/f6a 189477 0 2024-03-18T14:50:52.831 INFO:tasks.workunit.client.0.smithi012.stdout:0/318: dread - d1/d20/d4e/d54/d67/f72 zero size 2024-03-18T14:50:52.846 INFO:tasks.workunit.client.0.smithi012.stdout:0/319: dwrite d1/d20/d23/d3b/f2c [0,4194304] 0 2024-03-18T14:50:52.849 INFO:tasks.workunit.client.0.smithi012.stdout:0/320: mkdir d1/d20/d23/d6f/d77 0 2024-03-18T14:50:52.857 INFO:tasks.workunit.client.0.smithi012.stdout:0/321: read d1/d16/d19/f68 [2128751,101285] 0 2024-03-18T14:50:52.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/322: dread d1/f10 [0,4194304] 0 2024-03-18T14:50:52.874 INFO:tasks.workunit.client.0.smithi012.stdout:3/210: getdents dc/d30/d35 0 2024-03-18T14:50:52.876 INFO:tasks.workunit.client.0.smithi012.stdout:3/211: chown dc/le 247577 1 2024-03-18T14:50:52.880 INFO:tasks.workunit.client.0.smithi012.stdout:1/290: getdents d8/d1e/d27/d44 0 2024-03-18T14:50:52.883 INFO:tasks.workunit.client.0.smithi012.stdout:3/212: mknod dc/d3c/c41 0 2024-03-18T14:50:52.900 INFO:tasks.workunit.client.0.smithi012.stdout:1/291: creat d8/db/d10/d16/f5b x:0 0 0 2024-03-18T14:50:52.902 INFO:tasks.workunit.client.0.smithi012.stdout:9/335: rename d3/d1e to d3/d9/d19/d66 0 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/213: rmdir dc/d30 39 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/214: chown dc/f2b 12849593 1 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/215: readlink dc/d30/l3b 0 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/216: fsync f6 0 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/217: read - dc/d30/d35/f2c zero size 2024-03-18T14:50:52.904 INFO:tasks.workunit.client.0.smithi012.stdout:3/218: readlink dc/d30/d35/l2a 0 2024-03-18T14:50:52.905 INFO:tasks.workunit.client.0.smithi012.stdout:1/292: creat d8/db/d10/d16/d54/f5c x:0 0 0 2024-03-18T14:50:52.906 INFO:tasks.workunit.client.0.smithi012.stdout:1/293: dread - d8/db/d10/d31/f4d zero size 2024-03-18T14:50:52.906 INFO:tasks.workunit.client.0.smithi012.stdout:1/294: write d8/d1e/d27/f48 [341262,4032] 0 2024-03-18T14:50:52.908 INFO:tasks.workunit.client.0.smithi012.stdout:9/336: link d3/f3e d3/d9/d19/d66/d21/d35/d48/f67 0 2024-03-18T14:50:52.910 INFO:tasks.workunit.client.0.smithi012.stdout:1/295: symlink d8/db/d10/d31/d45/l5d 0 2024-03-18T14:50:52.912 INFO:tasks.workunit.client.0.smithi012.stdout:9/337: symlink d3/d9/d19/d66/d21/d35/l68 0 2024-03-18T14:50:52.912 INFO:tasks.workunit.client.0.smithi012.stdout:9/338: fsync d3/f4 0 2024-03-18T14:50:52.912 INFO:tasks.workunit.client.0.smithi012.stdout:9/339: stat d3/d9/d19/d66/d21/d35/d48/d54/f58 0 2024-03-18T14:50:52.913 INFO:tasks.workunit.client.0.smithi012.stdout:1/296: rmdir d8/d1e 39 2024-03-18T14:50:52.914 INFO:tasks.workunit.client.0.smithi012.stdout:9/340: mkdir d3/d9/d19/d66/d69 0 2024-03-18T14:50:52.916 INFO:tasks.workunit.client.0.smithi012.stdout:1/297: mkdir d8/db/d10/d31/d43/d5e 0 2024-03-18T14:50:52.919 INFO:tasks.workunit.client.0.smithi012.stdout:1/298: creat d8/db/d10/d31/d43/d5e/f5f x:0 0 0 2024-03-18T14:50:52.922 INFO:tasks.workunit.client.0.smithi012.stdout:3/219: dwrite dc/f26 [0,4194304] 0 2024-03-18T14:50:52.937 INFO:tasks.workunit.client.0.smithi012.stdout:1/299: dwrite f5 [4194304,4194304] 0 2024-03-18T14:50:52.940 INFO:tasks.workunit.client.0.smithi012.stdout:1/300: creat d8/db/d10/f60 x:0 0 0 2024-03-18T14:50:52.946 INFO:tasks.workunit.client.0.smithi012.stdout:3/220: dwrite dc/f32 [0,4194304] 0 2024-03-18T14:50:52.950 INFO:tasks.workunit.client.0.smithi012.stdout:3/221: readlink dc/l1c 0 2024-03-18T14:50:52.952 INFO:tasks.workunit.client.0.smithi012.stdout:3/222: symlink dc/d30/d3a/l42 0 2024-03-18T14:50:52.967 INFO:tasks.workunit.client.0.smithi012.stdout:2/311: dwrite d0/d10/d18/f1b [0,4194304] 0 2024-03-18T14:50:52.968 INFO:tasks.workunit.client.0.smithi012.stdout:3/223: dwrite dc/f22 [0,4194304] 0 2024-03-18T14:50:52.969 INFO:tasks.workunit.client.0.smithi012.stdout:2/312: creat d0/d10/d18/d4f/f5f x:0 0 0 2024-03-18T14:50:52.969 INFO:tasks.workunit.client.0.smithi012.stdout:2/313: creat d0/d10/d18/d4f/f60 x:0 0 0 2024-03-18T14:50:52.972 INFO:tasks.workunit.client.0.smithi012.stdout:3/224: dread dc/f24 [0,4194304] 0 2024-03-18T14:50:52.972 INFO:tasks.workunit.client.0.smithi012.stdout:3/225: write dc/f24 [778733,50602] 0 2024-03-18T14:50:52.983 INFO:tasks.workunit.client.0.smithi012.stdout:3/226: creat dc/f43 x:0 0 0 2024-03-18T14:50:52.983 INFO:tasks.workunit.client.0.smithi012.stdout:3/227: stat dc/f32 0 2024-03-18T14:50:52.991 INFO:tasks.workunit.client.0.smithi012.stdout:3/228: rename f2 to dc/f44 0 2024-03-18T14:50:52.991 INFO:tasks.workunit.client.0.smithi012.stdout:3/229: truncate dc/f44 1192907 0 2024-03-18T14:50:52.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/230: rename dc/d30/d35/l2a to dc/d3c/l45 0 2024-03-18T14:50:52.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/231: dread - dc/d30/f37 zero size 2024-03-18T14:50:52.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/232: dread - dc/f2b zero size 2024-03-18T14:50:52.995 INFO:tasks.workunit.client.0.smithi012.stdout:3/233: chown dc/l40 8480738 1 2024-03-18T14:50:52.999 INFO:tasks.workunit.client.0.smithi012.stdout:3/234: link dc/c19 dc/d30/c46 0 2024-03-18T14:50:52.999 INFO:tasks.workunit.client.0.smithi012.stdout:2/314: dwrite d0/d10/d18/f27 [0,4194304] 0 2024-03-18T14:50:53.000 INFO:tasks.workunit.client.0.smithi012.stdout:0/323: dwrite d1/f4f [0,4194304] 0 2024-03-18T14:50:53.000 INFO:tasks.workunit.client.0.smithi012.stdout:2/315: dread - d0/d10/f58 zero size 2024-03-18T14:50:53.001 INFO:tasks.workunit.client.0.smithi012.stdout:3/235: creat dc/d30/d35/f47 x:0 0 0 2024-03-18T14:50:53.006 INFO:tasks.workunit.client.0.smithi012.stdout:0/324: symlink d1/d20/d23/d3b/d33/l78 0 2024-03-18T14:50:53.008 INFO:tasks.workunit.client.0.smithi012.stdout:2/316: mkdir d0/d10/d18/d57/d61 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:3/236: truncate dc/d30/d35/f2e 3986828 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:3/237: write dc/f1f [3921460,27602] 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:3/238: fdatasync dc/d30/d35/f2e 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:9/341: truncate d3/d20/f31 191192 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:9/342: fsync d3/d9/f13 0 2024-03-18T14:50:53.010 INFO:tasks.workunit.client.0.smithi012.stdout:9/343: fsync d3/d9/fd 0 2024-03-18T14:50:53.017 INFO:tasks.workunit.client.0.smithi012.stdout:2/317: creat d0/d10/d18/d33/f62 x:0 0 0 2024-03-18T14:50:53.023 INFO:tasks.workunit.client.0.smithi012.stdout:0/325: dwrite d1/d16/d31/f50 [0,4194304] 0 2024-03-18T14:50:53.023 INFO:tasks.workunit.client.0.smithi012.stdout:0/326: dread - d1/d2d/f62 zero size 2024-03-18T14:50:53.026 INFO:tasks.workunit.client.0.smithi012.stdout:9/344: creat d3/d9/d19/f6a x:0 0 0 2024-03-18T14:50:53.027 INFO:tasks.workunit.client.0.smithi012.stdout:9/345: read d3/f4 [1281221,108829] 0 2024-03-18T14:50:53.027 INFO:tasks.workunit.client.0.smithi012.stdout:2/318: rmdir d0/d10/d18/d57 39 2024-03-18T14:50:53.031 INFO:tasks.workunit.client.0.smithi012.stdout:0/327: read d1/d20/d23/d26/f53 [2732548,104222] 0 2024-03-18T14:50:53.034 INFO:tasks.workunit.client.0.smithi012.stdout:9/346: mknod d3/d9/d19/d66/c6b 0 2024-03-18T14:50:53.036 INFO:tasks.workunit.client.0.smithi012.stdout:2/319: symlink d0/d10/d18/d57/d61/l63 0 2024-03-18T14:50:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:9/347: creat d3/d9/f6c x:0 0 0 2024-03-18T14:50:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:2/320: unlink d0/cb 0 2024-03-18T14:50:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:9/348: rename d3/d9/d19/d66/d21/d35/d48/d54/f58 to d3/d9/d19/d66/d21/d4a/d52/f6d 0 2024-03-18T14:50:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:9/349: fsync d3/d9/d19/f3a 0 2024-03-18T14:50:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:2/321: rename d0/d10/f2a to d0/d10/d18/d57/f64 0 2024-03-18T14:50:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:9/350: read d3/d9/d19/d66/f44 [2752713,84420] 0 2024-03-18T14:50:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:2/322: write d0/fa [6254689,96015] 0 2024-03-18T14:50:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:9/351: mkdir d3/d9/d19/d66/d6e 0 2024-03-18T14:50:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:9/352: readlink d3/d9/d19/d66/d21/d35/l38 0 2024-03-18T14:50:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:9/353: stat d3/d9 0 2024-03-18T14:50:53.059 INFO:tasks.workunit.client.0.smithi012.stdout:9/354: rmdir d3/d9/d19/d66/d21/d4a 39 2024-03-18T14:50:53.059 INFO:tasks.workunit.client.0.smithi012.stdout:9/355: stat d3/f64 0 2024-03-18T14:50:53.059 INFO:tasks.workunit.client.0.smithi012.stdout:9/356: dread - d3/d9/f6c zero size 2024-03-18T14:50:53.062 INFO:tasks.workunit.client.0.smithi012.stdout:2/323: write d0/d3c/d40/f4b [3253090,123978] 0 2024-03-18T14:50:53.062 INFO:tasks.workunit.client.0.smithi012.stdout:2/324: fdatasync d0/f5 0 2024-03-18T14:50:53.086 INFO:tasks.workunit.client.0.smithi012.stdout:9/357: dwrite d3/d9/f10 [0,4194304] 0 2024-03-18T14:50:53.086 INFO:tasks.workunit.client.0.smithi012.stdout:2/325: dwrite d0/f5 [0,4194304] 0 2024-03-18T14:50:53.091 INFO:tasks.workunit.client.0.smithi012.stdout:2/326: mknod d0/d3c/d40/d47/c65 0 2024-03-18T14:50:53.091 INFO:tasks.workunit.client.0.smithi012.stdout:2/327: write d0/d10/d18/d4f/f60 [468571,91113] 0 2024-03-18T14:50:53.093 INFO:tasks.workunit.client.0.smithi012.stdout:9/358: symlink d3/d9/l6f 0 2024-03-18T14:50:53.093 INFO:tasks.workunit.client.0.smithi012.stdout:9/359: dread - d3/d9/d19/d66/d21/d4a/d52/f60 zero size 2024-03-18T14:50:53.096 INFO:tasks.workunit.client.0.smithi012.stdout:9/360: fsync d3/d20/f47 0 2024-03-18T14:50:53.102 INFO:tasks.workunit.client.0.smithi012.stdout:9/361: symlink d3/d9/d19/d66/l70 0 2024-03-18T14:50:53.102 INFO:tasks.workunit.client.0.smithi012.stdout:9/362: chown d3/d9/c4c 268388 1 2024-03-18T14:50:53.102 INFO:tasks.workunit.client.0.smithi012.stdout:2/328: write d0/d10/d18/f1b [1521112,125070] 0 2024-03-18T14:50:53.102 INFO:tasks.workunit.client.0.smithi012.stdout:9/363: mkdir d3/d9/d19/d66/d21/d35/d71 0 2024-03-18T14:50:53.105 INFO:tasks.workunit.client.0.smithi012.stdout:2/329: symlink d0/d10/d18/d33/l66 0 2024-03-18T14:50:53.115 INFO:tasks.workunit.client.0.smithi012.stdout:9/364: creat d3/d9/d19/d66/d21/d35/d48/d54/f72 x:0 0 0 2024-03-18T14:50:53.117 INFO:tasks.workunit.client.0.smithi012.stdout:6/273: sync 2024-03-18T14:50:53.117 INFO:tasks.workunit.client.0.smithi012.stdout:5/318: sync 2024-03-18T14:50:53.117 INFO:tasks.workunit.client.0.smithi012.stdout:6/274: fsync ff 0 2024-03-18T14:50:53.117 INFO:tasks.workunit.client.0.smithi012.stdout:5/319: read d3/d15/f31 [1137634,123190] 0 2024-03-18T14:50:53.118 INFO:tasks.workunit.client.0.smithi012.stdout:9/365: symlink d3/d9/d19/d66/d6e/l73 0 2024-03-18T14:50:53.118 INFO:tasks.workunit.client.0.smithi012.stdout:9/366: write d3/d9/d19/f6a [719005,45805] 0 2024-03-18T14:50:53.120 INFO:tasks.workunit.client.0.smithi012.stdout:6/275: getdents d17/d26 0 2024-03-18T14:50:53.120 INFO:tasks.workunit.client.0.smithi012.stdout:6/276: fsync d17/d26/f4a 0 2024-03-18T14:50:53.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/330: dwrite d0/d10/f32 [4194304,4194304] 0 2024-03-18T14:50:53.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/331: fdatasync d0/d3c/d40/f4b 0 2024-03-18T14:50:53.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/332: fdatasync d0/f21 0 2024-03-18T14:50:53.123 INFO:tasks.workunit.client.0.smithi012.stdout:9/367: unlink d3/d9/d19/d66/d21/f59 0 2024-03-18T14:50:53.131 INFO:tasks.workunit.client.0.smithi012.stdout:2/333: link d0/d10/d18/d33/f62 d0/d10/d18/d31/d4c/f67 0 2024-03-18T14:50:53.132 INFO:tasks.workunit.client.0.smithi012.stdout:9/368: mkdir d3/d9/d19/d66/d21/d4a/d52/d65/d74 0 2024-03-18T14:50:53.135 INFO:tasks.workunit.client.0.smithi012.stdout:9/369: rename d3/d9/l6f to d3/d9/d19/d66/d21/d35/d48/d54/l75 0 2024-03-18T14:50:53.137 INFO:tasks.workunit.client.0.smithi012.stdout:5/320: dwrite d3/d15/f1d [4194304,4194304] 0 2024-03-18T14:50:53.137 INFO:tasks.workunit.client.0.smithi012.stdout:5/321: write d3/d15/f1c [744594,11956] 0 2024-03-18T14:50:53.138 INFO:tasks.workunit.client.0.smithi012.stdout:9/370: truncate d3/d9/f29 2183761 0 2024-03-18T14:50:53.141 INFO:tasks.workunit.client.0.smithi012.stdout:2/334: write d0/f8 [1238626,46110] 0 2024-03-18T14:50:53.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/277: dwrite f8 [4194304,4194304] 0 2024-03-18T14:50:53.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/278: write d17/f1b [821802,44516] 0 2024-03-18T14:50:53.142 INFO:tasks.workunit.client.0.smithi012.stdout:6/279: chown d17/d26/c31 0 1 2024-03-18T14:50:53.143 INFO:tasks.workunit.client.0.smithi012.stdout:2/335: fsync d0/d10/d18/f1b 0 2024-03-18T14:50:53.143 INFO:tasks.workunit.client.0.smithi012.stdout:5/322: unlink d3/d29/d35/l3c 0 2024-03-18T14:50:53.143 INFO:tasks.workunit.client.0.smithi012.stdout:5/323: chown d3/d15/d21/l2a 62716 1 2024-03-18T14:50:53.145 INFO:tasks.workunit.client.0.smithi012.stdout:9/371: truncate d3/d9/f17 198064 0 2024-03-18T14:50:53.157 INFO:tasks.workunit.client.0.smithi012.stdout:6/280: dwrite d17/f2c [0,4194304] 0 2024-03-18T14:50:53.157 INFO:tasks.workunit.client.0.smithi012.stdout:6/281: chown d17/d19/c3d 970399 1 2024-03-18T14:50:53.163 INFO:tasks.workunit.client.0.smithi012.stdout:5/324: unlink d3/d15/d42/f4f 0 2024-03-18T14:50:53.165 INFO:tasks.workunit.client.0.smithi012.stdout:2/336: truncate d0/d10/f16 680385 0 2024-03-18T14:50:53.165 INFO:tasks.workunit.client.0.smithi012.stdout:6/282: dread d17/d19/f20 [0,4194304] 0 2024-03-18T14:50:53.170 INFO:tasks.workunit.client.0.smithi012.stdout:5/325: rename d3/d15/f31 to d3/d29/d36/d3d/d65/f6c 0 2024-03-18T14:50:53.170 INFO:tasks.workunit.client.0.smithi012.stdout:5/326: readlink d3/d1e/l60 0 2024-03-18T14:50:53.172 INFO:tasks.workunit.client.0.smithi012.stdout:2/337: mknod d0/d3c/c68 0 2024-03-18T14:50:53.174 INFO:tasks.workunit.client.0.smithi012.stdout:6/283: link d17/d26/c31 d17/d33/c4c 0 2024-03-18T14:50:53.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/327: truncate d3/d29/d35/f48 5464944 0 2024-03-18T14:50:53.179 INFO:tasks.workunit.client.0.smithi012.stdout:2/338: rename d0/f5 to d0/d10/d18/d31/d4c/d59/f69 0 2024-03-18T14:50:53.181 INFO:tasks.workunit.client.0.smithi012.stdout:6/284: mkdir d17/d26/d36/d4d 0 2024-03-18T14:50:53.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/328: creat d3/d29/d36/f6d x:0 0 0 2024-03-18T14:50:53.190 INFO:tasks.workunit.client.0.smithi012.stdout:5/329: rmdir d3/d29/d35/d5b 39 2024-03-18T14:50:53.190 INFO:tasks.workunit.client.0.smithi012.stdout:5/330: creat d3/d29/d36/d3d/f6e x:0 0 0 2024-03-18T14:50:53.190 INFO:tasks.workunit.client.0.smithi012.stdout:5/331: write d3/d29/d36/d3d/f4a [554127,109451] 0 2024-03-18T14:50:53.194 INFO:tasks.workunit.client.0.smithi012.stdout:5/332: creat d3/d1e/f6f x:0 0 0 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/285: dwrite f14 [4194304,4194304] 0 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/286: chown d17/d33/c4c 1630173 1 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/287: fsync d17/f35 0 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/288: readlink d17/d26/d36/l40 0 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/289: readlink d17/d38/d3f/l47 0 2024-03-18T14:50:53.197 INFO:tasks.workunit.client.0.smithi012.stdout:6/290: chown d17/d19/d27/d2f 53404837 1 2024-03-18T14:50:53.207 INFO:tasks.workunit.client.0.smithi012.stdout:3/239: link dc/f44 dc/f48 0 2024-03-18T14:50:53.208 INFO:tasks.workunit.client.0.smithi012.stdout:6/291: dread f16 [0,4194304] 0 2024-03-18T14:50:53.212 INFO:tasks.workunit.client.0.smithi012.stdout:3/240: unlink dc/cd 0 2024-03-18T14:50:53.213 INFO:tasks.workunit.client.0.smithi012.stdout:6/292: write f16 [632439,92579] 0 2024-03-18T14:50:53.213 INFO:tasks.workunit.client.0.smithi012.stdout:5/333: dwrite d3/d29/d35/f5a [0,4194304] 0 2024-03-18T14:50:53.214 INFO:tasks.workunit.client.0.smithi012.stdout:3/241: dread dc/ff [0,4194304] 0 2024-03-18T14:50:53.220 INFO:tasks.workunit.client.0.smithi012.stdout:5/334: creat d3/d15/d5f/f70 x:0 0 0 2024-03-18T14:50:53.220 INFO:tasks.workunit.client.0.smithi012.stdout:6/293: read d17/f22 [3255504,50038] 0 2024-03-18T14:50:53.220 INFO:tasks.workunit.client.0.smithi012.stdout:5/335: write d3/f22 [512101,81280] 0 2024-03-18T14:50:53.223 INFO:tasks.workunit.client.0.smithi012.stdout:5/336: link d3/d1e/d33/f34 d3/d15/d21/f71 0 2024-03-18T14:50:53.224 INFO:tasks.workunit.client.0.smithi012.stdout:5/337: read d3/d29/d36/d3d/f47 [759462,114208] 0 2024-03-18T14:50:53.228 INFO:tasks.workunit.client.0.smithi012.stdout:8/407: sync 2024-03-18T14:50:53.228 INFO:tasks.workunit.client.0.smithi012.stdout:7/307: sync 2024-03-18T14:50:53.229 INFO:tasks.workunit.client.0.smithi012.stdout:6/294: dread f14 [4194304,4194304] 0 2024-03-18T14:50:53.229 INFO:tasks.workunit.client.0.smithi012.stdout:6/295: creat d17/d33/f4e x:0 0 0 2024-03-18T14:50:53.229 INFO:tasks.workunit.client.0.smithi012.stdout:6/296: write d17/d19/f2b [2551297,69342] 0 2024-03-18T14:50:53.230 INFO:tasks.workunit.client.0.smithi012.stdout:7/308: dread dd/d13/d23/d2f/f32 [0,4194304] 0 2024-03-18T14:50:53.230 INFO:tasks.workunit.client.0.smithi012.stdout:8/408: dread d0/d5a/d19/d57/f68 [0,4194304] 0 2024-03-18T14:50:53.233 INFO:tasks.workunit.client.0.smithi012.stdout:6/297: read d17/d19/f20 [3854382,84597] 0 2024-03-18T14:50:53.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/309: mkdir dd/d13/d23/d2a/d6f 0 2024-03-18T14:50:53.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/310: write dd/d13/d23/d60/f62 [342377,103852] 0 2024-03-18T14:50:53.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/311: fsync dd/d13/d23/d33/d48/f5c 0 2024-03-18T14:50:53.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/312: readlink dd/d13/d23/d2a/l35 0 2024-03-18T14:50:53.241 INFO:tasks.workunit.client.0.smithi012.stdout:8/409: mkdir d0/d15/d78/d80 0 2024-03-18T14:50:53.243 INFO:tasks.workunit.client.0.smithi012.stdout:6/298: mknod d17/d19/d27/c4f 0 2024-03-18T14:50:53.244 INFO:tasks.workunit.client.0.smithi012.stdout:8/410: dread d0/d5a/d55/f37 [0,4194304] 0 2024-03-18T14:50:53.246 INFO:tasks.workunit.client.0.smithi012.stdout:6/299: mknod d17/d38/d3f/c50 0 2024-03-18T14:50:53.247 INFO:tasks.workunit.client.0.smithi012.stdout:7/313: dread dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:53.249 INFO:tasks.workunit.client.0.smithi012.stdout:8/411: unlink d0/d5a/d19/d1b/d2a/d3e/f4a 0 2024-03-18T14:50:53.250 INFO:tasks.workunit.client.0.smithi012.stdout:6/300: dread d17/f22 [0,4194304] 0 2024-03-18T14:50:53.251 INFO:tasks.workunit.client.0.smithi012.stdout:8/412: rmdir d0/d5a/d55 39 2024-03-18T14:50:53.253 INFO:tasks.workunit.client.0.smithi012.stdout:6/301: truncate d17/d18/f23 400154 0 2024-03-18T14:50:53.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/314: dread dd/f5b [0,4194304] 0 2024-03-18T14:50:53.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/315: dread - dd/f59 zero size 2024-03-18T14:50:53.257 INFO:tasks.workunit.client.0.smithi012.stdout:6/302: mkdir d17/d38/d51 0 2024-03-18T14:50:53.257 INFO:tasks.workunit.client.0.smithi012.stdout:6/303: fdatasync d17/d33/f4e 0 2024-03-18T14:50:53.260 INFO:tasks.workunit.client.0.smithi012.stdout:7/316: rename dd/d13/l17 to dd/d13/d23/d60/l70 0 2024-03-18T14:50:53.263 INFO:tasks.workunit.client.0.smithi012.stdout:6/304: unlink f5 0 2024-03-18T14:50:53.263 INFO:tasks.workunit.client.0.smithi012.stdout:6/305: write d17/f44 [998314,92161] 0 2024-03-18T14:50:53.264 INFO:tasks.workunit.client.0.smithi012.stdout:7/317: truncate dd/d13/d23/f27 2426555 0 2024-03-18T14:50:53.264 INFO:tasks.workunit.client.0.smithi012.stdout:7/318: stat fa 0 2024-03-18T14:50:53.265 INFO:tasks.workunit.client.0.smithi012.stdout:7/319: fdatasync dd/f10 0 2024-03-18T14:50:53.265 INFO:tasks.workunit.client.0.smithi012.stdout:7/320: readlink dd/d13/d23/l3a 0 2024-03-18T14:50:53.265 INFO:tasks.workunit.client.0.smithi012.stdout:7/321: dread - dd/d13/f57 zero size 2024-03-18T14:50:53.267 INFO:tasks.workunit.client.0.smithi012.stdout:6/306: mknod d17/d38/c52 0 2024-03-18T14:50:53.268 INFO:tasks.workunit.client.0.smithi012.stdout:8/413: dwrite d0/d5a/d19/f3a [0,4194304] 0 2024-03-18T14:50:53.270 INFO:tasks.workunit.client.0.smithi012.stdout:7/322: mknod dd/d4f/c71 0 2024-03-18T14:50:53.271 INFO:tasks.workunit.client.0.smithi012.stdout:6/307: dread fb [0,4194304] 0 2024-03-18T14:50:53.272 INFO:tasks.workunit.client.0.smithi012.stdout:6/308: dread - d17/d26/d36/f4b zero size 2024-03-18T14:50:53.282 INFO:tasks.workunit.client.0.smithi012.stdout:8/414: mknod d0/d5a/d55/c81 0 2024-03-18T14:50:53.282 INFO:tasks.workunit.client.0.smithi012.stdout:8/415: truncate d0/d5a/f6d 935842 0 2024-03-18T14:50:53.282 INFO:tasks.workunit.client.0.smithi012.stdout:8/416: dread - d0/d5a/d19/d1b/f6b zero size 2024-03-18T14:50:53.282 INFO:tasks.workunit.client.0.smithi012.stdout:8/417: fsync d0/f6 0 2024-03-18T14:50:53.284 INFO:tasks.workunit.client.0.smithi012.stdout:7/323: unlink dd/f31 0 2024-03-18T14:50:53.285 INFO:tasks.workunit.client.0.smithi012.stdout:6/309: creat d17/d38/d51/f53 x:0 0 0 2024-03-18T14:50:53.287 INFO:tasks.workunit.client.0.smithi012.stdout:8/418: creat d0/d5a/d5e/f82 x:0 0 0 2024-03-18T14:50:53.289 INFO:tasks.workunit.client.0.smithi012.stdout:8/419: dread d0/d5a/d19/d57/f68 [0,4194304] 0 2024-03-18T14:50:53.290 INFO:tasks.workunit.client.0.smithi012.stdout:6/310: creat d17/d19/d27/f54 x:0 0 0 2024-03-18T14:50:53.290 INFO:tasks.workunit.client.0.smithi012.stdout:6/311: readlink d17/d19/d27/d2f/l32 0 2024-03-18T14:50:53.293 INFO:tasks.workunit.client.0.smithi012.stdout:8/420: creat d0/d47/d59/f83 x:0 0 0 2024-03-18T14:50:53.293 INFO:tasks.workunit.client.0.smithi012.stdout:8/421: write d0/d15/f1a [961095,73775] 0 2024-03-18T14:50:53.293 INFO:tasks.workunit.client.0.smithi012.stdout:8/422: readlink d0/d5a/d19/l56 0 2024-03-18T14:50:53.293 INFO:tasks.workunit.client.0.smithi012.stdout:8/423: write d0/d12/f71 [259412,67567] 0 2024-03-18T14:50:53.294 INFO:tasks.workunit.client.0.smithi012.stdout:6/312: creat d17/d38/d3f/f55 x:0 0 0 2024-03-18T14:50:53.300 INFO:tasks.workunit.client.0.smithi012.stdout:7/324: dwrite dd/d13/d23/d2f/d53/f5d [0,4194304] 0 2024-03-18T14:50:53.367 INFO:tasks.workunit.client.0.smithi012.stdout:2/339: fsync d0/d10/d18/d31/d4c/d59/f69 0 2024-03-18T14:50:53.370 INFO:tasks.workunit.client.0.smithi012.stdout:2/340: creat d0/d10/f6a x:0 0 0 2024-03-18T14:50:53.370 INFO:tasks.workunit.client.0.smithi012.stdout:2/341: dread - d0/d10/d18/d31/d4c/f51 zero size 2024-03-18T14:50:53.370 INFO:tasks.workunit.client.0.smithi012.stdout:2/342: fdatasync d0/d3c/f46 0 2024-03-18T14:50:53.371 INFO:tasks.workunit.client.0.smithi012.stdout:2/343: stat d0/d10/l4d 0 2024-03-18T14:50:53.372 INFO:tasks.workunit.client.0.smithi012.stdout:2/344: write d0/d10/d18/f24 [1770002,83889] 0 2024-03-18T14:50:53.374 INFO:tasks.workunit.client.0.smithi012.stdout:2/345: mkdir d0/d6b 0 2024-03-18T14:50:53.374 INFO:tasks.workunit.client.0.smithi012.stdout:2/346: stat d0/d10/d18/d57/f64 0 2024-03-18T14:50:53.376 INFO:tasks.workunit.client.0.smithi012.stdout:2/347: truncate d0/d10/d18/f3a 1572658 0 2024-03-18T14:50:53.381 INFO:tasks.workunit.client.0.smithi012.stdout:2/348: mkdir d0/d6c 0 2024-03-18T14:50:53.381 INFO:tasks.workunit.client.0.smithi012.stdout:2/349: chown d0/d10/d18/d33/l56 0 1 2024-03-18T14:50:53.381 INFO:tasks.workunit.client.0.smithi012.stdout:2/350: creat d0/d10/d18/d57/d61/f6d x:0 0 0 2024-03-18T14:50:53.382 INFO:tasks.workunit.client.0.smithi012.stdout:2/351: truncate d0/d3c/d40/f5e 486497 0 2024-03-18T14:50:53.382 INFO:tasks.workunit.client.0.smithi012.stdout:2/352: write d0/fa [645564,43432] 0 2024-03-18T14:50:53.382 INFO:tasks.workunit.client.0.smithi012.stdout:2/353: truncate d0/d3c/f46 377799 0 2024-03-18T14:50:53.390 INFO:tasks.workunit.client.0.smithi012.stdout:4/325: sync 2024-03-18T14:50:53.395 INFO:tasks.workunit.client.0.smithi012.stdout:3/242: rmdir dc 39 2024-03-18T14:50:53.397 INFO:tasks.workunit.client.0.smithi012.stdout:4/326: dread d6/f25 [0,4194304] 0 2024-03-18T14:50:53.399 INFO:tasks.workunit.client.0.smithi012.stdout:3/243: truncate dc/f44 702270 0 2024-03-18T14:50:53.401 INFO:tasks.workunit.client.0.smithi012.stdout:4/327: write d6/d1a/f29 [38868,78644] 0 2024-03-18T14:50:53.407 INFO:tasks.workunit.client.0.smithi012.stdout:3/244: write dc/f43 [186992,78362] 0 2024-03-18T14:50:53.419 INFO:tasks.workunit.client.0.smithi012.stdout:4/328: dwrite d6/d1a/f29 [4194304,4194304] 0 2024-03-18T14:50:53.419 INFO:tasks.workunit.client.0.smithi012.stdout:4/329: creat d6/d27/f70 x:0 0 0 2024-03-18T14:50:53.423 INFO:tasks.workunit.client.0.smithi012.stdout:7/325: getdents dd/d13/d23/d2a 0 2024-03-18T14:50:53.423 INFO:tasks.workunit.client.0.smithi012.stdout:7/326: chown dd/l1b 1747 1 2024-03-18T14:50:53.426 INFO:tasks.workunit.client.0.smithi012.stdout:4/330: creat d6/d1a/d30/d46/d6f/f71 x:0 0 0 2024-03-18T14:50:53.429 INFO:tasks.workunit.client.0.smithi012.stdout:7/327: creat dd/d13/f72 x:0 0 0 2024-03-18T14:50:53.429 INFO:tasks.workunit.client.0.smithi012.stdout:7/328: write dd/d13/d23/d2f/f4c [208518,50879] 0 2024-03-18T14:50:53.429 INFO:tasks.workunit.client.0.smithi012.stdout:4/331: dread d6/f5c [0,4194304] 0 2024-03-18T14:50:53.430 INFO:tasks.workunit.client.0.smithi012.stdout:7/329: dread dd/d13/d23/d2f/f32 [0,4194304] 0 2024-03-18T14:50:53.432 INFO:tasks.workunit.client.0.smithi012.stdout:4/332: mknod d6/d19/d4f/c72 0 2024-03-18T14:50:53.432 INFO:tasks.workunit.client.0.smithi012.stdout:4/333: chown d6/d1a/f2a 48 1 2024-03-18T14:50:53.436 INFO:tasks.workunit.client.0.smithi012.stdout:4/334: truncate d6/f28 1686004 0 2024-03-18T14:50:53.439 INFO:tasks.workunit.client.0.smithi012.stdout:4/335: mknod d6/d27/c73 0 2024-03-18T14:50:53.439 INFO:tasks.workunit.client.0.smithi012.stdout:4/336: chown d6/d1a/d30/d46 1 1 2024-03-18T14:50:53.439 INFO:tasks.workunit.client.0.smithi012.stdout:4/337: write d6/d1a/d30/d46/d6f/f71 [324056,109310] 0 2024-03-18T14:50:53.439 INFO:tasks.workunit.client.0.smithi012.stdout:4/338: dread - d6/d1a/d30/f47 zero size 2024-03-18T14:50:53.440 INFO:tasks.workunit.client.0.smithi012.stdout:8/424: rename d0/d5a/d19/d1b to d0/d15/d6e/d84 0 2024-03-18T14:50:53.443 INFO:tasks.workunit.client.0.smithi012.stdout:4/339: mknod d6/c74 0 2024-03-18T14:50:53.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/313: rename d17/c2e to d17/d38/d51/c56 0 2024-03-18T14:50:53.448 INFO:tasks.workunit.client.0.smithi012.stdout:4/340: symlink d6/d1a/d30/l75 0 2024-03-18T14:50:53.452 INFO:tasks.workunit.client.0.smithi012.stdout:7/330: rename dd/d13/d23/d33/d37/f52 to dd/d13/f73 0 2024-03-18T14:50:53.452 INFO:tasks.workunit.client.0.smithi012.stdout:7/331: write dd/f10 [2656181,72265] 0 2024-03-18T14:50:53.455 INFO:tasks.workunit.client.0.smithi012.stdout:6/314: unlink d17/d38/d3f/l47 0 2024-03-18T14:50:53.460 INFO:tasks.workunit.client.0.smithi012.stdout:8/425: rename d0/d15/f1a to d0/d15/d6e/d84/d54/f85 0 2024-03-18T14:50:53.461 INFO:tasks.workunit.client.0.smithi012.stdout:8/426: truncate d0/f10 1897997 0 2024-03-18T14:50:53.463 INFO:tasks.workunit.client.0.smithi012.stdout:7/332: link dd/d13/d23/d60/l70 dd/d4f/l74 0 2024-03-18T14:50:53.466 INFO:tasks.workunit.client.0.smithi012.stdout:6/315: write f14 [1322593,103557] 0 2024-03-18T14:50:53.467 INFO:tasks.workunit.client.0.smithi012.stdout:4/341: dwrite d6/f33 [0,4194304] 0 2024-03-18T14:50:53.469 INFO:tasks.workunit.client.0.smithi012.stdout:7/333: link dd/d13/f72 dd/d13/f75 0 2024-03-18T14:50:53.469 INFO:tasks.workunit.client.0.smithi012.stdout:7/334: read dd/d13/d23/d33/d48/f5c [3068750,112650] 0 2024-03-18T14:50:53.471 INFO:tasks.workunit.client.0.smithi012.stdout:0/328: sync 2024-03-18T14:50:53.471 INFO:tasks.workunit.client.0.smithi012.stdout:1/301: sync 2024-03-18T14:50:53.472 INFO:tasks.workunit.client.0.smithi012.stdout:1/302: chown d8/db/d10/c3a 15315 1 2024-03-18T14:50:53.472 INFO:tasks.workunit.client.0.smithi012.stdout:1/303: chown d8/db/l20 10 1 2024-03-18T14:50:53.472 INFO:tasks.workunit.client.0.smithi012.stdout:0/329: truncate d1/d20/d23/d6f/f75 374300 0 2024-03-18T14:50:53.472 INFO:tasks.workunit.client.0.smithi012.stdout:0/330: chown d1/d16/d18/f30 86 1 2024-03-18T14:50:53.476 INFO:tasks.workunit.client.0.smithi012.stdout:8/427: dwrite d0/d15/d6e/d84/f77 [0,4194304] 0 2024-03-18T14:50:53.477 INFO:tasks.workunit.client.0.smithi012.stdout:1/304: mkdir d8/d1e/d61 0 2024-03-18T14:50:53.480 INFO:tasks.workunit.client.0.smithi012.stdout:6/316: mknod d17/d38/d51/c57 0 2024-03-18T14:50:53.480 INFO:tasks.workunit.client.0.smithi012.stdout:6/317: chown d17/d26/d36/l40 1669814 1 2024-03-18T14:50:53.480 INFO:tasks.workunit.client.0.smithi012.stdout:6/318: chown d17/d33/f48 22718 1 2024-03-18T14:50:53.486 INFO:tasks.workunit.client.0.smithi012.stdout:0/331: write d1/d16/d19/f47 [1831289,29478] 0 2024-03-18T14:50:53.486 INFO:tasks.workunit.client.0.smithi012.stdout:0/332: read d1/d20/d23/d6f/f75 [44713,53482] 0 2024-03-18T14:50:53.493 INFO:tasks.workunit.client.0.smithi012.stdout:8/428: mkdir d0/d47/d59/d86 0 2024-03-18T14:50:53.493 INFO:tasks.workunit.client.0.smithi012.stdout:8/429: chown d0/d15/d6e/f7b 0 1 2024-03-18T14:50:53.493 INFO:tasks.workunit.client.0.smithi012.stdout:8/430: chown d0/d15/d6e/d84/d54/f85 1071066609 1 2024-03-18T14:50:53.494 INFO:tasks.workunit.client.0.smithi012.stdout:4/342: dwrite d6/d1a/d30/f47 [0,4194304] 0 2024-03-18T14:50:53.495 INFO:tasks.workunit.client.0.smithi012.stdout:7/335: dwrite dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:50:53.495 INFO:tasks.workunit.client.0.smithi012.stdout:7/336: write dd/f45 [193091,39958] 0 2024-03-18T14:50:53.495 INFO:tasks.workunit.client.0.smithi012.stdout:7/337: stat dd/d13/d23/d33/l46 0 2024-03-18T14:50:53.499 INFO:tasks.workunit.client.0.smithi012.stdout:7/338: dread f1 [0,4194304] 0 2024-03-18T14:50:53.506 INFO:tasks.workunit.client.0.smithi012.stdout:1/305: rename d8/db/d10/d31/d43/d5e/f5f to d8/db/d10/d31/d43/d5e/f62 0 2024-03-18T14:50:53.506 INFO:tasks.workunit.client.0.smithi012.stdout:1/306: fsync d8/f40 0 2024-03-18T14:50:53.508 INFO:tasks.workunit.client.0.smithi012.stdout:6/319: mkdir d17/d19/d27/d58 0 2024-03-18T14:50:53.514 INFO:tasks.workunit.client.0.smithi012.stdout:8/431: rmdir d0/d15/d4e 39 2024-03-18T14:50:53.514 INFO:tasks.workunit.client.0.smithi012.stdout:8/432: dread - d0/d47/d59/f83 zero size 2024-03-18T14:50:53.520 INFO:tasks.workunit.client.0.smithi012.stdout:7/339: dwrite dd/d13/d23/d2a/f68 [0,4194304] 0 2024-03-18T14:50:53.523 INFO:tasks.workunit.client.0.smithi012.stdout:1/307: unlink d8/db/ff 0 2024-03-18T14:50:53.523 INFO:tasks.workunit.client.0.smithi012.stdout:1/308: write d8/db/d10/d31/f4d [290469,90687] 0 2024-03-18T14:50:53.526 INFO:tasks.workunit.client.0.smithi012.stdout:6/320: truncate d17/d19/f3e 479322 0 2024-03-18T14:50:53.529 INFO:tasks.workunit.client.0.smithi012.stdout:0/333: mkdir d1/d16/d79 0 2024-03-18T14:50:53.529 INFO:tasks.workunit.client.0.smithi012.stdout:0/334: chown d1/d20/d23/d26/f40 25303392 1 2024-03-18T14:50:53.531 INFO:tasks.workunit.client.0.smithi012.stdout:8/433: unlink d0/d15/d6e/d84/f27 0 2024-03-18T14:50:53.539 INFO:tasks.workunit.client.0.smithi012.stdout:8/434: mkdir d0/d47/d59/d86/d87 0 2024-03-18T14:50:53.539 INFO:tasks.workunit.client.0.smithi012.stdout:8/435: truncate d0/d5a/d55/f53 468065 0 2024-03-18T14:50:53.540 INFO:tasks.workunit.client.0.smithi012.stdout:7/340: dwrite dd/d13/f75 [0,4194304] 0 2024-03-18T14:50:53.540 INFO:tasks.workunit.client.0.smithi012.stdout:7/341: fsync dd/d13/d23/d60/f6c 0 2024-03-18T14:50:53.547 INFO:tasks.workunit.client.0.smithi012.stdout:7/342: rename dd/d13/d23/d2a to dd/d13/d23/d76 0 2024-03-18T14:50:53.547 INFO:tasks.workunit.client.0.smithi012.stdout:7/343: write dd/d13/d23/d2f/d53/f5d [4605486,120747] 0 2024-03-18T14:50:53.547 INFO:tasks.workunit.client.0.smithi012.stdout:7/344: readlink dd/d13/d23/d33/l46 0 2024-03-18T14:50:53.551 INFO:tasks.workunit.client.0.smithi012.stdout:7/345: dread dd/f5b [0,4194304] 0 2024-03-18T14:50:53.552 INFO:tasks.workunit.client.0.smithi012.stdout:1/309: dwrite d8/db/d10/d31/d43/d5e/f62 [0,4194304] 0 2024-03-18T14:50:53.552 INFO:tasks.workunit.client.0.smithi012.stdout:0/335: dwrite d1/d16/d19/f37 [0,4194304] 0 2024-03-18T14:50:53.552 INFO:tasks.workunit.client.0.smithi012.stdout:0/336: chown d1/d20/d23/c25 4350750 1 2024-03-18T14:50:53.554 INFO:tasks.workunit.client.0.smithi012.stdout:7/346: symlink dd/d13/l77 0 2024-03-18T14:50:53.554 INFO:tasks.workunit.client.0.smithi012.stdout:7/347: read - dd/f5f zero size 2024-03-18T14:50:53.574 INFO:tasks.workunit.client.0.smithi012.stdout:7/348: dwrite dd/d13/d26/f5e [0,4194304] 0 2024-03-18T14:50:53.574 INFO:tasks.workunit.client.0.smithi012.stdout:7/349: write dd/d13/f73 [1446126,14586] 0 2024-03-18T14:50:53.574 INFO:tasks.workunit.client.0.smithi012.stdout:7/350: creat dd/d13/d23/d33/f78 x:0 0 0 2024-03-18T14:50:53.575 INFO:tasks.workunit.client.0.smithi012.stdout:7/351: stat dd/d13/d23/d2f/f32 0 2024-03-18T14:50:53.591 INFO:tasks.workunit.client.0.smithi012.stdout:7/352: mknod dd/d13/d23/d76/d6f/c79 0 2024-03-18T14:50:53.591 INFO:tasks.workunit.client.0.smithi012.stdout:7/353: dread - dd/d13/d23/f55 zero size 2024-03-18T14:50:53.594 INFO:tasks.workunit.client.0.smithi012.stdout:7/354: mkdir dd/d13/d23/d2f/d7a 0 2024-03-18T14:50:53.595 INFO:tasks.workunit.client.0.smithi012.stdout:7/355: truncate dd/d13/d23/f55 459092 0 2024-03-18T14:50:53.631 INFO:tasks.workunit.client.0.smithi012.stdout:3/245: dwrite dc/f44 [0,4194304] 0 2024-03-18T14:50:53.636 INFO:tasks.workunit.client.0.smithi012.stdout:3/246: link dc/c10 dc/d3c/c49 0 2024-03-18T14:50:53.640 INFO:tasks.workunit.client.0.smithi012.stdout:3/247: mknod dc/d30/c4a 0 2024-03-18T14:50:53.643 INFO:tasks.workunit.client.0.smithi012.stdout:3/248: creat dc/d30/d3a/f4b x:0 0 0 2024-03-18T14:50:53.645 INFO:tasks.workunit.client.0.smithi012.stdout:3/249: mkdir dc/d3c/d4c 0 2024-03-18T14:50:53.647 INFO:tasks.workunit.client.0.smithi012.stdout:9/372: sync 2024-03-18T14:50:53.647 INFO:tasks.workunit.client.0.smithi012.stdout:5/338: sync 2024-03-18T14:50:53.648 INFO:tasks.workunit.client.0.smithi012.stdout:4/343: getdents d6 0 2024-03-18T14:50:53.653 INFO:tasks.workunit.client.0.smithi012.stdout:6/321: rename d17/d38/d51/c56 to d17/d19/d27/d2f/c59 0 2024-03-18T14:50:53.654 INFO:tasks.workunit.client.0.smithi012.stdout:5/339: unlink d3/d1e/l12 0 2024-03-18T14:50:53.656 INFO:tasks.workunit.client.0.smithi012.stdout:4/344: chown d6/d1a/c55 49 1 2024-03-18T14:50:53.656 INFO:tasks.workunit.client.0.smithi012.stdout:4/345: fdatasync d6/f33 0 2024-03-18T14:50:53.659 INFO:tasks.workunit.client.0.smithi012.stdout:3/250: truncate dc/f11 1699407 0 2024-03-18T14:50:53.661 INFO:tasks.workunit.client.0.smithi012.stdout:9/373: dwrite d3/d9/d19/d66/f45 [0,4194304] 0 2024-03-18T14:50:53.663 INFO:tasks.workunit.client.0.smithi012.stdout:8/436: rename d0/d5a/l2e to d0/d15/l88 0 2024-03-18T14:50:53.669 INFO:tasks.workunit.client.0.smithi012.stdout:8/437: dread d0/f16 [0,4194304] 0 2024-03-18T14:50:53.669 INFO:tasks.workunit.client.0.smithi012.stdout:8/438: write d0/d15/d6e/d84/d54/f85 [195508,47897] 0 2024-03-18T14:50:53.671 INFO:tasks.workunit.client.0.smithi012.stdout:6/322: link d17/d26/f4a d17/d38/f5a 0 2024-03-18T14:50:53.674 INFO:tasks.workunit.client.0.smithi012.stdout:5/340: symlink d3/d29/d36/l72 0 2024-03-18T14:50:53.674 INFO:tasks.workunit.client.0.smithi012.stdout:5/341: fdatasync d3/f43 0 2024-03-18T14:50:53.674 INFO:tasks.workunit.client.0.smithi012.stdout:4/346: dwrite d6/fe [0,4194304] 0 2024-03-18T14:50:53.678 INFO:tasks.workunit.client.0.smithi012.stdout:6/323: dread d17/d19/f2b [0,4194304] 0 2024-03-18T14:50:53.681 INFO:tasks.workunit.client.0.smithi012.stdout:0/337: rename d1/d16/d18/c12 to d1/d16/d31/c7a 0 2024-03-18T14:50:53.681 INFO:tasks.workunit.client.0.smithi012.stdout:0/338: write d1/d20/d23/d26/f53 [1639450,130766] 0 2024-03-18T14:50:53.686 INFO:tasks.workunit.client.0.smithi012.stdout:6/324: mkdir d17/d38/d5b 0 2024-03-18T14:50:53.686 INFO:tasks.workunit.client.0.smithi012.stdout:6/325: truncate d17/d26/f28 680371 0 2024-03-18T14:50:53.686 INFO:tasks.workunit.client.0.smithi012.stdout:6/326: write d17/d19/f2b [4882443,44130] 0 2024-03-18T14:50:53.688 INFO:tasks.workunit.client.0.smithi012.stdout:9/374: dwrite d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:53.688 INFO:tasks.workunit.client.0.smithi012.stdout:9/375: write d3/f41 [349231,66909] 0 2024-03-18T14:50:53.688 INFO:tasks.workunit.client.0.smithi012.stdout:3/251: dwrite dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:50:53.689 INFO:tasks.workunit.client.0.smithi012.stdout:7/356: rename dd/d13/d23/d33/c54 to dd/d13/d23/c7b 0 2024-03-18T14:50:53.693 INFO:tasks.workunit.client.0.smithi012.stdout:0/339: unlink d1/d16/d31/l66 0 2024-03-18T14:50:53.707 INFO:tasks.workunit.client.0.smithi012.stdout:4/347: rmdir d6/da/d58 39 2024-03-18T14:50:53.710 INFO:tasks.workunit.client.0.smithi012.stdout:6/327: creat d17/d26/d36/d4d/f5c x:0 0 0 2024-03-18T14:50:53.719 INFO:tasks.workunit.client.0.smithi012.stdout:9/376: getdents d3/d9/d19/d66/d21/d4a 0 2024-03-18T14:50:53.719 INFO:tasks.workunit.client.0.smithi012.stdout:9/377: fsync d3/f63 0 2024-03-18T14:50:53.721 INFO:tasks.workunit.client.0.smithi012.stdout:3/252: creat dc/d3c/f4d x:0 0 0 2024-03-18T14:50:53.724 INFO:tasks.workunit.client.0.smithi012.stdout:8/439: rename d0/d15/l52 to d0/d15/d78/d80/l89 0 2024-03-18T14:50:53.725 INFO:tasks.workunit.client.0.smithi012.stdout:6/328: dwrite d17/d33/f4e [0,4194304] 0 2024-03-18T14:50:53.728 INFO:tasks.workunit.client.0.smithi012.stdout:0/340: truncate d1/d16/d31/f58 158343 0 2024-03-18T14:50:53.728 INFO:tasks.workunit.client.0.smithi012.stdout:0/341: readlink d1/d20/d23/l27 0 2024-03-18T14:50:53.731 INFO:tasks.workunit.client.0.smithi012.stdout:1/310: rmdir d8 39 2024-03-18T14:50:53.731 INFO:tasks.workunit.client.0.smithi012.stdout:1/311: dread - d8/db/d10/d4e/f53 zero size 2024-03-18T14:50:53.731 INFO:tasks.workunit.client.0.smithi012.stdout:1/312: readlink d8/db/d10/l1d 0 2024-03-18T14:50:53.731 INFO:tasks.workunit.client.0.smithi012.stdout:1/313: readlink d8/db/d10/d16/d54/l58 0 2024-03-18T14:50:53.735 INFO:tasks.workunit.client.0.smithi012.stdout:6/329: dread f14 [0,4194304] 0 2024-03-18T14:50:53.735 INFO:tasks.workunit.client.0.smithi012.stdout:9/378: link d3/d9/f10 d3/d9/d19/d66/d21/d35/d71/f76 0 2024-03-18T14:50:53.735 INFO:tasks.workunit.client.0.smithi012.stdout:9/379: fdatasync d3/f2e 0 2024-03-18T14:50:53.736 INFO:tasks.workunit.client.0.smithi012.stdout:0/342: dread d1/d20/d4e/f52 [0,4194304] 0 2024-03-18T14:50:53.738 INFO:tasks.workunit.client.0.smithi012.stdout:3/253: chown dc/c17 1833 1 2024-03-18T14:50:53.738 INFO:tasks.workunit.client.0.smithi012.stdout:3/254: write dc/d30/f37 [299081,67817] 0 2024-03-18T14:50:53.743 INFO:tasks.workunit.client.0.smithi012.stdout:8/440: rename d0/d47/c4c to d0/d15/d78/c8a 0 2024-03-18T14:50:53.747 INFO:tasks.workunit.client.0.smithi012.stdout:3/255: write dc/d30/d3a/f4b [2835768,117033] 0 2024-03-18T14:50:53.752 INFO:tasks.workunit.client.0.smithi012.stdout:4/348: rmdir d6/d19 39 2024-03-18T14:50:53.756 INFO:tasks.workunit.client.0.smithi012.stdout:1/314: dwrite d8/d1e/f4a [0,4194304] 0 2024-03-18T14:50:53.758 INFO:tasks.workunit.client.0.smithi012.stdout:9/380: rename d3/d9/d19/d66/d21/d4a to d3/d9/d19/d77 0 2024-03-18T14:50:53.767 INFO:tasks.workunit.client.0.smithi012.stdout:3/256: fdatasync dc/f22 0 2024-03-18T14:50:53.773 INFO:tasks.workunit.client.0.smithi012.stdout:4/349: truncate d6/d1a/f31 2408961 0 2024-03-18T14:50:53.774 INFO:tasks.workunit.client.0.smithi012.stdout:1/315: dwrite f5 [0,4194304] 0 2024-03-18T14:50:53.784 INFO:tasks.workunit.client.0.smithi012.stdout:4/350: mkdir d6/d76 0 2024-03-18T14:50:53.786 INFO:tasks.workunit.client.0.smithi012.stdout:1/316: creat d8/d1e/f63 x:0 0 0 2024-03-18T14:50:53.786 INFO:tasks.workunit.client.0.smithi012.stdout:1/317: chown d8/d1e/l36 79 1 2024-03-18T14:50:53.789 INFO:tasks.workunit.client.0.smithi012.stdout:9/381: dwrite d3/f4 [0,4194304] 0 2024-03-18T14:50:53.789 INFO:tasks.workunit.client.0.smithi012.stdout:8/441: dwrite d0/d5a/f25 [0,4194304] 0 2024-03-18T14:50:53.792 INFO:tasks.workunit.client.0.smithi012.stdout:4/351: symlink d6/da/d58/l77 0 2024-03-18T14:50:53.793 INFO:tasks.workunit.client.0.smithi012.stdout:8/442: read d0/d15/f43 [2038587,60941] 0 2024-03-18T14:50:53.793 INFO:tasks.workunit.client.0.smithi012.stdout:8/443: creat d0/d5a/f8b x:0 0 0 2024-03-18T14:50:53.795 INFO:tasks.workunit.client.0.smithi012.stdout:1/318: unlink f4 0 2024-03-18T14:50:53.795 INFO:tasks.workunit.client.0.smithi012.stdout:1/319: fsync d8/db/fd 0 2024-03-18T14:50:53.795 INFO:tasks.workunit.client.0.smithi012.stdout:1/320: readlink d8/db/d10/d16/l49 0 2024-03-18T14:50:53.797 INFO:tasks.workunit.client.0.smithi012.stdout:9/382: dread d3/d9/d19/d66/f45 [0,4194304] 0 2024-03-18T14:50:53.800 INFO:tasks.workunit.client.0.smithi012.stdout:4/352: creat d6/d1a/d64/f78 x:0 0 0 2024-03-18T14:50:53.815 INFO:tasks.workunit.client.0.smithi012.stdout:9/383: dwrite d3/d9/f6c [0,4194304] 0 2024-03-18T14:50:53.815 INFO:tasks.workunit.client.0.smithi012.stdout:1/321: dwrite d8/db/d10/d16/d1c/f4c [0,4194304] 0 2024-03-18T14:50:53.820 INFO:tasks.workunit.client.0.smithi012.stdout:9/384: mkdir d3/d9/d19/d77/d52/d78 0 2024-03-18T14:50:53.825 INFO:tasks.workunit.client.0.smithi012.stdout:1/322: mknod d8/db/d10/d31/d43/d5e/c64 0 2024-03-18T14:50:53.826 INFO:tasks.workunit.client.0.smithi012.stdout:9/385: mknod d3/d9/d19/d77/d52/c79 0 2024-03-18T14:50:53.828 INFO:tasks.workunit.client.0.smithi012.stdout:3/257: getdents dc/d30/d3a 0 2024-03-18T14:50:53.829 INFO:tasks.workunit.client.0.smithi012.stdout:1/323: mkdir d8/db/d10/d31/d43/d5e/d65 0 2024-03-18T14:50:53.830 INFO:tasks.workunit.client.0.smithi012.stdout:9/386: symlink d3/d49/l7a 0 2024-03-18T14:50:53.833 INFO:tasks.workunit.client.0.smithi012.stdout:2/354: sync 2024-03-18T14:50:53.836 INFO:tasks.workunit.client.0.smithi012.stdout:1/324: link f5 d8/db/d10/d16/f66 0 2024-03-18T14:50:53.838 INFO:tasks.workunit.client.0.smithi012.stdout:3/258: mknod dc/c4e 0 2024-03-18T14:50:53.838 INFO:tasks.workunit.client.0.smithi012.stdout:3/259: write dc/d3c/f4d [515477,74287] 0 2024-03-18T14:50:53.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/355: rmdir d0/d10/d18/d57 39 2024-03-18T14:50:53.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/356: getdents d0/d6b 0 2024-03-18T14:50:53.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/357: chown d0/l29 0 1 2024-03-18T14:50:53.844 INFO:tasks.workunit.client.0.smithi012.stdout:9/387: dwrite d3/f25 [4194304,4194304] 0 2024-03-18T14:50:53.847 INFO:tasks.workunit.client.0.smithi012.stdout:5/342: rmdir d3/d29 39 2024-03-18T14:50:53.850 INFO:tasks.workunit.client.0.smithi012.stdout:5/343: rmdir d3/d29/d36 39 2024-03-18T14:50:53.851 INFO:tasks.workunit.client.0.smithi012.stdout:5/344: write d3/d1e/f28 [8822532,76760] 0 2024-03-18T14:50:53.855 INFO:tasks.workunit.client.0.smithi012.stdout:0/343: rename d1/d16/d18 to d1/d20/d7b 0 2024-03-18T14:50:53.855 INFO:tasks.workunit.client.0.smithi012.stdout:8/444: rename d0/d15/d6e/d84/d54 to d0/d15/d6e/d84/d54/d62/d8c 22 2024-03-18T14:50:53.855 INFO:tasks.workunit.client.0.smithi012.stdout:0/344: write d1/d20/d23/d26/f36 [4322483,89531] 0 2024-03-18T14:50:53.855 INFO:tasks.workunit.client.0.smithi012.stdout:8/445: write d0/d5a/f8b [556095,125366] 0 2024-03-18T14:50:53.855 INFO:tasks.workunit.client.0.smithi012.stdout:0/345: write d1/f59 [1020546,69234] 0 2024-03-18T14:50:53.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/357: getdents dd/d13/d23/d33 0 2024-03-18T14:50:53.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/358: dread - dd/d13/d23/d33/f78 zero size 2024-03-18T14:50:53.856 INFO:tasks.workunit.client.0.smithi012.stdout:7/359: write dd/d13/d23/d2f/f4c [1243464,130527] 0 2024-03-18T14:50:53.858 INFO:tasks.workunit.client.0.smithi012.stdout:4/353: rename d6/f44 to d6/d19/d4f/f79 0 2024-03-18T14:50:53.858 INFO:tasks.workunit.client.0.smithi012.stdout:4/354: truncate d6/d19/d4f/f79 1024584 0 2024-03-18T14:50:53.862 INFO:tasks.workunit.client.0.smithi012.stdout:8/446: creat d0/d15/d6e/d84/d54/f8d x:0 0 0 2024-03-18T14:50:53.862 INFO:tasks.workunit.client.0.smithi012.stdout:8/447: read d0/d5a/d19/f29 [3117779,42072] 0 2024-03-18T14:50:53.862 INFO:tasks.workunit.client.0.smithi012.stdout:8/448: chown d0/d12/d33/d50/c74 64803125 1 2024-03-18T14:50:53.867 INFO:tasks.workunit.client.0.smithi012.stdout:4/355: creat d6/d19/d6c/f7a x:0 0 0 2024-03-18T14:50:53.869 INFO:tasks.workunit.client.0.smithi012.stdout:0/346: getdents d1/d2d 0 2024-03-18T14:50:53.870 INFO:tasks.workunit.client.0.smithi012.stdout:8/449: write d0/d15/d6e/d84/d54/f72 [2193730,21275] 0 2024-03-18T14:50:53.870 INFO:tasks.workunit.client.0.smithi012.stdout:8/450: dread - d0/d5a/d5e/f82 zero size 2024-03-18T14:50:53.870 INFO:tasks.workunit.client.0.smithi012.stdout:7/360: rmdir dd/d13/d23/d60 39 2024-03-18T14:50:53.873 INFO:tasks.workunit.client.0.smithi012.stdout:3/260: rename dc/ff to dc/d30/d3a/f4f 0 2024-03-18T14:50:53.875 INFO:tasks.workunit.client.0.smithi012.stdout:4/356: rmdir d6/da/d58 39 2024-03-18T14:50:53.875 INFO:tasks.workunit.client.0.smithi012.stdout:4/357: dread - d6/da/f16 zero size 2024-03-18T14:50:53.878 INFO:tasks.workunit.client.0.smithi012.stdout:0/347: mknod d1/d20/d23/d6f/d77/c7c 0 2024-03-18T14:50:53.878 INFO:tasks.workunit.client.0.smithi012.stdout:0/348: fsync d1/d16/d31/f35 0 2024-03-18T14:50:53.885 INFO:tasks.workunit.client.0.smithi012.stdout:4/358: unlink d6/d1a/d30/d46/c5a 0 2024-03-18T14:50:53.888 INFO:tasks.workunit.client.0.smithi012.stdout:0/349: link d1/d20/f6a d1/d16/d19/f7d 0 2024-03-18T14:50:53.891 INFO:tasks.workunit.client.0.smithi012.stdout:5/345: rename d3/d1e/d33/d62 to d3/d29/d36/d3d/d73 0 2024-03-18T14:50:53.893 INFO:tasks.workunit.client.0.smithi012.stdout:3/261: dwrite dc/d30/f37 [0,4194304] 0 2024-03-18T14:50:53.894 INFO:tasks.workunit.client.0.smithi012.stdout:8/451: dwrite d0/d5a/d19/d57/f68 [0,4194304] 0 2024-03-18T14:50:53.894 INFO:tasks.workunit.client.0.smithi012.stdout:8/452: write d0/d47/f4f [1120289,12310] 0 2024-03-18T14:50:53.897 INFO:tasks.workunit.client.0.smithi012.stdout:7/361: rename dd/d13/d23/d76/l34 to dd/d13/d26/l7c 0 2024-03-18T14:50:53.901 INFO:tasks.workunit.client.0.smithi012.stdout:5/346: creat d3/d1e/d33/f74 x:0 0 0 2024-03-18T14:50:53.901 INFO:tasks.workunit.client.0.smithi012.stdout:4/359: read d6/d1a/d30/f47 [3734887,86332] 0 2024-03-18T14:50:53.903 INFO:tasks.workunit.client.0.smithi012.stdout:4/360: dread d6/d19/d39/f6d [0,4194304] 0 2024-03-18T14:50:53.905 INFO:tasks.workunit.client.0.smithi012.stdout:3/262: mkdir dc/d3c/d4c/d50 0 2024-03-18T14:50:53.914 INFO:tasks.workunit.client.0.smithi012.stdout:4/361: truncate d6/d19/f2f 2176651 0 2024-03-18T14:50:53.914 INFO:tasks.workunit.client.0.smithi012.stdout:4/362: write d6/f5c [1516143,71892] 0 2024-03-18T14:50:53.914 INFO:tasks.workunit.client.0.smithi012.stdout:4/363: truncate d6/d19/d39/f40 1122563 0 2024-03-18T14:50:53.914 INFO:tasks.workunit.client.0.smithi012.stdout:3/263: readlink dc/le 0 2024-03-18T14:50:53.914 INFO:tasks.workunit.client.0.smithi012.stdout:0/350: dwrite d1/d20/f5a [4194304,4194304] 0 2024-03-18T14:50:53.915 INFO:tasks.workunit.client.0.smithi012.stdout:8/453: dwrite d0/d15/d6e/d84/f6b [0,4194304] 0 2024-03-18T14:50:53.916 INFO:tasks.workunit.client.0.smithi012.stdout:8/454: write d0/d5a/f60 [3686075,61453] 0 2024-03-18T14:50:53.916 INFO:tasks.workunit.client.0.smithi012.stdout:8/455: fsync d0/d5a/d19/d57/f69 0 2024-03-18T14:50:53.916 INFO:tasks.workunit.client.0.smithi012.stdout:8/456: creat d0/d47/d59/f8e x:0 0 0 2024-03-18T14:50:53.918 INFO:tasks.workunit.client.0.smithi012.stdout:4/364: unlink d6/d1a/d30/f6e 0 2024-03-18T14:50:53.918 INFO:tasks.workunit.client.0.smithi012.stdout:4/365: truncate d6/f2e 683474 0 2024-03-18T14:50:53.930 INFO:tasks.workunit.client.0.smithi012.stdout:5/347: dwrite d3/d15/f2c [0,4194304] 0 2024-03-18T14:50:53.938 INFO:tasks.workunit.client.0.smithi012.stdout:0/351: dwrite d1/d16/d31/f50 [0,4194304] 0 2024-03-18T14:50:53.938 INFO:tasks.workunit.client.0.smithi012.stdout:3/264: dwrite dc/f2b [0,4194304] 0 2024-03-18T14:50:53.945 INFO:tasks.workunit.client.0.smithi012.stdout:3/265: dread dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:50:53.959 INFO:tasks.workunit.client.0.smithi012.stdout:8/457: mknod d0/d15/d6e/d84/d2a/d3e/c8f 0 2024-03-18T14:50:53.960 INFO:tasks.workunit.client.0.smithi012.stdout:8/458: write d0/d47/d59/f8e [669326,103148] 0 2024-03-18T14:50:53.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/366: rename d6/d1a/d30/c6a to d6/da/d58/c7b 0 2024-03-18T14:50:53.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/367: write d6/d19/f42 [447881,123697] 0 2024-03-18T14:50:53.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/368: fdatasync d6/d19/d39/f40 0 2024-03-18T14:50:53.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/369: fdatasync d6/da/f54 0 2024-03-18T14:50:53.966 INFO:tasks.workunit.client.0.smithi012.stdout:5/348: symlink d3/d15/l75 0 2024-03-18T14:50:53.981 INFO:tasks.workunit.client.0.smithi012.stdout:0/352: rename c0 to d1/d20/d23/d6f/c7e 0 2024-03-18T14:50:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:0/353: read d1/d16/fe [66630,38638] 0 2024-03-18T14:50:53.982 INFO:tasks.workunit.client.0.smithi012.stdout:0/354: fdatasync d1/d16/d31/f1d 0 2024-03-18T14:50:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:3/266: read dc/f32 [3100353,40374] 0 2024-03-18T14:50:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:4/370: mkdir d6/d1a/d30/d46/d6f/d7c 0 2024-03-18T14:50:53.987 INFO:tasks.workunit.client.0.smithi012.stdout:4/371: write d6/d1a/d30/d46/f48 [508605,127815] 0 2024-03-18T14:50:53.991 INFO:tasks.workunit.client.0.smithi012.stdout:5/349: dwrite d3/d29/d36/d3d/d73/f58 [0,4194304] 0 2024-03-18T14:50:53.995 INFO:tasks.workunit.client.0.smithi012.stdout:8/459: rename d0/d15/d6e/d84/c36 to d0/d5a/d19/c90 0 2024-03-18T14:50:53.996 INFO:tasks.workunit.client.0.smithi012.stdout:8/460: write d0/d15/d6e/d84/d54/f72 [853992,99255] 0 2024-03-18T14:50:53.998 INFO:tasks.workunit.client.0.smithi012.stdout:4/372: mknod d6/c7d 0 2024-03-18T14:50:53.999 INFO:tasks.workunit.client.0.smithi012.stdout:4/373: stat d6/c10 0 2024-03-18T14:50:54.003 INFO:tasks.workunit.client.0.smithi012.stdout:1/325: truncate d8/db/d10/d16/f66 7954756 0 2024-03-18T14:50:54.007 INFO:tasks.workunit.client.0.smithi012.stdout:4/374: mknod d6/d1a/d30/d46/d6f/d7c/c7e 0 2024-03-18T14:50:54.009 INFO:tasks.workunit.client.0.smithi012.stdout:5/350: getdents d3/d1e 0 2024-03-18T14:50:54.010 INFO:tasks.workunit.client.0.smithi012.stdout:0/355: dwrite d1/d20/d7b/ff [0,4194304] 0 2024-03-18T14:50:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:0/356: write d1/d20/d23/d6f/f75 [467354,102698] 0 2024-03-18T14:50:54.011 INFO:tasks.workunit.client.0.smithi012.stdout:0/357: fdatasync d1/d16/f17 0 2024-03-18T14:50:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:1/326: symlink d8/db/d10/d16/l67 0 2024-03-18T14:50:54.012 INFO:tasks.workunit.client.0.smithi012.stdout:1/327: readlink d8/db/l3d 0 2024-03-18T14:50:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:8/461: dwrite d0/d5a/d19/d57/f68 [0,4194304] 0 2024-03-18T14:50:54.014 INFO:tasks.workunit.client.0.smithi012.stdout:4/375: unlink d6/da/l3f 0 2024-03-18T14:50:54.015 INFO:tasks.workunit.client.0.smithi012.stdout:3/267: dwrite dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:54.016 INFO:tasks.workunit.client.0.smithi012.stdout:5/351: link d3/d15/f1d d3/d29/d36/d3d/d73/f76 0 2024-03-18T14:50:54.020 INFO:tasks.workunit.client.0.smithi012.stdout:1/328: creat d8/db/d10/d31/f68 x:0 0 0 2024-03-18T14:50:54.026 INFO:tasks.workunit.client.0.smithi012.stdout:4/376: symlink d6/da/d58/l7f 0 2024-03-18T14:50:54.035 INFO:tasks.workunit.client.0.smithi012.stdout:0/358: rename d1/d20/d23/d6f/d77 to d1/d20/d23/d7f 0 2024-03-18T14:50:54.042 INFO:tasks.workunit.client.0.smithi012.stdout:5/352: symlink d3/d29/d36/d3d/d73/l77 0 2024-03-18T14:50:54.045 INFO:tasks.workunit.client.0.smithi012.stdout:1/329: creat d8/db/d10/d31/d43/f69 x:0 0 0 2024-03-18T14:50:54.045 INFO:tasks.workunit.client.0.smithi012.stdout:1/330: chown d8/db/f30 5725387 1 2024-03-18T14:50:54.048 INFO:tasks.workunit.client.0.smithi012.stdout:3/268: dread dc/f48 [0,4194304] 0 2024-03-18T14:50:54.048 INFO:tasks.workunit.client.0.smithi012.stdout:3/269: chown dc/d30/c46 21395215 1 2024-03-18T14:50:54.050 INFO:tasks.workunit.client.0.smithi012.stdout:4/377: creat d6/f80 x:0 0 0 2024-03-18T14:50:54.051 INFO:tasks.workunit.client.0.smithi012.stdout:0/359: mknod d1/d16/d79/c80 0 2024-03-18T14:50:54.053 INFO:tasks.workunit.client.0.smithi012.stdout:5/353: rmdir d3/d29/d35/d5b 39 2024-03-18T14:50:54.053 INFO:tasks.workunit.client.0.smithi012.stdout:5/354: write f2 [1348451,15254] 0 2024-03-18T14:50:54.053 INFO:tasks.workunit.client.0.smithi012.stdout:5/355: write d3/d15/f1c [2334745,49070] 0 2024-03-18T14:50:54.056 INFO:tasks.workunit.client.0.smithi012.stdout:4/378: rename d6/ff to d6/d1a/d30/d46/d6f/d7c/f81 0 2024-03-18T14:50:54.064 INFO:tasks.workunit.client.0.smithi012.stdout:5/356: symlink d3/d29/l78 0 2024-03-18T14:50:54.065 INFO:tasks.workunit.client.0.smithi012.stdout:3/270: dwrite dc/d30/d35/f47 [0,4194304] 0 2024-03-18T14:50:54.067 INFO:tasks.workunit.client.0.smithi012.stdout:0/360: getdents d1/d16/d19 0 2024-03-18T14:50:54.075 INFO:tasks.workunit.client.0.smithi012.stdout:5/357: dread d3/d29/d36/d3d/d73/f58 [0,4194304] 0 2024-03-18T14:50:54.075 INFO:tasks.workunit.client.0.smithi012.stdout:3/271: unlink dc/c1a 0 2024-03-18T14:50:54.078 INFO:tasks.workunit.client.0.smithi012.stdout:5/358: rename d3/d15/d42/c64 to d3/d15/c79 0 2024-03-18T14:50:54.081 INFO:tasks.workunit.client.0.smithi012.stdout:5/359: creat d3/d29/f7a x:0 0 0 2024-03-18T14:50:54.082 INFO:tasks.workunit.client.0.smithi012.stdout:5/360: readlink d3/d29/d35/l53 0 2024-03-18T14:50:54.084 INFO:tasks.workunit.client.0.smithi012.stdout:3/272: rename dc/f27 to dc/d30/f51 0 2024-03-18T14:50:54.084 INFO:tasks.workunit.client.0.smithi012.stdout:3/273: write dc/f43 [1211503,66973] 0 2024-03-18T14:50:54.086 INFO:tasks.workunit.client.0.smithi012.stdout:0/361: dwrite d1/fa [4194304,4194304] 0 2024-03-18T14:50:54.088 INFO:tasks.workunit.client.0.smithi012.stdout:3/274: creat dc/d3c/d4c/d50/f52 x:0 0 0 2024-03-18T14:50:54.089 INFO:tasks.workunit.client.0.smithi012.stdout:3/275: chown dc/d3c/d4c 915560 1 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:0/362: creat d1/d20/d7b/f81 x:0 0 0 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:0/363: readlink d1/l8 0 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:0/364: fsync d1/d16/d19/f68 0 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:3/276: dread f3 [0,4194304] 0 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:3/277: readlink dc/d3c/l45 0 2024-03-18T14:50:54.093 INFO:tasks.workunit.client.0.smithi012.stdout:5/361: read d3/d15/f1d [7536778,65091] 0 2024-03-18T14:50:54.098 INFO:tasks.workunit.client.0.smithi012.stdout:5/362: mknod d3/d15/d21/d59/c7b 0 2024-03-18T14:50:54.098 INFO:tasks.workunit.client.0.smithi012.stdout:5/363: dread - d3/d1e/d33/f74 zero size 2024-03-18T14:50:54.098 INFO:tasks.workunit.client.0.smithi012.stdout:5/364: chown d3/d15/d5f/f70 7889 1 2024-03-18T14:50:54.099 INFO:tasks.workunit.client.0.smithi012.stdout:8/462: truncate d0/d15/d6e/d84/f6b 2516928 0 2024-03-18T14:50:54.099 INFO:tasks.workunit.client.0.smithi012.stdout:8/463: dread - d0/d47/d59/f83 zero size 2024-03-18T14:50:54.107 INFO:tasks.workunit.client.0.smithi012.stdout:5/365: rename d3/d1e/d33/f34 to d3/d15/d21/d59/f7c 0 2024-03-18T14:50:54.107 INFO:tasks.workunit.client.0.smithi012.stdout:5/366: fsync d3/d15/f1b 0 2024-03-18T14:50:54.110 INFO:tasks.workunit.client.0.smithi012.stdout:5/367: creat d3/d29/d36/d3d/d73/f7d x:0 0 0 2024-03-18T14:50:54.110 INFO:tasks.workunit.client.0.smithi012.stdout:5/368: write d3/d1e/f28 [6245178,25051] 0 2024-03-18T14:50:54.115 INFO:tasks.workunit.client.0.smithi012.stdout:0/365: dwrite d1/d20/d7b/f81 [0,4194304] 0 2024-03-18T14:50:54.124 INFO:tasks.workunit.client.0.smithi012.stdout:8/464: dwrite d0/d12/f2d [0,4194304] 0 2024-03-18T14:50:54.124 INFO:tasks.workunit.client.0.smithi012.stdout:8/465: chown d0/d47/d59/f83 1055 1 2024-03-18T14:50:54.124 INFO:tasks.workunit.client.0.smithi012.stdout:5/369: dread d3/d29/d35/f5a [0,4194304] 0 2024-03-18T14:50:54.124 INFO:tasks.workunit.client.0.smithi012.stdout:3/278: dwrite dc/f48 [0,4194304] 0 2024-03-18T14:50:54.124 INFO:tasks.workunit.client.0.smithi012.stdout:3/279: readlink dc/d30/l3b 0 2024-03-18T14:50:54.131 INFO:tasks.workunit.client.0.smithi012.stdout:8/466: rename d0/d5a/d55/c64 to d0/d5a/d55/c91 0 2024-03-18T14:50:54.132 INFO:tasks.workunit.client.0.smithi012.stdout:8/467: chown d0/d47/d59/d67 29645 1 2024-03-18T14:50:54.135 INFO:tasks.workunit.client.0.smithi012.stdout:3/280: mknod dc/d30/d35/c53 0 2024-03-18T14:50:54.137 INFO:tasks.workunit.client.0.smithi012.stdout:0/366: dwrite d1/d20/d23/d3b/d33/f70 [0,4194304] 0 2024-03-18T14:50:54.138 INFO:tasks.workunit.client.0.smithi012.stdout:0/367: readlink d1/d16/d19/l34 0 2024-03-18T14:50:54.138 INFO:tasks.workunit.client.0.smithi012.stdout:0/368: read d1/f4f [3732273,106084] 0 2024-03-18T14:50:54.138 INFO:tasks.workunit.client.0.smithi012.stdout:0/369: fdatasync d1/f3 0 2024-03-18T14:50:54.148 INFO:tasks.workunit.client.0.smithi012.stdout:5/370: dwrite f2 [0,4194304] 0 2024-03-18T14:50:54.148 INFO:tasks.workunit.client.0.smithi012.stdout:3/281: read dc/d30/f51 [4013480,44078] 0 2024-03-18T14:50:54.150 INFO:tasks.workunit.client.0.smithi012.stdout:8/468: dwrite d0/d5a/f7d [0,4194304] 0 2024-03-18T14:50:54.151 INFO:tasks.workunit.client.0.smithi012.stdout:0/370: dread d1/d20/d7b/f81 [0,4194304] 0 2024-03-18T14:50:54.154 INFO:tasks.workunit.client.0.smithi012.stdout:5/371: unlink d3/d1e/f6f 0 2024-03-18T14:50:54.154 INFO:tasks.workunit.client.0.smithi012.stdout:5/372: dread - d3/d29/f57 zero size 2024-03-18T14:50:54.154 INFO:tasks.workunit.client.0.smithi012.stdout:5/373: truncate d3/d29/d36/d3d/f6e 844828 0 2024-03-18T14:50:54.154 INFO:tasks.workunit.client.0.smithi012.stdout:5/374: fdatasync d3/d15/f1b 0 2024-03-18T14:50:54.155 INFO:tasks.workunit.client.0.smithi012.stdout:3/282: mkdir dc/d3c/d4c/d50/d54 0 2024-03-18T14:50:54.157 INFO:tasks.workunit.client.0.smithi012.stdout:5/375: dread d3/d15/f37 [0,4194304] 0 2024-03-18T14:50:54.157 INFO:tasks.workunit.client.0.smithi012.stdout:5/376: readlink d3/d1e/l32 0 2024-03-18T14:50:54.157 INFO:tasks.workunit.client.0.smithi012.stdout:5/377: creat d3/d29/d36/d3d/d73/f7e x:0 0 0 2024-03-18T14:50:54.157 INFO:tasks.workunit.client.0.smithi012.stdout:5/378: readlink d3/d29/d35/d5b/l66 0 2024-03-18T14:50:54.160 INFO:tasks.workunit.client.0.smithi012.stdout:8/469: creat d0/d15/d78/d80/f92 x:0 0 0 2024-03-18T14:50:54.161 INFO:tasks.workunit.client.0.smithi012.stdout:8/470: write d0/d12/f7c [493484,334] 0 2024-03-18T14:50:54.161 INFO:tasks.workunit.client.0.smithi012.stdout:8/471: fdatasync d0/fc 0 2024-03-18T14:50:54.161 INFO:tasks.workunit.client.0.smithi012.stdout:8/472: readlink d0/d5a/d19/l56 0 2024-03-18T14:50:54.163 INFO:tasks.workunit.client.0.smithi012.stdout:0/371: unlink d1/d20/f6a 0 2024-03-18T14:50:54.164 INFO:tasks.workunit.client.0.smithi012.stdout:0/372: read d1/d20/d23/d6f/f75 [349086,75844] 0 2024-03-18T14:50:54.168 INFO:tasks.workunit.client.0.smithi012.stdout:5/379: link d3/d29/f7a d3/d29/d36/d3d/f7f 0 2024-03-18T14:50:54.168 INFO:tasks.workunit.client.0.smithi012.stdout:5/380: write d3/d29/d36/f6d [1040607,10043] 0 2024-03-18T14:50:54.170 INFO:tasks.workunit.client.0.smithi012.stdout:8/473: creat d0/d5a/d19/f93 x:0 0 0 2024-03-18T14:50:54.171 INFO:tasks.workunit.client.0.smithi012.stdout:8/474: write d0/d47/d59/f8e [155878,97902] 0 2024-03-18T14:50:54.171 INFO:tasks.workunit.client.0.smithi012.stdout:0/373: mkdir d1/d20/d23/d82 0 2024-03-18T14:50:54.173 INFO:tasks.workunit.client.0.smithi012.stdout:5/381: symlink d3/d15/d21/l80 0 2024-03-18T14:50:54.173 INFO:tasks.workunit.client.0.smithi012.stdout:5/382: chown d3/d15/f1d 25129800 1 2024-03-18T14:50:54.174 INFO:tasks.workunit.client.0.smithi012.stdout:5/383: truncate d3/d29/d36/d3d/d73/f7e 569524 0 2024-03-18T14:50:54.174 INFO:tasks.workunit.client.0.smithi012.stdout:3/283: dwrite dc/d30/f51 [0,4194304] 0 2024-03-18T14:50:54.179 INFO:tasks.workunit.client.0.smithi012.stdout:0/374: rename d1/d16/d31/c76 to d1/d2d/c83 0 2024-03-18T14:50:54.179 INFO:tasks.workunit.client.0.smithi012.stdout:0/375: write d1/d20/d7b/f30 [88199,18614] 0 2024-03-18T14:50:54.179 INFO:tasks.workunit.client.0.smithi012.stdout:0/376: write d1/d16/d19/f68 [36850,122063] 0 2024-03-18T14:50:54.180 INFO:tasks.workunit.client.0.smithi012.stdout:3/284: symlink dc/d3c/l55 0 2024-03-18T14:50:54.182 INFO:tasks.workunit.client.0.smithi012.stdout:8/475: rmdir d0/d5a/d5e 39 2024-03-18T14:50:54.183 INFO:tasks.workunit.client.0.smithi012.stdout:8/476: read d0/d5a/f6d [678160,127105] 0 2024-03-18T14:50:54.186 INFO:tasks.workunit.client.0.smithi012.stdout:0/377: dread d1/f9 [0,4194304] 0 2024-03-18T14:50:54.186 INFO:tasks.workunit.client.0.smithi012.stdout:0/378: rename d1 to d1/d20/d4e/d54/d67/d84 22 2024-03-18T14:50:54.186 INFO:tasks.workunit.client.0.smithi012.stdout:0/379: fsync d1/d20/d23/f46 0 2024-03-18T14:50:54.188 INFO:tasks.workunit.client.0.smithi012.stdout:3/285: symlink dc/d3c/d4c/d50/d54/l56 0 2024-03-18T14:50:54.188 INFO:tasks.workunit.client.0.smithi012.stdout:3/286: fdatasync dc/d30/f51 0 2024-03-18T14:50:54.189 INFO:tasks.workunit.client.0.smithi012.stdout:5/384: dwrite d3/d29/d36/d3d/f7f [0,4194304] 0 2024-03-18T14:50:54.190 INFO:tasks.workunit.client.0.smithi012.stdout:8/477: creat d0/d15/f94 x:0 0 0 2024-03-18T14:50:54.191 INFO:tasks.workunit.client.0.smithi012.stdout:0/380: getdents d1/d16/d79 0 2024-03-18T14:50:54.193 INFO:tasks.workunit.client.0.smithi012.stdout:3/287: mkdir dc/d3c/d4c/d57 0 2024-03-18T14:50:54.199 INFO:tasks.workunit.client.0.smithi012.stdout:5/385: symlink d3/d29/d35/l81 0 2024-03-18T14:50:54.201 INFO:tasks.workunit.client.0.smithi012.stdout:8/478: mknod d0/d15/d6e/d84/c95 0 2024-03-18T14:50:54.201 INFO:tasks.workunit.client.0.smithi012.stdout:8/479: write d0/d5a/f8b [782766,32886] 0 2024-03-18T14:50:54.201 INFO:tasks.workunit.client.0.smithi012.stdout:8/480: creat d0/d15/d6e/d84/d54/f96 x:0 0 0 2024-03-18T14:50:54.203 INFO:tasks.workunit.client.0.smithi012.stdout:0/381: mknod d1/d2d/c85 0 2024-03-18T14:50:54.203 INFO:tasks.workunit.client.0.smithi012.stdout:0/382: dread - d1/d2d/f62 zero size 2024-03-18T14:50:54.206 INFO:tasks.workunit.client.0.smithi012.stdout:3/288: creat dc/d3c/d4c/d50/d54/f58 x:0 0 0 2024-03-18T14:50:54.224 INFO:tasks.workunit.client.0.smithi012.stdout:5/386: rename d3/d15/d5f/c51 to d3/d29/d36/c82 0 2024-03-18T14:50:54.228 INFO:tasks.workunit.client.0.smithi012.stdout:8/481: creat d0/d12/d33/f97 x:0 0 0 2024-03-18T14:50:54.228 INFO:tasks.workunit.client.0.smithi012.stdout:8/482: read - d0/d15/d78/d80/f92 zero size 2024-03-18T14:50:54.229 INFO:tasks.workunit.client.0.smithi012.stdout:5/387: dread d3/d29/d36/f6d [0,4194304] 0 2024-03-18T14:50:54.229 INFO:tasks.workunit.client.0.smithi012.stdout:5/388: dread - d3/d29/d36/d3d/d73/f7d zero size 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:2/358: sync 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:7/362: sync 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:6/330: sync 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:9/388: sync 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:2/359: creat d0/f6e x:0 0 0 2024-03-18T14:50:54.231 INFO:tasks.workunit.client.0.smithi012.stdout:2/360: creat d0/d10/d18/d4f/f6f x:0 0 0 2024-03-18T14:50:54.232 INFO:tasks.workunit.client.0.smithi012.stdout:0/383: unlink d1/f59 0 2024-03-18T14:50:54.233 INFO:tasks.workunit.client.0.smithi012.stdout:3/289: mkdir dc/d3c/d59 0 2024-03-18T14:50:54.236 INFO:tasks.workunit.client.0.smithi012.stdout:8/483: mknod d0/d5a/c98 0 2024-03-18T14:50:54.239 INFO:tasks.workunit.client.0.smithi012.stdout:5/389: creat d3/d15/f83 x:0 0 0 2024-03-18T14:50:54.239 INFO:tasks.workunit.client.0.smithi012.stdout:5/390: readlink d3/l52 0 2024-03-18T14:50:54.239 INFO:tasks.workunit.client.0.smithi012.stdout:5/391: chown d3/d15/d5f/f70 38031842 1 2024-03-18T14:50:54.239 INFO:tasks.workunit.client.0.smithi012.stdout:5/392: readlink d3/l6 0 2024-03-18T14:50:54.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/363: dread dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:50:54.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:53 smithi012 ceph-mon[29182]: pgmap v244: 65 pgs: 65 active+clean; 873 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 74 MiB/s rd, 100 MiB/s wr, 529 op/s 2024-03-18T14:50:54.244 INFO:tasks.workunit.client.0.smithi012.stdout:6/331: mknod d17/d26/d36/d4d/c5d 0 2024-03-18T14:50:54.246 INFO:tasks.workunit.client.0.smithi012.stdout:9/389: mknod d3/d9/d5c/c7b 0 2024-03-18T14:50:54.246 INFO:tasks.workunit.client.0.smithi012.stdout:9/390: readlink d3/d9/d19/d66/d6e/l73 0 2024-03-18T14:50:54.247 INFO:tasks.workunit.client.0.smithi012.stdout:3/290: dwrite dc/f2b [0,4194304] 0 2024-03-18T14:50:54.252 INFO:tasks.workunit.client.0.smithi012.stdout:0/384: truncate d1/f9 2178001 0 2024-03-18T14:50:54.252 INFO:tasks.workunit.client.0.smithi012.stdout:0/385: fsync d1/d16/d31/f58 0 2024-03-18T14:50:54.253 INFO:tasks.workunit.client.0.smithi012.stdout:8/484: stat d0/d5a/d19/l56 0 2024-03-18T14:50:54.253 INFO:tasks.workunit.client.0.smithi012.stdout:8/485: chown d0/l2 1993754133 1 2024-03-18T14:50:54.256 INFO:tasks.workunit.client.0.smithi012.stdout:5/393: mkdir d3/d84 0 2024-03-18T14:50:54.256 INFO:tasks.workunit.client.0.smithi012.stdout:5/394: write d3/d15/d5f/f70 [1017180,33499] 0 2024-03-18T14:50:54.258 INFO:tasks.workunit.client.0.smithi012.stdout:3/291: read f3 [1976944,42188] 0 2024-03-18T14:50:54.259 INFO:tasks.workunit.client.0.smithi012.stdout:7/364: symlink dd/d13/d23/d76/d6b/l7d 0 2024-03-18T14:50:54.261 INFO:tasks.workunit.client.0.smithi012.stdout:6/332: symlink d17/l5e 0 2024-03-18T14:50:54.265 INFO:tasks.workunit.client.0.smithi012.stdout:9/391: dwrite d3/d9/f29 [0,4194304] 0 2024-03-18T14:50:54.266 INFO:tasks.workunit.client.0.smithi012.stdout:2/361: rmdir d0/d10/d18/d31/d4c 39 2024-03-18T14:50:54.266 INFO:tasks.workunit.client.0.smithi012.stdout:2/362: stat d0/c19 0 2024-03-18T14:50:54.268 INFO:tasks.workunit.client.0.smithi012.stdout:0/386: mknod d1/d20/d23/d3b/d33/d51/c86 0 2024-03-18T14:50:54.272 INFO:tasks.workunit.client.0.smithi012.stdout:8/486: creat d0/d47/f99 x:0 0 0 2024-03-18T14:50:54.272 INFO:tasks.workunit.client.0.smithi012.stdout:8/487: creat d0/d5a/f9a x:0 0 0 2024-03-18T14:50:54.272 INFO:tasks.workunit.client.0.smithi012.stdout:8/488: chown d0/d12/f20 890026089 1 2024-03-18T14:50:54.274 INFO:tasks.workunit.client.0.smithi012.stdout:7/365: read dd/d13/d26/f5e [2715391,52269] 0 2024-03-18T14:50:54.274 INFO:tasks.workunit.client.0.smithi012.stdout:5/395: creat d3/d84/f85 x:0 0 0 2024-03-18T14:50:54.275 INFO:tasks.workunit.client.0.smithi012.stdout:5/396: fsync d3/d15/d21/f27 0 2024-03-18T14:50:54.276 INFO:tasks.workunit.client.0.smithi012.stdout:3/292: creat dc/d3c/d59/f5a x:0 0 0 2024-03-18T14:50:54.279 INFO:tasks.workunit.client.0.smithi012.stdout:9/392: truncate d3/d9/f10 3753115 0 2024-03-18T14:50:54.281 INFO:tasks.workunit.client.0.smithi012.stdout:2/363: symlink d0/d6b/l70 0 2024-03-18T14:50:54.282 INFO:tasks.workunit.client.0.smithi012.stdout:6/333: dwrite f14 [4194304,4194304] 0 2024-03-18T14:50:54.282 INFO:tasks.workunit.client.0.smithi012.stdout:6/334: chown d17/d26/d36/d4d/c5d 29 1 2024-03-18T14:50:54.290 INFO:tasks.workunit.client.0.smithi012.stdout:8/489: link d0/d47/f4f d0/d15/d6e/d84/d54/f9b 0 2024-03-18T14:50:54.290 INFO:tasks.workunit.client.0.smithi012.stdout:8/490: fdatasync d0/d15/f38 0 2024-03-18T14:50:54.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/366: mkdir dd/d13/d23/d2f/d7a/d7e 0 2024-03-18T14:50:54.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/367: write dd/f5f [318756,54057] 0 2024-03-18T14:50:54.293 INFO:tasks.workunit.client.0.smithi012.stdout:7/368: write dd/d13/d23/d2f/f3d [1809843,10354] 0 2024-03-18T14:50:54.294 INFO:tasks.workunit.client.0.smithi012.stdout:5/397: mkdir d3/d29/d36/d3d/d86 0 2024-03-18T14:50:54.294 INFO:tasks.workunit.client.0.smithi012.stdout:5/398: fsync d3/d1e/f1a 0 2024-03-18T14:50:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:53 smithi100 ceph-mon[37807]: pgmap v244: 65 pgs: 65 active+clean; 873 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 74 MiB/s rd, 100 MiB/s wr, 529 op/s 2024-03-18T14:50:54.297 INFO:tasks.workunit.client.0.smithi012.stdout:2/364: dwrite d0/d3c/d40/d47/f49 [0,4194304] 0 2024-03-18T14:50:54.299 INFO:tasks.workunit.client.0.smithi012.stdout:5/399: read d3/d29/d35/f5a [2414151,69436] 0 2024-03-18T14:50:54.301 INFO:tasks.workunit.client.1.smithi100.stdout:9/6: unlink f0 0 2024-03-18T14:50:54.301 INFO:tasks.workunit.client.1.smithi100.stdout:9/7: dwrite - no filename 2024-03-18T14:50:54.301 INFO:tasks.workunit.client.1.smithi100.stdout:9/8: readlink - no filename 2024-03-18T14:50:54.301 INFO:tasks.workunit.client.1.smithi100.stdout:9/9: rmdir - no directory 2024-03-18T14:50:54.305 INFO:tasks.workunit.client.0.smithi012.stdout:5/400: dread d3/d15/f1c [4194304,4194304] 0 2024-03-18T14:50:54.305 INFO:tasks.workunit.client.0.smithi012.stdout:5/401: fdatasync d3/d29/d36/d3d/f6e 0 2024-03-18T14:50:54.307 INFO:tasks.workunit.client.0.smithi012.stdout:3/293: write dc/f26 [2046229,37419] 0 2024-03-18T14:50:54.311 INFO:tasks.workunit.client.0.smithi012.stdout:7/369: rmdir dd/d13/d23/d2f/d7a/d7e 0 2024-03-18T14:50:54.316 INFO:tasks.workunit.client.0.smithi012.stdout:2/365: mkdir d0/d10/d18/d33/d71 0 2024-03-18T14:50:54.316 INFO:tasks.workunit.client.0.smithi012.stdout:2/366: fdatasync d0/d10/d18/f3a 0 2024-03-18T14:50:54.316 INFO:tasks.workunit.client.0.smithi012.stdout:2/367: truncate d0/d10/d18/d31/d4c/f51 363170 0 2024-03-18T14:50:54.320 INFO:tasks.workunit.client.0.smithi012.stdout:9/393: mknod d3/c7c 0 2024-03-18T14:50:54.320 INFO:tasks.workunit.client.0.smithi012.stdout:9/394: truncate d3/d9/f17 414278 0 2024-03-18T14:50:54.320 INFO:tasks.workunit.client.0.smithi012.stdout:6/335: dwrite f4 [4194304,4194304] 0 2024-03-18T14:50:54.321 INFO:tasks.workunit.client.1.smithi100.stdout:7/19: rename d5/f6 to d5/fb 0 2024-03-18T14:50:54.322 INFO:tasks.workunit.client.0.smithi012.stdout:5/402: creat d3/d1e/f87 x:0 0 0 2024-03-18T14:50:54.326 INFO:tasks.workunit.client.0.smithi012.stdout:3/294: symlink dc/d30/l5b 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:9/10: symlink l1 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:9/11: write - no filename 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:9/12: creat f2 x:0 0 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/20: symlink d5/lc 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/21: fsync f4 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/22: creat d5/fd x:0 0 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/23: fdatasync d5/fd 0 2024-03-18T14:50:54.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/24: dread - d5/fd zero size 2024-03-18T14:50:54.328 INFO:tasks.workunit.client.0.smithi012.stdout:7/370: mknod dd/d13/d23/d33/d37/c7f 0 2024-03-18T14:50:54.328 INFO:tasks.workunit.client.1.smithi100.stdout:9/13: rename l1 to l3 0 2024-03-18T14:50:54.328 INFO:tasks.workunit.client.1.smithi100.stdout:9/14: creat f4 x:0 0 0 2024-03-18T14:50:54.330 INFO:tasks.workunit.client.0.smithi012.stdout:9/395: link d3/d9/d19/d66/d21/c27 d3/d9/d5c/c7d 0 2024-03-18T14:50:54.331 INFO:tasks.workunit.client.1.smithi100.stdout:7/25: mknod d5/ce 0 2024-03-18T14:50:54.332 INFO:tasks.workunit.client.0.smithi012.stdout:6/336: symlink d17/d26/d36/l5f 0 2024-03-18T14:50:54.336 INFO:tasks.workunit.client.0.smithi012.stdout:5/403: mknod d3/d15/d5f/c88 0 2024-03-18T14:50:54.337 INFO:tasks.workunit.client.0.smithi012.stdout:3/295: mknod dc/d30/d35/c5c 0 2024-03-18T14:50:54.337 INFO:tasks.workunit.client.0.smithi012.stdout:3/296: creat dc/d3c/d4c/d50/f5d x:0 0 0 2024-03-18T14:50:54.338 INFO:tasks.workunit.client.0.smithi012.stdout:7/371: mknod dd/d13/d23/d2f/d7a/c80 0 2024-03-18T14:50:54.339 INFO:tasks.workunit.client.0.smithi012.stdout:9/396: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:54.339 INFO:tasks.workunit.client.0.smithi012.stdout:9/397: readlink d3/le 0 2024-03-18T14:50:54.341 INFO:tasks.workunit.client.0.smithi012.stdout:7/372: dread dd/f45 [0,4194304] 0 2024-03-18T14:50:54.341 INFO:tasks.workunit.client.0.smithi012.stdout:2/368: link d0/d3c/c50 d0/d10/c72 0 2024-03-18T14:50:54.342 INFO:tasks.workunit.client.0.smithi012.stdout:2/369: creat d0/d10/d18/f73 x:0 0 0 2024-03-18T14:50:54.344 INFO:tasks.workunit.client.0.smithi012.stdout:9/398: write d3/d9/d19/d66/d21/d35/d48/f67 [3002629,109480] 0 2024-03-18T14:50:54.344 INFO:tasks.workunit.client.0.smithi012.stdout:9/399: fsync d3/d20/f31 0 2024-03-18T14:50:54.346 INFO:tasks.workunit.client.0.smithi012.stdout:3/297: creat dc/d30/d3a/f5e x:0 0 0 2024-03-18T14:50:54.348 INFO:tasks.workunit.client.0.smithi012.stdout:7/373: mknod dd/d13/d26/c81 0 2024-03-18T14:50:54.349 INFO:tasks.workunit.client.0.smithi012.stdout:7/374: dread dd/d13/d23/f55 [0,4194304] 0 2024-03-18T14:50:54.354 INFO:tasks.workunit.client.0.smithi012.stdout:9/400: truncate d3/d20/f5f 3932403 0 2024-03-18T14:50:54.356 INFO:tasks.workunit.client.0.smithi012.stdout:5/404: rename d3/d15/d21 to d3/d1e/d33/d89 0 2024-03-18T14:50:54.358 INFO:tasks.workunit.client.0.smithi012.stdout:7/375: rename dd/c25 to dd/d13/d23/d2f/d7a/c82 0 2024-03-18T14:50:54.360 INFO:tasks.workunit.client.0.smithi012.stdout:9/401: rename d3/f25 to d3/d9/d19/d77/d52/d78/f7e 0 2024-03-18T14:50:54.360 INFO:tasks.workunit.client.0.smithi012.stdout:9/402: read d3/f6 [1508736,10579] 0 2024-03-18T14:50:54.361 INFO:tasks.workunit.client.0.smithi012.stdout:5/405: dread d3/d29/d36/d3d/d73/f76 [4194304,4194304] 0 2024-03-18T14:50:54.362 INFO:tasks.workunit.client.0.smithi012.stdout:7/376: mknod dd/d13/d23/c83 0 2024-03-18T14:50:54.363 INFO:tasks.workunit.client.0.smithi012.stdout:9/403: mkdir d3/d9/d19/d66/d69/d7f 0 2024-03-18T14:50:54.363 INFO:tasks.workunit.client.0.smithi012.stdout:9/404: fdatasync d3/f63 0 2024-03-18T14:50:54.364 INFO:tasks.workunit.client.0.smithi012.stdout:5/406: truncate d3/f3a 2616807 0 2024-03-18T14:50:54.364 INFO:tasks.workunit.client.0.smithi012.stdout:5/407: fdatasync d3/d29/d36/d3d/f7f 0 2024-03-18T14:50:54.366 INFO:tasks.workunit.client.0.smithi012.stdout:7/377: rename dd/d13/d23/d2f/f65 to dd/d13/d23/d33/f84 0 2024-03-18T14:50:54.366 INFO:tasks.workunit.client.0.smithi012.stdout:7/378: readlink dd/d13/l4b 0 2024-03-18T14:50:54.366 INFO:tasks.workunit.client.0.smithi012.stdout:4/379: sync 2024-03-18T14:50:54.366 INFO:tasks.workunit.client.0.smithi012.stdout:1/331: sync 2024-03-18T14:50:54.366 INFO:tasks.workunit.client.0.smithi012.stdout:7/379: write dd/d13/f15 [4550428,121294] 0 2024-03-18T14:50:54.367 INFO:tasks.workunit.client.0.smithi012.stdout:2/370: dwrite d0/d10/d18/d31/d4c/d59/f69 [4194304,4194304] 0 2024-03-18T14:50:54.368 INFO:tasks.workunit.client.0.smithi012.stdout:9/405: symlink d3/d9/d19/d66/d21/d35/d48/l80 0 2024-03-18T14:50:54.370 INFO:tasks.workunit.client.0.smithi012.stdout:5/408: truncate d3/d29/d36/d3d/d73/f76 305314 0 2024-03-18T14:50:54.371 INFO:tasks.workunit.client.0.smithi012.stdout:4/380: unlink d6/d1a/d30/f49 0 2024-03-18T14:50:54.372 INFO:tasks.workunit.client.0.smithi012.stdout:4/381: chown d6/da 4 1 2024-03-18T14:50:54.372 INFO:tasks.workunit.client.0.smithi012.stdout:4/382: creat d6/da/f82 x:0 0 0 2024-03-18T14:50:54.373 INFO:tasks.workunit.client.0.smithi012.stdout:1/332: truncate d8/d1e/f2f 1325259 0 2024-03-18T14:50:54.374 INFO:tasks.workunit.client.0.smithi012.stdout:2/371: symlink d0/d10/d18/d57/l74 0 2024-03-18T14:50:54.375 INFO:tasks.workunit.client.0.smithi012.stdout:6/337: dwrite d17/f1b [0,4194304] 0 2024-03-18T14:50:54.376 INFO:tasks.workunit.client.0.smithi012.stdout:9/406: getdents d3/d9/d19/d66/d6e 0 2024-03-18T14:50:54.376 INFO:tasks.workunit.client.0.smithi012.stdout:9/407: stat d3/d9/d5c/c7b 0 2024-03-18T14:50:54.376 INFO:tasks.workunit.client.0.smithi012.stdout:9/408: readlink d3/d9/d19/d66/l70 0 2024-03-18T14:50:54.377 INFO:tasks.workunit.client.0.smithi012.stdout:7/380: dread dd/d13/f72 [0,4194304] 0 2024-03-18T14:50:54.377 INFO:tasks.workunit.client.0.smithi012.stdout:5/409: mknod d3/d1e/d33/d89/c8a 0 2024-03-18T14:50:54.378 INFO:tasks.workunit.client.0.smithi012.stdout:2/372: dread d0/d10/d18/d31/d4c/f5c [0,4194304] 0 2024-03-18T14:50:54.379 INFO:tasks.workunit.client.0.smithi012.stdout:1/333: getdents d8/db/d10/d16/d1c 0 2024-03-18T14:50:54.379 INFO:tasks.workunit.client.0.smithi012.stdout:1/334: chown d8 238959 1 2024-03-18T14:50:54.382 INFO:tasks.workunit.client.1.smithi100.stdout:7/26: dwrite d5/fb [0,4194304] 0 2024-03-18T14:50:54.382 INFO:tasks.workunit.client.1.smithi100.stdout:7/27: creat d5/ff x:0 0 0 2024-03-18T14:50:54.384 INFO:tasks.workunit.client.0.smithi012.stdout:6/338: truncate d17/d18/f23 75030 0 2024-03-18T14:50:54.389 INFO:tasks.workunit.client.0.smithi012.stdout:9/409: rmdir d3/d9/d19/d77/d52/d65/d74 0 2024-03-18T14:50:54.392 INFO:tasks.workunit.client.1.smithi100.stdout:7/28: rename f4 to d5/f10 0 2024-03-18T14:50:54.396 INFO:tasks.workunit.client.1.smithi100.stdout:7/29: mknod d5/c11 0 2024-03-18T14:50:54.396 INFO:tasks.workunit.client.1.smithi100.stdout:7/30: chown d5/f10 1 1 2024-03-18T14:50:54.398 INFO:tasks.workunit.client.1.smithi100.stdout:7/31: mknod d5/c12 0 2024-03-18T14:50:54.398 INFO:tasks.workunit.client.1.smithi100.stdout:7/32: write d5/f10 [420079,15525] 0 2024-03-18T14:50:54.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/373: mknod d0/d10/d18/d57/d61/c75 0 2024-03-18T14:50:54.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/374: fsync d0/f21 0 2024-03-18T14:50:54.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/375: write d0/d10/f58 [55634,24837] 0 2024-03-18T14:50:54.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/376: fsync d0/d10/d18/f5b 0 2024-03-18T14:50:54.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/377: truncate d0/d10/d18/f5b 196337 0 2024-03-18T14:50:54.402 INFO:tasks.workunit.client.0.smithi012.stdout:6/339: mknod d17/d19/d27/d58/c60 0 2024-03-18T14:50:54.402 INFO:tasks.workunit.client.0.smithi012.stdout:6/340: truncate d17/f2c 4935330 0 2024-03-18T14:50:54.403 INFO:tasks.workunit.client.0.smithi012.stdout:7/381: dwrite dd/d13/d23/d33/f84 [0,4194304] 0 2024-03-18T14:50:54.403 INFO:tasks.workunit.client.0.smithi012.stdout:7/382: chown dd/d4f/l74 169427 1 2024-03-18T14:50:54.403 INFO:tasks.workunit.client.0.smithi012.stdout:9/410: symlink d3/d9/d19/d66/d6e/l81 0 2024-03-18T14:50:54.403 INFO:tasks.workunit.client.0.smithi012.stdout:2/378: rmdir d0/d10/d18/d57 39 2024-03-18T14:50:54.404 INFO:tasks.workunit.client.0.smithi012.stdout:1/335: dwrite d8/db/d10/d16/d54/f5c [0,4194304] 0 2024-03-18T14:50:54.405 INFO:tasks.workunit.client.0.smithi012.stdout:6/341: symlink d17/d38/d3f/l61 0 2024-03-18T14:50:54.406 INFO:tasks.workunit.client.0.smithi012.stdout:7/383: mknod dd/d4f/c85 0 2024-03-18T14:50:54.407 INFO:tasks.workunit.client.0.smithi012.stdout:9/411: creat d3/d9/d19/d66/d69/d7f/f82 x:0 0 0 2024-03-18T14:50:54.407 INFO:tasks.workunit.client.0.smithi012.stdout:6/342: dread d17/d33/f48 [0,4194304] 0 2024-03-18T14:50:54.408 INFO:tasks.workunit.client.0.smithi012.stdout:6/343: creat d17/d19/d27/d2f/f62 x:0 0 0 2024-03-18T14:50:54.408 INFO:tasks.workunit.client.0.smithi012.stdout:2/379: symlink d0/d6c/l76 0 2024-03-18T14:50:54.408 INFO:tasks.workunit.client.0.smithi012.stdout:2/380: write d0/f21 [1965613,12962] 0 2024-03-18T14:50:54.412 INFO:tasks.workunit.client.0.smithi012.stdout:1/336: rename d8/db/d10/l29 to d8/db/d10/d16/d1c/d2e/l6a 0 2024-03-18T14:50:54.412 INFO:tasks.workunit.client.0.smithi012.stdout:7/384: dread dd/d13/d26/f5e [0,4194304] 0 2024-03-18T14:50:54.412 INFO:tasks.workunit.client.0.smithi012.stdout:7/385: truncate dd/d13/d23/f2b 4611330 0 2024-03-18T14:50:54.413 INFO:tasks.workunit.client.0.smithi012.stdout:3/298: rmdir dc/d3c/d59 39 2024-03-18T14:50:54.413 INFO:tasks.workunit.client.0.smithi012.stdout:3/299: write dc/f11 [1232615,60544] 0 2024-03-18T14:50:54.413 INFO:tasks.workunit.client.0.smithi012.stdout:3/300: write dc/f32 [4637010,36963] 0 2024-03-18T14:50:54.414 INFO:tasks.workunit.client.0.smithi012.stdout:6/344: mkdir d17/d38/d51/d63 0 2024-03-18T14:50:54.414 INFO:tasks.workunit.client.0.smithi012.stdout:6/345: creat d17/d33/f64 x:0 0 0 2024-03-18T14:50:54.414 INFO:tasks.workunit.client.0.smithi012.stdout:6/346: dread - d17/d19/f3c zero size 2024-03-18T14:50:54.415 INFO:tasks.workunit.client.0.smithi012.stdout:2/381: truncate d0/d10/d18/d57/f64 475187 0 2024-03-18T14:50:54.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/337: mkdir d8/db/d10/d16/d6b 0 2024-03-18T14:50:54.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/338: fsync d8/d1e/d27/f3c 0 2024-03-18T14:50:54.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/339: fdatasync d8/d1e/d27/d44/f4f 0 2024-03-18T14:50:54.420 INFO:tasks.workunit.client.0.smithi012.stdout:7/386: dread dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:54.420 INFO:tasks.workunit.client.0.smithi012.stdout:7/387: fdatasync dd/d13/d23/d76/f68 0 2024-03-18T14:50:54.426 INFO:tasks.workunit.client.0.smithi012.stdout:2/382: mkdir d0/d10/d18/d33/d77 0 2024-03-18T14:50:54.429 INFO:tasks.workunit.client.0.smithi012.stdout:1/340: mkdir d8/db/d10/d56/d6c 0 2024-03-18T14:50:54.429 INFO:tasks.workunit.client.0.smithi012.stdout:1/341: stat d8/db/d10/d16/d1c/d2e 0 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:9/412: rename d3/d9/f10 to d3/f83 0 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:9/413: chown d3/d9/f6c 97 1 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:6/347: dwrite d17/d19/f2b [0,4194304] 0 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:6/348: stat d17/d19/f3e 0 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:6/349: truncate d17/d38/d3f/f55 456454 0 2024-03-18T14:50:54.431 INFO:tasks.workunit.client.0.smithi012.stdout:1/342: stat d8/db/c12 0 2024-03-18T14:50:54.432 INFO:tasks.workunit.client.0.smithi012.stdout:2/383: fdatasync d0/f21 0 2024-03-18T14:50:54.432 INFO:tasks.workunit.client.0.smithi012.stdout:9/414: symlink d3/d9/d19/d77/d52/l84 0 2024-03-18T14:50:54.432 INFO:tasks.workunit.client.0.smithi012.stdout:9/415: fsync d3/d9/d19/f6a 0 2024-03-18T14:50:54.436 INFO:tasks.workunit.client.0.smithi012.stdout:6/350: rename d17/d38/d51/c57 to d17/d18/c65 0 2024-03-18T14:50:54.438 INFO:tasks.workunit.client.0.smithi012.stdout:6/351: mkdir d17/d38/d51/d66 0 2024-03-18T14:50:54.442 INFO:tasks.workunit.client.0.smithi012.stdout:9/416: read d3/d9/d19/d77/d52/d78/f7e [7087103,35547] 0 2024-03-18T14:50:54.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/417: creat d3/d9/d5c/f85 x:0 0 0 2024-03-18T14:50:54.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/418: fdatasync d3/f83 0 2024-03-18T14:50:54.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/419: chown d3/d9/d19/d66/d6e/l73 892771 1 2024-03-18T14:50:54.445 INFO:tasks.workunit.client.1.smithi100.stdout:7/33: rename d5/fb to d5/f13 0 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:2/6: sync 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:1/15: sync 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:8/10: sync 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:1/16: chown d0 11641634 1 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:2/7: chown f0 27 1 2024-03-18T14:50:54.447 INFO:tasks.workunit.client.1.smithi100.stdout:4/0: sync 2024-03-18T14:50:54.448 INFO:tasks.workunit.client.0.smithi012.stdout:9/420: creat d3/d9/d19/d77/d52/d65/f86 x:0 0 0 2024-03-18T14:50:54.448 INFO:tasks.workunit.client.0.smithi012.stdout:9/421: fdatasync d3/d9/d19/d66/f44 0 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:8/11: mknod c2 0 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:8/12: creat f3 x:0 0 0 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:6/12: sync 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:3/11: sync 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:3/12: write - no filename 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:3/13: dread - no filename 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:0/33: sync 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:5/7: sync 2024-03-18T14:50:54.451 INFO:tasks.workunit.client.1.smithi100.stdout:5/8: chown f2 4052 1 2024-03-18T14:50:54.452 INFO:tasks.workunit.client.1.smithi100.stdout:5/9: creat f4 x:0 0 0 2024-03-18T14:50:54.452 INFO:tasks.workunit.client.1.smithi100.stdout:5/10: write f2 [135697,86610] 0 2024-03-18T14:50:54.452 INFO:tasks.workunit.client.1.smithi100.stdout:2/8: symlink l1 0 2024-03-18T14:50:54.452 INFO:tasks.workunit.client.0.smithi012.stdout:7/388: dwrite dd/d13/d26/f5e [0,4194304] 0 2024-03-18T14:50:54.453 INFO:tasks.workunit.client.1.smithi100.stdout:4/1: creat f0 x:0 0 0 2024-03-18T14:50:54.453 INFO:tasks.workunit.client.1.smithi100.stdout:4/2: write f0 [781734,21576] 0 2024-03-18T14:50:54.457 INFO:tasks.workunit.client.1.smithi100.stdout:8/13: mkdir d4 0 2024-03-18T14:50:54.458 INFO:tasks.workunit.client.1.smithi100.stdout:3/14: creat f1 x:0 0 0 2024-03-18T14:50:54.458 INFO:tasks.workunit.client.1.smithi100.stdout:3/15: creat f2 x:0 0 0 2024-03-18T14:50:54.458 INFO:tasks.workunit.client.1.smithi100.stdout:3/16: chown f2 13986 1 2024-03-18T14:50:54.459 INFO:tasks.workunit.client.0.smithi012.stdout:7/389: dread dd/d13/d23/d2f/f4c [0,4194304] 0 2024-03-18T14:50:54.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/11: stat l3 0 2024-03-18T14:50:54.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/12: write f4 [57511,2746] 0 2024-03-18T14:50:54.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/13: chown f4 0 1 2024-03-18T14:50:54.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/14: chown l3 11384 1 2024-03-18T14:50:54.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/15: stat f0 0 2024-03-18T14:50:54.461 INFO:tasks.workunit.client.1.smithi100.stdout:5/16: write f4 [653846,78915] 0 2024-03-18T14:50:54.461 INFO:tasks.workunit.client.1.smithi100.stdout:5/17: creat f5 x:0 0 0 2024-03-18T14:50:54.461 INFO:tasks.workunit.client.0.smithi012.stdout:7/390: symlink dd/l86 0 2024-03-18T14:50:54.461 INFO:tasks.workunit.client.1.smithi100.stdout:2/9: truncate f0 413140 0 2024-03-18T14:50:54.462 INFO:tasks.workunit.client.0.smithi012.stdout:3/301: getdents dc/d30/d35 0 2024-03-18T14:50:54.463 INFO:tasks.workunit.client.1.smithi100.stdout:0/34: dread f2 [0,4194304] 0 2024-03-18T14:50:54.464 INFO:tasks.workunit.client.1.smithi100.stdout:1/17: rmdir d0 0 2024-03-18T14:50:54.464 INFO:tasks.workunit.client.1.smithi100.stdout:1/18: fdatasync - no filename 2024-03-18T14:50:54.464 INFO:tasks.workunit.client.1.smithi100.stdout:1/19: write - no filename 2024-03-18T14:50:54.466 INFO:tasks.workunit.client.1.smithi100.stdout:8/14: symlink d4/l5 0 2024-03-18T14:50:54.466 INFO:tasks.workunit.client.1.smithi100.stdout:8/15: chown d4 853940413 1 2024-03-18T14:50:54.466 INFO:tasks.workunit.client.1.smithi100.stdout:8/16: readlink d4/l5 0 2024-03-18T14:50:54.466 INFO:tasks.workunit.client.1.smithi100.stdout:8/17: stat d4/l5 0 2024-03-18T14:50:54.467 INFO:tasks.workunit.client.1.smithi100.stdout:8/18: readlink d4/l5 0 2024-03-18T14:50:54.469 INFO:tasks.workunit.client.0.smithi012.stdout:2/384: dwrite d0/d3c/d40/f5a [0,4194304] 0 2024-03-18T14:50:54.470 INFO:tasks.workunit.client.1.smithi100.stdout:2/10: mknod c2 0 2024-03-18T14:50:54.471 INFO:tasks.workunit.client.1.smithi100.stdout:1/20: creat f2 x:0 0 0 2024-03-18T14:50:54.474 INFO:tasks.workunit.client.0.smithi012.stdout:2/385: link d0/d6c/l76 d0/d10/d18/d31/l78 0 2024-03-18T14:50:54.475 INFO:tasks.workunit.client.0.smithi012.stdout:3/302: write dc/d30/f37 [2450830,56810] 0 2024-03-18T14:50:54.475 INFO:tasks.workunit.client.0.smithi012.stdout:3/303: chown dc/d30/d35/l34 0 1 2024-03-18T14:50:54.477 INFO:tasks.workunit.client.1.smithi100.stdout:8/19: link f3 d4/f6 0 2024-03-18T14:50:54.478 INFO:tasks.workunit.client.1.smithi100.stdout:4/3: write f0 [218007,33661] 0 2024-03-18T14:50:54.479 INFO:tasks.workunit.client.1.smithi100.stdout:0/35: dread f3 [0,4194304] 0 2024-03-18T14:50:54.480 INFO:tasks.workunit.client.0.smithi012.stdout:3/304: unlink dc/d3c/c41 0 2024-03-18T14:50:54.485 INFO:tasks.workunit.client.1.smithi100.stdout:0/36: mkdir dd/d12 0 2024-03-18T14:50:54.490 INFO:tasks.workunit.client.0.smithi012.stdout:6/352: dwrite d17/d33/f48 [0,4194304] 0 2024-03-18T14:50:54.490 INFO:tasks.workunit.client.0.smithi012.stdout:6/353: truncate d17/d38/d3f/f55 875053 0 2024-03-18T14:50:54.491 INFO:tasks.workunit.client.0.smithi012.stdout:6/354: dread - d17/d38/d51/f53 zero size 2024-03-18T14:50:54.491 INFO:tasks.workunit.client.1.smithi100.stdout:0/37: unlink c8 0 2024-03-18T14:50:54.495 INFO:tasks.workunit.client.1.smithi100.stdout:8/20: dread f1 [0,4194304] 0 2024-03-18T14:50:54.495 INFO:tasks.workunit.client.1.smithi100.stdout:8/21: dread - f3 zero size 2024-03-18T14:50:54.497 INFO:tasks.workunit.client.1.smithi100.stdout:0/38: rmdir dd/d12 0 2024-03-18T14:50:54.497 INFO:tasks.workunit.client.1.smithi100.stdout:7/34: dwrite d5/f10 [0,4194304] 0 2024-03-18T14:50:54.499 INFO:tasks.workunit.client.1.smithi100.stdout:3/17: dwrite f1 [0,4194304] 0 2024-03-18T14:50:54.499 INFO:tasks.workunit.client.1.smithi100.stdout:3/18: stat f1 0 2024-03-18T14:50:54.499 INFO:tasks.workunit.client.1.smithi100.stdout:3/19: read - f2 zero size 2024-03-18T14:50:54.499 INFO:tasks.workunit.client.1.smithi100.stdout:3/20: fdatasync f2 0 2024-03-18T14:50:54.499 INFO:tasks.workunit.client.1.smithi100.stdout:3/21: write f2 [835512,22322] 0 2024-03-18T14:50:54.500 INFO:tasks.workunit.client.1.smithi100.stdout:6/13: dwrite f1 [0,4194304] 0 2024-03-18T14:50:54.500 INFO:tasks.workunit.client.1.smithi100.stdout:6/14: getdents . 0 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.1.smithi100.stdout:3/22: write f2 [381900,93533] 0 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/391: dwrite dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/392: read dd/d13/d23/d76/f3e [3745317,56325] 0 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/393: chown dd/d13/d23/d76/f3e 99449246 1 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/394: fsync dd/d13/f72 0 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/395: chown dd/d13/f73 240437411 1 2024-03-18T14:50:54.501 INFO:tasks.workunit.client.1.smithi100.stdout:8/22: rmdir d4 39 2024-03-18T14:50:54.502 INFO:tasks.workunit.client.1.smithi100.stdout:8/23: write f3 [22236,88682] 0 2024-03-18T14:50:54.502 INFO:tasks.workunit.client.1.smithi100.stdout:0/39: mknod dd/c13 0 2024-03-18T14:50:54.502 INFO:tasks.workunit.client.1.smithi100.stdout:7/35: mkdir d5/d14 0 2024-03-18T14:50:54.506 INFO:tasks.workunit.client.0.smithi012.stdout:7/396: rename dd/c20 to dd/d13/d23/d2f/d53/c87 0 2024-03-18T14:50:54.506 INFO:tasks.workunit.client.0.smithi012.stdout:7/397: truncate dd/d13/d23/d33/f78 973401 0 2024-03-18T14:50:54.510 INFO:tasks.workunit.client.0.smithi012.stdout:7/398: link dd/d13/c1a dd/d13/d23/d2f/d53/c88 0 2024-03-18T14:50:54.512 INFO:tasks.workunit.client.1.smithi100.stdout:6/15: dread f1 [0,4194304] 0 2024-03-18T14:50:54.512 INFO:tasks.workunit.client.1.smithi100.stdout:6/16: chown c2 0 1 2024-03-18T14:50:54.512 INFO:tasks.workunit.client.1.smithi100.stdout:1/21: dwrite f2 [0,4194304] 0 2024-03-18T14:50:54.513 INFO:tasks.workunit.client.1.smithi100.stdout:3/23: dread f1 [0,4194304] 0 2024-03-18T14:50:54.514 INFO:tasks.workunit.client.1.smithi100.stdout:8/24: symlink d4/l7 0 2024-03-18T14:50:54.516 INFO:tasks.workunit.client.0.smithi012.stdout:5/410: dread d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:54.517 INFO:tasks.workunit.client.0.smithi012.stdout:7/399: rename dd/f59 to dd/d13/d23/d60/f89 0 2024-03-18T14:50:54.518 INFO:tasks.workunit.client.1.smithi100.stdout:0/40: mknod dd/c14 0 2024-03-18T14:50:54.518 INFO:tasks.workunit.client.1.smithi100.stdout:0/41: write dd/f11 [44431,13779] 0 2024-03-18T14:50:54.518 INFO:tasks.workunit.client.0.smithi012.stdout:5/411: dread d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:54.520 INFO:tasks.workunit.client.1.smithi100.stdout:7/36: mkdir d5/d14/d15 0 2024-03-18T14:50:54.521 INFO:tasks.workunit.client.0.smithi012.stdout:7/400: symlink dd/d13/l8a 0 2024-03-18T14:50:54.522 INFO:tasks.workunit.client.0.smithi012.stdout:5/412: getdents d3/d29/d36 0 2024-03-18T14:50:54.522 INFO:tasks.workunit.client.0.smithi012.stdout:5/413: chown d3/d1e/l60 8 1 2024-03-18T14:50:54.526 INFO:tasks.workunit.client.1.smithi100.stdout:5/18: dwrite f4 [0,4194304] 0 2024-03-18T14:50:54.526 INFO:tasks.workunit.client.1.smithi100.stdout:5/19: write f0 [627237,67047] 0 2024-03-18T14:50:54.527 INFO:tasks.workunit.client.1.smithi100.stdout:6/17: link c0 c3 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:3/305: dwrite dc/d30/d35/f2c [0,4194304] 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:3/306: chown dc/f26 1328083 1 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:3/307: write dc/f26 [3876488,110685] 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:3/308: readlink dc/d3c/d4c/d50/d54/l56 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:3/309: truncate dc/d30/d35/f2c 4414833 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:6/355: dwrite d17/d33/f48 [0,4194304] 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:2/386: dwrite d0/d10/f58 [0,4194304] 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:2/387: creat d0/f79 x:0 0 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:5/414: stat d3/d1e/d33/f4b 0 2024-03-18T14:50:54.528 INFO:tasks.workunit.client.0.smithi012.stdout:5/415: write d3/d1e/f28 [2197146,42434] 0 2024-03-18T14:50:54.529 INFO:tasks.workunit.client.1.smithi100.stdout:5/20: fsync f2 0 2024-03-18T14:50:54.529 INFO:tasks.workunit.client.1.smithi100.stdout:5/21: fdatasync f5 0 2024-03-18T14:50:54.530 INFO:tasks.workunit.client.1.smithi100.stdout:1/22: symlink l3 0 2024-03-18T14:50:54.530 INFO:tasks.workunit.client.1.smithi100.stdout:1/23: creat f4 x:0 0 0 2024-03-18T14:50:54.530 INFO:tasks.workunit.client.1.smithi100.stdout:1/24: dread - f4 zero size 2024-03-18T14:50:54.530 INFO:tasks.workunit.client.1.smithi100.stdout:1/25: creat f5 x:0 0 0 2024-03-18T14:50:54.530 INFO:tasks.workunit.client.1.smithi100.stdout:1/26: readlink l3 0 2024-03-18T14:50:54.531 INFO:tasks.workunit.client.1.smithi100.stdout:1/27: chown f5 106931888 1 2024-03-18T14:50:54.531 INFO:tasks.workunit.client.1.smithi100.stdout:1/28: chown f5 246 1 2024-03-18T14:50:54.531 INFO:tasks.workunit.client.1.smithi100.stdout:1/29: rmdir - no directory 2024-03-18T14:50:54.531 INFO:tasks.workunit.client.1.smithi100.stdout:1/30: fdatasync f5 0 2024-03-18T14:50:54.532 INFO:tasks.workunit.client.1.smithi100.stdout:8/25: symlink d4/l8 0 2024-03-18T14:50:54.533 INFO:tasks.workunit.client.0.smithi012.stdout:5/416: dread d3/d15/d5f/f70 [0,4194304] 0 2024-03-18T14:50:54.535 INFO:tasks.workunit.client.1.smithi100.stdout:6/18: dread f1 [0,4194304] 0 2024-03-18T14:50:54.537 INFO:tasks.workunit.client.1.smithi100.stdout:7/37: rmdir d5/d14/d15 0 2024-03-18T14:50:54.541 INFO:tasks.workunit.client.0.smithi012.stdout:3/310: truncate f3 3458173 0 2024-03-18T14:50:54.543 INFO:tasks.workunit.client.1.smithi100.stdout:1/31: dread f2 [0,4194304] 0 2024-03-18T14:50:54.543 INFO:tasks.workunit.client.1.smithi100.stdout:1/32: read - f4 zero size 2024-03-18T14:50:54.544 INFO:tasks.workunit.client.1.smithi100.stdout:5/22: mkdir d6 0 2024-03-18T14:50:54.551 INFO:tasks.workunit.client.1.smithi100.stdout:7/38: write d5/f13 [286969,1025] 0 2024-03-18T14:50:54.555 INFO:tasks.workunit.client.1.smithi100.stdout:3/24: dwrite f2 [0,4194304] 0 2024-03-18T14:50:54.555 INFO:tasks.workunit.client.1.smithi100.stdout:3/25: readlink - no filename 2024-03-18T14:50:54.556 INFO:tasks.workunit.client.0.smithi012.stdout:7/401: dwrite dd/d13/d23/d60/f62 [0,4194304] 0 2024-03-18T14:50:54.556 INFO:tasks.workunit.client.1.smithi100.stdout:0/42: dwrite dd/f11 [0,4194304] 0 2024-03-18T14:50:54.557 INFO:tasks.workunit.client.1.smithi100.stdout:1/33: symlink l6 0 2024-03-18T14:50:54.561 INFO:tasks.workunit.client.1.smithi100.stdout:5/23: rmdir d6 0 2024-03-18T14:50:54.562 INFO:tasks.workunit.client.1.smithi100.stdout:5/24: dread - f5 zero size 2024-03-18T14:50:54.562 INFO:tasks.workunit.client.1.smithi100.stdout:5/25: write f5 [281141,16838] 0 2024-03-18T14:50:54.562 INFO:tasks.workunit.client.1.smithi100.stdout:5/26: stat f4 0 2024-03-18T14:50:54.562 INFO:tasks.workunit.client.1.smithi100.stdout:5/27: chown f5 9 1 2024-03-18T14:50:54.564 INFO:tasks.workunit.client.0.smithi012.stdout:7/402: read dd/d13/f72 [1574283,91329] 0 2024-03-18T14:50:54.565 INFO:tasks.workunit.client.0.smithi012.stdout:1/343: getdents d8/db/d10/d56 0 2024-03-18T14:50:54.567 INFO:tasks.workunit.client.1.smithi100.stdout:5/28: dread f5 [0,4194304] 0 2024-03-18T14:50:54.567 INFO:tasks.workunit.client.0.smithi012.stdout:7/403: dread dd/f5b [0,4194304] 0 2024-03-18T14:50:54.568 INFO:tasks.workunit.client.1.smithi100.stdout:7/39: mkdir d5/d14/d16 0 2024-03-18T14:50:54.569 INFO:tasks.workunit.client.0.smithi012.stdout:7/404: dread dd/d4f/f69 [0,4194304] 0 2024-03-18T14:50:54.571 INFO:tasks.workunit.client.0.smithi012.stdout:6/356: write f16 [3620103,5595] 0 2024-03-18T14:50:54.572 INFO:tasks.workunit.client.1.smithi100.stdout:5/29: dread f0 [0,4194304] 0 2024-03-18T14:50:54.573 INFO:tasks.workunit.client.0.smithi012.stdout:6/357: write d17/f44 [218957,6704] 0 2024-03-18T14:50:54.577 INFO:tasks.workunit.client.0.smithi012.stdout:2/388: rename d0/d3c/c68 to d0/d10/d18/d31/c7a 0 2024-03-18T14:50:54.579 INFO:tasks.workunit.client.1.smithi100.stdout:0/43: mknod dd/c15 0 2024-03-18T14:50:54.579 INFO:tasks.workunit.client.0.smithi012.stdout:5/417: mkdir d3/d1e/d33/d89/d59/d8b 0 2024-03-18T14:50:54.582 INFO:tasks.workunit.client.0.smithi012.stdout:3/311: mkdir dc/d3c/d4c/d5f 0 2024-03-18T14:50:54.584 INFO:tasks.workunit.client.1.smithi100.stdout:7/40: rename d5/ff to d5/f17 0 2024-03-18T14:50:54.588 INFO:tasks.workunit.client.1.smithi100.stdout:5/30: link f5 f7 0 2024-03-18T14:50:54.588 INFO:tasks.workunit.client.1.smithi100.stdout:5/31: creat f8 x:0 0 0 2024-03-18T14:50:54.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/44: truncate f2 4524652 0 2024-03-18T14:50:54.591 INFO:tasks.workunit.client.1.smithi100.stdout:2/11: fsync f0 0 2024-03-18T14:50:54.598 INFO:tasks.workunit.client.0.smithi012.stdout:2/389: dwrite d0/d10/d18/d31/d4c/f67 [0,4194304] 0 2024-03-18T14:50:54.598 INFO:tasks.workunit.client.0.smithi012.stdout:2/390: chown d0/d10/d18/d31/d4c/d59/f69 1926140 1 2024-03-18T14:50:54.599 INFO:tasks.workunit.client.1.smithi100.stdout:7/41: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:54.599 INFO:tasks.workunit.client.1.smithi100.stdout:7/42: dread - d5/f17 zero size 2024-03-18T14:50:54.599 INFO:tasks.workunit.client.1.smithi100.stdout:2/12: dread f0 [0,4194304] 0 2024-03-18T14:50:54.599 INFO:tasks.workunit.client.0.smithi012.stdout:2/391: dread d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:50:54.600 INFO:tasks.workunit.client.0.smithi012.stdout:7/405: creat dd/d13/d23/d2f/d7a/f8b x:0 0 0 2024-03-18T14:50:54.601 INFO:tasks.workunit.client.1.smithi100.stdout:0/45: dread dd/f11 [0,4194304] 0 2024-03-18T14:50:54.605 INFO:tasks.workunit.client.0.smithi012.stdout:5/418: truncate d3/d29/d35/f5a 1397266 0 2024-03-18T14:50:54.607 INFO:tasks.workunit.client.1.smithi100.stdout:7/43: dread d5/fa [0,4194304] 0 2024-03-18T14:50:54.607 INFO:tasks.workunit.client.1.smithi100.stdout:5/32: dwrite f5 [0,4194304] 0 2024-03-18T14:50:54.607 INFO:tasks.workunit.client.1.smithi100.stdout:5/33: readlink l3 0 2024-03-18T14:50:54.609 INFO:tasks.workunit.client.1.smithi100.stdout:2/13: mkdir d3 0 2024-03-18T14:50:54.612 INFO:tasks.workunit.client.1.smithi100.stdout:4/4: truncate f0 351003 0 2024-03-18T14:50:54.620 INFO:tasks.workunit.client.1.smithi100.stdout:5/34: symlink l9 0 2024-03-18T14:50:54.621 INFO:tasks.workunit.client.0.smithi012.stdout:2/392: truncate d0/d3c/f46 258233 0 2024-03-18T14:50:54.621 INFO:tasks.workunit.client.0.smithi012.stdout:2/393: creat d0/f7b x:0 0 0 2024-03-18T14:50:54.622 INFO:tasks.workunit.client.1.smithi100.stdout:2/14: creat d3/f4 x:0 0 0 2024-03-18T14:50:54.624 INFO:tasks.workunit.client.0.smithi012.stdout:6/358: truncate fe 4863332 0 2024-03-18T14:50:54.625 INFO:tasks.workunit.client.1.smithi100.stdout:4/5: symlink l1 0 2024-03-18T14:50:54.627 INFO:tasks.workunit.client.0.smithi012.stdout:3/312: rename dc/c19 to dc/d3c/d4c/d5f/c60 0 2024-03-18T14:50:54.629 INFO:tasks.workunit.client.1.smithi100.stdout:5/35: link f4 fa 0 2024-03-18T14:50:54.631 INFO:tasks.workunit.client.1.smithi100.stdout:7/44: dwrite d5/fa [0,4194304] 0 2024-03-18T14:50:54.631 INFO:tasks.workunit.client.1.smithi100.stdout:7/45: read - d5/fd zero size 2024-03-18T14:50:54.631 INFO:tasks.workunit.client.1.smithi100.stdout:7/46: read - d5/fd zero size 2024-03-18T14:50:54.632 INFO:tasks.workunit.client.1.smithi100.stdout:2/15: symlink d3/l5 0 2024-03-18T14:50:54.632 INFO:tasks.workunit.client.0.smithi012.stdout:5/419: dwrite d3/d29/d36/d3d/d73/f7d [0,4194304] 0 2024-03-18T14:50:54.632 INFO:tasks.workunit.client.0.smithi012.stdout:5/420: readlink d3/d15/l44 0 2024-03-18T14:50:54.632 INFO:tasks.workunit.client.0.smithi012.stdout:7/406: dwrite dd/f5a [4194304,4194304] 0 2024-03-18T14:50:54.633 INFO:tasks.workunit.client.1.smithi100.stdout:2/16: dread f0 [0,4194304] 0 2024-03-18T14:50:54.633 INFO:tasks.workunit.client.1.smithi100.stdout:2/17: stat l1 0 2024-03-18T14:50:54.633 INFO:tasks.workunit.client.1.smithi100.stdout:2/18: dread - d3/f4 zero size 2024-03-18T14:50:54.639 INFO:tasks.workunit.client.0.smithi012.stdout:3/313: mknod dc/d3c/d4c/c61 0 2024-03-18T14:50:54.643 INFO:tasks.workunit.client.1.smithi100.stdout:6/19: getdents . 0 2024-03-18T14:50:54.645 INFO:tasks.workunit.client.0.smithi012.stdout:7/407: mknod dd/d13/d23/d2f/d7a/c8c 0 2024-03-18T14:50:54.645 INFO:tasks.workunit.client.0.smithi012.stdout:7/408: fdatasync f5 0 2024-03-18T14:50:54.646 INFO:tasks.workunit.client.1.smithi100.stdout:5/36: dwrite f7 [0,4194304] 0 2024-03-18T14:50:54.646 INFO:tasks.workunit.client.1.smithi100.stdout:6/20: truncate f1 4101575 0 2024-03-18T14:50:54.648 INFO:tasks.workunit.client.1.smithi100.stdout:8/26: getdents d4 0 2024-03-18T14:50:54.656 INFO:tasks.workunit.client.0.smithi012.stdout:7/409: truncate dd/d13/d23/d2f/f3d 201564 0 2024-03-18T14:50:54.656 INFO:tasks.workunit.client.1.smithi100.stdout:5/37: write f4 [3359052,96903] 0 2024-03-18T14:50:54.658 INFO:tasks.workunit.client.0.smithi012.stdout:7/410: mkdir dd/d13/d23/d2f/d53/d8d 0 2024-03-18T14:50:54.661 INFO:tasks.workunit.client.1.smithi100.stdout:5/38: unlink f4 0 2024-03-18T14:50:54.661 INFO:tasks.workunit.client.1.smithi100.stdout:5/39: chown f7 196501 1 2024-03-18T14:50:54.661 INFO:tasks.workunit.client.1.smithi100.stdout:5/40: creat fb x:0 0 0 2024-03-18T14:50:54.662 INFO:tasks.workunit.client.1.smithi100.stdout:1/34: fsync f5 0 2024-03-18T14:50:54.662 INFO:tasks.workunit.client.1.smithi100.stdout:1/35: write f5 [130123,4125] 0 2024-03-18T14:50:54.665 INFO:tasks.workunit.client.1.smithi100.stdout:1/36: write f2 [2713022,91145] 0 2024-03-18T14:50:54.665 INFO:tasks.workunit.client.1.smithi100.stdout:1/37: creat f7 x:0 0 0 2024-03-18T14:50:54.665 INFO:tasks.workunit.client.1.smithi100.stdout:1/38: readlink l3 0 2024-03-18T14:50:54.672 INFO:tasks.workunit.client.1.smithi100.stdout:7/47: unlink d5/f17 0 2024-03-18T14:50:54.673 INFO:tasks.workunit.client.1.smithi100.stdout:0/46: write dd/f11 [4161070,60309] 0 2024-03-18T14:50:54.681 INFO:tasks.workunit.client.0.smithi012.stdout:6/359: write d17/d26/f28 [1590770,79533] 0 2024-03-18T14:50:54.681 INFO:tasks.workunit.client.0.smithi012.stdout:6/360: creat d17/d26/d36/f67 x:0 0 0 2024-03-18T14:50:54.685 INFO:tasks.workunit.client.0.smithi012.stdout:6/361: dread f12 [0,4194304] 0 2024-03-18T14:50:54.704 INFO:tasks.workunit.client.1.smithi100.stdout:1/39: read f5 [80742,15258] 0 2024-03-18T14:50:54.704 INFO:tasks.workunit.client.1.smithi100.stdout:1/40: write f7 [281778,65968] 0 2024-03-18T14:50:54.704 INFO:tasks.workunit.client.1.smithi100.stdout:1/41: write f5 [571643,67271] 0 2024-03-18T14:50:54.704 INFO:tasks.workunit.client.1.smithi100.stdout:1/42: truncate f4 280230 0 2024-03-18T14:50:54.711 INFO:tasks.workunit.client.1.smithi100.stdout:5/41: write f5 [249692,33967] 0 2024-03-18T14:50:54.712 INFO:tasks.workunit.client.1.smithi100.stdout:5/42: stat f5 0 2024-03-18T14:50:54.712 INFO:tasks.workunit.client.1.smithi100.stdout:4/6: dread f0 [0,4194304] 0 2024-03-18T14:50:54.715 INFO:tasks.workunit.client.0.smithi012.stdout:0/387: sync 2024-03-18T14:50:54.716 INFO:tasks.workunit.client.0.smithi012.stdout:0/388: chown d1/d20/d23/l27 25079766 1 2024-03-18T14:50:54.716 INFO:tasks.workunit.client.0.smithi012.stdout:0/389: write d1/d20/d23/d3b/d33/f70 [4285779,58749] 0 2024-03-18T14:50:54.716 INFO:tasks.workunit.client.0.smithi012.stdout:3/314: dwrite dc/f14 [0,4194304] 0 2024-03-18T14:50:54.717 INFO:tasks.workunit.client.1.smithi100.stdout:4/7: mknod c2 0 2024-03-18T14:50:54.718 INFO:tasks.workunit.client.0.smithi012.stdout:0/390: stat d1/d20/d23/l27 0 2024-03-18T14:50:54.724 INFO:tasks.workunit.client.1.smithi100.stdout:4/8: symlink l3 0 2024-03-18T14:50:54.725 INFO:tasks.workunit.client.0.smithi012.stdout:7/411: truncate dd/d13/d23/d2f/f3d 226137 0 2024-03-18T14:50:54.725 INFO:tasks.workunit.client.0.smithi012.stdout:7/412: chown dd/d13/d23/d60/f6c 1253154 1 2024-03-18T14:50:54.728 INFO:tasks.workunit.client.1.smithi100.stdout:4/9: mknod c4 0 2024-03-18T14:50:54.733 INFO:tasks.workunit.client.0.smithi012.stdout:7/413: link dd/f5b dd/d13/d23/d2f/d53/f8e 0 2024-03-18T14:50:54.733 INFO:tasks.workunit.client.0.smithi012.stdout:7/414: readlink dd/d13/d23/d2f/l43 0 2024-03-18T14:50:54.734 INFO:tasks.workunit.client.0.smithi012.stdout:3/315: dread dc/f24 [0,4194304] 0 2024-03-18T14:50:54.743 INFO:tasks.workunit.client.1.smithi100.stdout:2/19: truncate f0 92189 0 2024-03-18T14:50:54.745 INFO:tasks.workunit.client.1.smithi100.stdout:2/20: mkdir d3/d6 0 2024-03-18T14:50:54.746 INFO:tasks.workunit.client.1.smithi100.stdout:2/21: mknod d3/c7 0 2024-03-18T14:50:54.746 INFO:tasks.workunit.client.1.smithi100.stdout:2/22: write d3/f4 [329751,39094] 0 2024-03-18T14:50:54.767 INFO:tasks.workunit.client.1.smithi100.stdout:3/26: dwrite f2 [4194304,4194304] 0 2024-03-18T14:50:54.767 INFO:tasks.workunit.client.0.smithi012.stdout:6/362: dwrite d17/d19/f2b [0,4194304] 0 2024-03-18T14:50:54.768 INFO:tasks.workunit.client.0.smithi012.stdout:6/363: rename d17/f3b to d17/d19/d27/f68 0 2024-03-18T14:50:54.770 INFO:tasks.workunit.client.1.smithi100.stdout:8/27: truncate d4/f6 67666 0 2024-03-18T14:50:54.770 INFO:tasks.workunit.client.1.smithi100.stdout:6/21: dwrite f1 [0,4194304] 0 2024-03-18T14:50:54.775 INFO:tasks.workunit.client.1.smithi100.stdout:0/47: dwrite dd/f11 [0,4194304] 0 2024-03-18T14:50:54.775 INFO:tasks.workunit.client.1.smithi100.stdout:0/48: creat dd/f16 x:0 0 0 2024-03-18T14:50:54.776 INFO:tasks.workunit.client.1.smithi100.stdout:7/48: dwrite d5/f10 [0,4194304] 0 2024-03-18T14:50:54.776 INFO:tasks.workunit.client.1.smithi100.stdout:5/43: dwrite fb [0,4194304] 0 2024-03-18T14:50:54.776 INFO:tasks.workunit.client.1.smithi100.stdout:1/43: dwrite f2 [0,4194304] 0 2024-03-18T14:50:54.776 INFO:tasks.workunit.client.1.smithi100.stdout:0/49: dread f6 [0,4194304] 0 2024-03-18T14:50:54.778 INFO:tasks.workunit.client.1.smithi100.stdout:7/49: rename d5/f13 to d5/d14/f18 0 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/391: dwrite d1/d20/d4e/d54/d67/f72 [0,4194304] 0 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/392: chown d1/d20/d4e/d54/d67/c71 116 1 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:3/316: dwrite dc/f48 [0,4194304] 0 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/393: chown d1/d20/d23/d26/f36 105 1 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:3/317: truncate dc/d30/d3a/f4f 1025657 0 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:3/318: write dc/d3c/d4c/d50/d54/f58 [151949,113334] 0 2024-03-18T14:50:54.780 INFO:tasks.workunit.client.0.smithi012.stdout:3/319: write dc/d3c/d4c/d50/f5d [113848,52481] 0 2024-03-18T14:50:54.781 INFO:tasks.workunit.client.1.smithi100.stdout:0/50: write f2 [4168022,24548] 0 2024-03-18T14:50:54.781 INFO:tasks.workunit.client.1.smithi100.stdout:0/51: readlink lb 0 2024-03-18T14:50:54.781 INFO:tasks.workunit.client.1.smithi100.stdout:4/10: dwrite f0 [0,4194304] 0 2024-03-18T14:50:54.781 INFO:tasks.workunit.client.1.smithi100.stdout:5/44: readlink l9 0 2024-03-18T14:50:54.782 INFO:tasks.workunit.client.0.smithi012.stdout:7/415: dwrite dd/d13/d23/d60/f89 [0,4194304] 0 2024-03-18T14:50:54.782 INFO:tasks.workunit.client.1.smithi100.stdout:6/22: dread f1 [0,4194304] 0 2024-03-18T14:50:54.784 INFO:tasks.workunit.client.1.smithi100.stdout:1/44: mkdir d8 0 2024-03-18T14:50:54.786 INFO:tasks.workunit.client.1.smithi100.stdout:7/50: mknod d5/d14/d16/c19 0 2024-03-18T14:50:54.789 INFO:tasks.workunit.client.1.smithi100.stdout:3/27: dwrite f2 [4194304,4194304] 0 2024-03-18T14:50:54.789 INFO:tasks.workunit.client.1.smithi100.stdout:3/28: fdatasync f2 0 2024-03-18T14:50:54.790 INFO:tasks.workunit.client.1.smithi100.stdout:3/29: chown f2 6756073 1 2024-03-18T14:50:54.790 INFO:tasks.workunit.client.1.smithi100.stdout:3/30: chown f2 3 1 2024-03-18T14:50:54.791 INFO:tasks.workunit.client.1.smithi100.stdout:8/28: dread f3 [0,4194304] 0 2024-03-18T14:50:54.792 INFO:tasks.workunit.client.1.smithi100.stdout:8/29: dread f3 [0,4194304] 0 2024-03-18T14:50:54.793 INFO:tasks.workunit.client.1.smithi100.stdout:8/30: dread d4/f6 [0,4194304] 0 2024-03-18T14:50:54.793 INFO:tasks.workunit.client.0.smithi012.stdout:0/394: mknod d1/d20/d23/c87 0 2024-03-18T14:50:54.794 INFO:tasks.workunit.client.1.smithi100.stdout:2/23: write f0 [459030,89222] 0 2024-03-18T14:50:54.796 INFO:tasks.workunit.client.1.smithi100.stdout:4/11: dread f0 [0,4194304] 0 2024-03-18T14:50:54.796 INFO:tasks.workunit.client.1.smithi100.stdout:4/12: fsync f0 0 2024-03-18T14:50:54.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/52: dwrite dd/ff [0,4194304] 0 2024-03-18T14:50:54.799 INFO:tasks.workunit.client.0.smithi012.stdout:6/364: getdents d17 0 2024-03-18T14:50:54.799 INFO:tasks.workunit.client.1.smithi100.stdout:9/15: sync 2024-03-18T14:50:54.799 INFO:tasks.workunit.client.1.smithi100.stdout:9/16: dread - f2 zero size 2024-03-18T14:50:54.799 INFO:tasks.workunit.client.1.smithi100.stdout:9/17: creat f5 x:0 0 0 2024-03-18T14:50:54.800 INFO:tasks.workunit.client.1.smithi100.stdout:9/18: chown f5 4939626 1 2024-03-18T14:50:54.800 INFO:tasks.workunit.client.1.smithi100.stdout:9/19: creat f6 x:0 0 0 2024-03-18T14:50:54.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/45: write f5 [5203676,45237] 0 2024-03-18T14:50:54.802 INFO:tasks.workunit.client.1.smithi100.stdout:6/23: unlink f1 0 2024-03-18T14:50:54.802 INFO:tasks.workunit.client.1.smithi100.stdout:6/24: dwrite - no filename 2024-03-18T14:50:54.803 INFO:tasks.workunit.client.0.smithi012.stdout:7/416: symlink dd/d13/d23/d76/d6b/l8f 0 2024-03-18T14:50:54.803 INFO:tasks.workunit.client.0.smithi012.stdout:7/417: chown dd/d13/d23/d76/d6f/c79 1 1 2024-03-18T14:50:54.807 INFO:tasks.workunit.client.1.smithi100.stdout:1/45: mknod d8/c9 0 2024-03-18T14:50:54.807 INFO:tasks.workunit.client.1.smithi100.stdout:1/46: stat f4 0 2024-03-18T14:50:54.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/51: creat d5/d14/d16/f1a x:0 0 0 2024-03-18T14:50:54.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/52: fdatasync d5/fa 0 2024-03-18T14:50:54.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/53: readlink d5/lc 0 2024-03-18T14:50:54.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/54: stat d5/l8 0 2024-03-18T14:50:54.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/55: dread - d5/fd zero size 2024-03-18T14:50:54.812 INFO:tasks.workunit.client.1.smithi100.stdout:3/31: mkdir d3 0 2024-03-18T14:50:54.813 INFO:tasks.workunit.client.1.smithi100.stdout:0/53: dwrite dd/ff [0,4194304] 0 2024-03-18T14:50:54.813 INFO:tasks.workunit.client.1.smithi100.stdout:0/54: write dd/f16 [966645,79803] 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/31: creat d4/f9 x:0 0 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/32: write f1 [4521772,53714] 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/33: write d4/f6 [895288,36775] 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/34: dread - d4/f9 zero size 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/35: write d4/f6 [1799943,32241] 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/36: write d4/f9 [532722,2870] 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/37: creat d4/fa x:0 0 0 2024-03-18T14:50:54.817 INFO:tasks.workunit.client.1.smithi100.stdout:4/13: write f0 [4989514,129369] 0 2024-03-18T14:50:54.818 INFO:tasks.workunit.client.0.smithi012.stdout:0/395: mkdir d1/d20/d23/d88 0 2024-03-18T14:50:54.819 INFO:tasks.workunit.client.1.smithi100.stdout:2/24: mknod d3/d6/c8 0 2024-03-18T14:50:54.820 INFO:tasks.workunit.client.0.smithi012.stdout:3/320: rmdir dc/d30/d3a 39 2024-03-18T14:50:54.822 INFO:tasks.workunit.client.0.smithi012.stdout:6/365: symlink d17/d26/l69 0 2024-03-18T14:50:54.822 INFO:tasks.workunit.client.0.smithi012.stdout:6/366: readlink d17/d26/d36/l5f 0 2024-03-18T14:50:54.822 INFO:tasks.workunit.client.0.smithi012.stdout:6/367: stat d17/d33/f48 0 2024-03-18T14:50:54.823 INFO:tasks.workunit.client.0.smithi012.stdout:7/418: dwrite dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:54.825 INFO:tasks.workunit.client.0.smithi012.stdout:0/396: symlink d1/d2d/l89 0 2024-03-18T14:50:54.830 INFO:tasks.workunit.client.0.smithi012.stdout:0/397: mkdir d1/d20/d4e/d54/d8a 0 2024-03-18T14:50:54.831 INFO:tasks.workunit.client.1.smithi100.stdout:5/46: write f0 [1337593,130952] 0 2024-03-18T14:50:54.832 INFO:tasks.workunit.client.0.smithi012.stdout:7/419: getdents dd/d13/d23/d2f/d7a 0 2024-03-18T14:50:54.834 INFO:tasks.workunit.client.1.smithi100.stdout:6/25: unlink c3 0 2024-03-18T14:50:54.834 INFO:tasks.workunit.client.0.smithi012.stdout:6/368: dread d17/d26/f4a [0,4194304] 0 2024-03-18T14:50:54.835 INFO:tasks.workunit.client.1.smithi100.stdout:1/47: creat d8/fa x:0 0 0 2024-03-18T14:50:54.836 INFO:tasks.workunit.client.1.smithi100.stdout:3/32: rename f2 to d3/f4 0 2024-03-18T14:50:54.837 INFO:tasks.workunit.client.1.smithi100.stdout:0/55: mknod dd/c17 0 2024-03-18T14:50:54.840 INFO:tasks.workunit.client.1.smithi100.stdout:8/38: mknod d4/cb 0 2024-03-18T14:50:54.842 INFO:tasks.workunit.client.1.smithi100.stdout:4/14: symlink l5 0 2024-03-18T14:50:54.842 INFO:tasks.workunit.client.1.smithi100.stdout:4/15: rmdir - no directory 2024-03-18T14:50:54.842 INFO:tasks.workunit.client.1.smithi100.stdout:4/16: stat l3 0 2024-03-18T14:50:54.843 INFO:tasks.workunit.client.1.smithi100.stdout:4/17: stat l3 0 2024-03-18T14:50:54.843 INFO:tasks.workunit.client.1.smithi100.stdout:2/25: creat d3/d6/f9 x:0 0 0 2024-03-18T14:50:54.846 INFO:tasks.workunit.client.0.smithi012.stdout:0/398: dread d1/d16/d31/f35 [0,4194304] 0 2024-03-18T14:50:54.846 INFO:tasks.workunit.client.0.smithi012.stdout:3/321: dwrite f6 [0,4194304] 0 2024-03-18T14:50:54.846 INFO:tasks.workunit.client.0.smithi012.stdout:0/399: creat d1/d20/d23/d3b/d33/f8b x:0 0 0 2024-03-18T14:50:54.847 INFO:tasks.workunit.client.1.smithi100.stdout:1/48: write f2 [2304710,10218] 0 2024-03-18T14:50:54.848 INFO:tasks.workunit.client.0.smithi012.stdout:3/322: mknod dc/d3c/d4c/d5f/c62 0 2024-03-18T14:50:54.850 INFO:tasks.workunit.client.0.smithi012.stdout:3/323: truncate dc/d30/f37 2870660 0 2024-03-18T14:50:54.851 INFO:tasks.workunit.client.1.smithi100.stdout:0/56: write f3 [2985594,44909] 0 2024-03-18T14:50:54.851 INFO:tasks.workunit.client.0.smithi012.stdout:7/420: dread dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:54.851 INFO:tasks.workunit.client.0.smithi012.stdout:3/324: rmdir dc/d3c/d4c/d5f 39 2024-03-18T14:50:54.852 INFO:tasks.workunit.client.0.smithi012.stdout:0/400: dread d1/d20/d23/d26/f36 [0,4194304] 0 2024-03-18T14:50:54.852 INFO:tasks.workunit.client.0.smithi012.stdout:0/401: getdents d1/d20/d23/d88 0 2024-03-18T14:50:54.853 INFO:tasks.workunit.client.0.smithi012.stdout:7/421: stat dd/d13/d23/d2f/d53/c87 0 2024-03-18T14:50:54.856 INFO:tasks.workunit.client.0.smithi012.stdout:0/402: link d1/l4 d1/d20/d23/d7f/l8c 0 2024-03-18T14:50:54.859 INFO:tasks.workunit.client.0.smithi012.stdout:7/422: creat dd/d13/d23/d2f/d53/f90 x:0 0 0 2024-03-18T14:50:54.861 INFO:tasks.workunit.client.0.smithi012.stdout:6/369: dwrite d17/d33/f48 [0,4194304] 0 2024-03-18T14:50:54.862 INFO:tasks.workunit.client.0.smithi012.stdout:3/325: rename dc/c21 to dc/d30/c63 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.0.smithi012.stdout:3/326: dread dc/d30/d3a/f4f [0,4194304] 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/20: getdents . 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/21: write f4 [860257,76199] 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/22: truncate f2 592656 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/23: write f2 [690967,53030] 0 2024-03-18T14:50:54.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/24: getdents . 0 2024-03-18T14:50:54.867 INFO:tasks.workunit.client.1.smithi100.stdout:5/47: write f8 [697327,35653] 0 2024-03-18T14:50:54.867 INFO:tasks.workunit.client.1.smithi100.stdout:5/48: chown l9 7 1 2024-03-18T14:50:54.867 INFO:tasks.workunit.client.1.smithi100.stdout:5/49: chown f0 40460434 1 2024-03-18T14:50:54.869 INFO:tasks.workunit.client.1.smithi100.stdout:6/26: creat f4 x:0 0 0 2024-03-18T14:50:54.869 INFO:tasks.workunit.client.1.smithi100.stdout:6/27: fdatasync f4 0 2024-03-18T14:50:54.869 INFO:tasks.workunit.client.1.smithi100.stdout:6/28: read - f4 zero size 2024-03-18T14:50:54.872 INFO:tasks.workunit.client.1.smithi100.stdout:3/33: getdents d3 0 2024-03-18T14:50:54.872 INFO:tasks.workunit.client.1.smithi100.stdout:3/34: stat d3/f4 0 2024-03-18T14:50:54.874 INFO:tasks.workunit.client.0.smithi012.stdout:6/370: creat d17/d38/d51/d63/f6a x:0 0 0 2024-03-18T14:50:54.876 INFO:tasks.workunit.client.1.smithi100.stdout:8/39: mkdir d4/dc 0 2024-03-18T14:50:54.878 INFO:tasks.workunit.client.1.smithi100.stdout:4/18: symlink l6 0 2024-03-18T14:50:54.878 INFO:tasks.workunit.client.1.smithi100.stdout:4/19: chown c4 26 1 2024-03-18T14:50:54.878 INFO:tasks.workunit.client.0.smithi012.stdout:6/371: chown c10 7 1 2024-03-18T14:50:54.880 INFO:tasks.workunit.client.1.smithi100.stdout:2/26: mkdir d3/da 0 2024-03-18T14:50:54.880 INFO:tasks.workunit.client.1.smithi100.stdout:2/27: creat d3/fb x:0 0 0 2024-03-18T14:50:54.880 INFO:tasks.workunit.client.1.smithi100.stdout:2/28: write d3/f4 [910187,95950] 0 2024-03-18T14:50:54.881 INFO:tasks.workunit.client.1.smithi100.stdout:2/29: dread - d3/d6/f9 zero size 2024-03-18T14:50:54.881 INFO:tasks.workunit.client.1.smithi100.stdout:2/30: creat d3/fc x:0 0 0 2024-03-18T14:50:54.881 INFO:tasks.workunit.client.1.smithi100.stdout:2/31: chown d3/c7 26966646 1 2024-03-18T14:50:54.881 INFO:tasks.workunit.client.1.smithi100.stdout:8/40: dread d4/f9 [0,4194304] 0 2024-03-18T14:50:54.882 INFO:tasks.workunit.client.1.smithi100.stdout:1/49: rename d8/c9 to d8/cb 0 2024-03-18T14:50:54.885 INFO:tasks.workunit.client.1.smithi100.stdout:8/41: dread f1 [0,4194304] 0 2024-03-18T14:50:54.885 INFO:tasks.workunit.client.1.smithi100.stdout:9/25: link f4 f7 0 2024-03-18T14:50:54.888 INFO:tasks.workunit.client.1.smithi100.stdout:3/35: unlink d3/f4 0 2024-03-18T14:50:54.888 INFO:tasks.workunit.client.1.smithi100.stdout:4/20: mknod c7 0 2024-03-18T14:50:54.888 INFO:tasks.workunit.client.1.smithi100.stdout:4/21: creat f8 x:0 0 0 2024-03-18T14:50:54.891 INFO:tasks.workunit.client.1.smithi100.stdout:2/32: mknod d3/d6/cd 0 2024-03-18T14:50:54.893 INFO:tasks.workunit.client.1.smithi100.stdout:8/42: mkdir d4/dc/dd 0 2024-03-18T14:50:54.893 INFO:tasks.workunit.client.0.smithi012.stdout:3/327: dwrite dc/d3c/d4c/d50/f5d [0,4194304] 0 2024-03-18T14:50:54.893 INFO:tasks.workunit.client.1.smithi100.stdout:6/29: dwrite f4 [0,4194304] 0 2024-03-18T14:50:54.894 INFO:tasks.workunit.client.1.smithi100.stdout:0/57: dwrite f7 [0,4194304] 0 2024-03-18T14:50:54.896 INFO:tasks.workunit.client.1.smithi100.stdout:2/33: mkdir d3/da/de 0 2024-03-18T14:50:54.897 INFO:tasks.workunit.client.0.smithi012.stdout:9/422: sync 2024-03-18T14:50:54.897 INFO:tasks.workunit.client.0.smithi012.stdout:4/383: sync 2024-03-18T14:50:54.897 INFO:tasks.workunit.client.0.smithi012.stdout:8/491: sync 2024-03-18T14:50:54.905 INFO:tasks.workunit.client.0.smithi012.stdout:8/492: symlink d0/d12/d33/d50/l9c 0 2024-03-18T14:50:54.905 INFO:tasks.workunit.client.0.smithi012.stdout:8/493: write d0/d12/f4d [927876,104365] 0 2024-03-18T14:50:54.910 INFO:tasks.workunit.client.1.smithi100.stdout:8/43: mkdir d4/dc/dd/de 0 2024-03-18T14:50:54.910 INFO:tasks.workunit.client.1.smithi100.stdout:8/44: creat d4/ff x:0 0 0 2024-03-18T14:50:54.911 INFO:tasks.workunit.client.0.smithi012.stdout:3/328: unlink dc/l40 0 2024-03-18T14:50:54.913 INFO:tasks.workunit.client.0.smithi012.stdout:8/494: link d0/d12/f20 d0/d5a/d19/f9d 0 2024-03-18T14:50:54.919 INFO:tasks.workunit.client.0.smithi012.stdout:8/495: mkdir d0/d5a/d19/d57/d9e 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.1.smithi100.stdout:0/58: truncate f2 1115994 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.1.smithi100.stdout:0/59: fdatasync dd/ff 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.1.smithi100.stdout:0/60: creat dd/f18 x:0 0 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.1.smithi100.stdout:0/61: write fa [417338,108401] 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.1.smithi100.stdout:0/62: write f9 [673646,103795] 0 2024-03-18T14:50:54.922 INFO:tasks.workunit.client.0.smithi012.stdout:8/496: mkdir d0/d15/d6e/d84/d54/d9f 0 2024-03-18T14:50:54.923 INFO:tasks.workunit.client.1.smithi100.stdout:2/34: creat d3/da/de/ff x:0 0 0 2024-03-18T14:50:54.924 INFO:tasks.workunit.client.1.smithi100.stdout:2/35: creat d3/f10 x:0 0 0 2024-03-18T14:50:54.925 INFO:tasks.workunit.client.0.smithi012.stdout:8/497: symlink d0/d47/la0 0 2024-03-18T14:50:54.926 INFO:tasks.workunit.client.1.smithi100.stdout:8/45: rmdir d4/dc 39 2024-03-18T14:50:54.927 INFO:tasks.workunit.client.1.smithi100.stdout:1/50: dwrite f7 [0,4194304] 0 2024-03-18T14:50:54.927 INFO:tasks.workunit.client.1.smithi100.stdout:1/51: fdatasync f7 0 2024-03-18T14:50:54.927 INFO:tasks.workunit.client.1.smithi100.stdout:1/52: fdatasync f2 0 2024-03-18T14:50:54.928 INFO:tasks.workunit.client.0.smithi012.stdout:8/498: rename d0/d47/f99 to d0/d15/d6e/d84/d2a/fa1 0 2024-03-18T14:50:54.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/63: link dd/c13 dd/c19 0 2024-03-18T14:50:54.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/64: creat dd/f1a x:0 0 0 2024-03-18T14:50:54.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/65: stat dd/f1a 0 2024-03-18T14:50:54.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/66: fsync f7 0 2024-03-18T14:50:54.934 INFO:tasks.workunit.client.1.smithi100.stdout:8/46: mknod d4/dc/c10 0 2024-03-18T14:50:54.934 INFO:tasks.workunit.client.1.smithi100.stdout:1/53: symlink d8/lc 0 2024-03-18T14:50:54.934 INFO:tasks.workunit.client.1.smithi100.stdout:1/54: write f4 [217622,58135] 0 2024-03-18T14:50:54.934 INFO:tasks.workunit.client.1.smithi100.stdout:1/55: chown d8/fa 682137 1 2024-03-18T14:50:54.934 INFO:tasks.workunit.client.1.smithi100.stdout:1/56: dread - d8/fa zero size 2024-03-18T14:50:54.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/47: creat d4/dc/dd/f11 x:0 0 0 2024-03-18T14:50:54.935 INFO:tasks.workunit.client.1.smithi100.stdout:8/48: read d4/f6 [833747,99258] 0 2024-03-18T14:50:54.935 INFO:tasks.workunit.client.1.smithi100.stdout:1/57: unlink f4 0 2024-03-18T14:50:54.936 INFO:tasks.workunit.client.1.smithi100.stdout:8/49: mknod d4/dc/dd/de/c12 0 2024-03-18T14:50:54.937 INFO:tasks.workunit.client.1.smithi100.stdout:8/50: read - d4/ff zero size 2024-03-18T14:50:54.955 INFO:tasks.workunit.client.0.smithi012.stdout:8/499: dread d0/d5a/d19/f9d [0,4194304] 0 2024-03-18T14:50:54.955 INFO:tasks.workunit.client.0.smithi012.stdout:8/500: write d0/d15/d6e/d84/d54/f85 [2233299,50966] 0 2024-03-18T14:50:54.955 INFO:tasks.workunit.client.0.smithi012.stdout:8/501: creat d0/d5a/fa2 x:0 0 0 2024-03-18T14:50:54.955 INFO:tasks.workunit.client.0.smithi012.stdout:8/502: write d0/d5a/f1d [4828210,46543] 0 2024-03-18T14:50:54.965 INFO:tasks.workunit.client.0.smithi012.stdout:4/384: dwrite d6/d19/d4f/f5e [0,4194304] 0 2024-03-18T14:50:54.966 INFO:tasks.workunit.client.1.smithi100.stdout:3/36: dwrite f1 [0,4194304] 0 2024-03-18T14:50:54.967 INFO:tasks.workunit.client.0.smithi012.stdout:9/423: dwrite d3/d9/d19/d77/d52/f6d [0,4194304] 0 2024-03-18T14:50:54.967 INFO:tasks.workunit.client.0.smithi012.stdout:3/329: dwrite dc/d30/d3a/f5e [0,4194304] 0 2024-03-18T14:50:54.967 INFO:tasks.workunit.client.1.smithi100.stdout:9/26: dwrite f5 [0,4194304] 0 2024-03-18T14:50:54.968 INFO:tasks.workunit.client.1.smithi100.stdout:4/22: dwrite f8 [0,4194304] 0 2024-03-18T14:50:54.968 INFO:tasks.workunit.client.0.smithi012.stdout:4/385: mknod d6/d1a/d30/d46/d6f/c83 0 2024-03-18T14:50:54.969 INFO:tasks.workunit.client.1.smithi100.stdout:6/30: dwrite f4 [0,4194304] 0 2024-03-18T14:50:54.971 INFO:tasks.workunit.client.0.smithi012.stdout:8/503: dread d0/d12/f2d [0,4194304] 0 2024-03-18T14:50:54.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/372: dwrite ff [0,4194304] 0 2024-03-18T14:50:54.971 INFO:tasks.workunit.client.0.smithi012.stdout:9/424: dread d3/d9/d19/d66/f51 [0,4194304] 0 2024-03-18T14:50:54.971 INFO:tasks.workunit.client.0.smithi012.stdout:8/504: creat d0/d5a/d19/fa3 x:0 0 0 2024-03-18T14:50:54.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/51: dwrite d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:54.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/52: fdatasync d4/dc/dd/f11 0 2024-03-18T14:50:54.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/53: dread - d4/fa zero size 2024-03-18T14:50:54.972 INFO:tasks.workunit.client.0.smithi012.stdout:3/330: dread dc/d3c/f4d [0,4194304] 0 2024-03-18T14:50:54.973 INFO:tasks.workunit.client.1.smithi100.stdout:9/27: symlink l8 0 2024-03-18T14:50:54.973 INFO:tasks.workunit.client.1.smithi100.stdout:9/28: truncate f2 1595872 0 2024-03-18T14:50:54.973 INFO:tasks.workunit.client.0.smithi012.stdout:4/386: dread d6/f25 [0,4194304] 0 2024-03-18T14:50:54.973 INFO:tasks.workunit.client.1.smithi100.stdout:4/23: write f0 [1184074,94917] 0 2024-03-18T14:50:54.974 INFO:tasks.workunit.client.1.smithi100.stdout:4/24: rmdir - no directory 2024-03-18T14:50:54.975 INFO:tasks.workunit.client.0.smithi012.stdout:9/425: write d3/f4 [1966825,7815] 0 2024-03-18T14:50:54.976 INFO:tasks.workunit.client.1.smithi100.stdout:3/37: dread f1 [0,4194304] 0 2024-03-18T14:50:54.976 INFO:tasks.workunit.client.1.smithi100.stdout:3/38: fsync f1 0 2024-03-18T14:50:54.976 INFO:tasks.workunit.client.1.smithi100.stdout:2/36: write d3/f4 [112397,89599] 0 2024-03-18T14:50:54.976 INFO:tasks.workunit.client.1.smithi100.stdout:2/37: read - d3/da/de/ff zero size 2024-03-18T14:50:54.977 INFO:tasks.workunit.client.1.smithi100.stdout:9/29: dread f2 [0,4194304] 0 2024-03-18T14:50:54.977 INFO:tasks.workunit.client.1.smithi100.stdout:9/30: rmdir - no directory 2024-03-18T14:50:54.977 INFO:tasks.workunit.client.0.smithi012.stdout:6/373: dread f16 [0,4194304] 0 2024-03-18T14:50:54.978 INFO:tasks.workunit.client.0.smithi012.stdout:8/505: write d0/d5a/f1d [3897677,67739] 0 2024-03-18T14:50:54.984 INFO:tasks.workunit.client.1.smithi100.stdout:4/25: dread f0 [0,4194304] 0 2024-03-18T14:50:54.985 INFO:tasks.workunit.client.0.smithi012.stdout:4/387: dread d6/da/f13 [0,4194304] 0 2024-03-18T14:50:54.988 INFO:tasks.workunit.client.1.smithi100.stdout:6/31: symlink l5 0 2024-03-18T14:50:54.989 INFO:tasks.workunit.client.0.smithi012.stdout:3/331: mkdir dc/d3c/d4c/d50/d64 0 2024-03-18T14:50:54.991 INFO:tasks.workunit.client.0.smithi012.stdout:9/426: creat d3/d9/d19/d66/d21/d35/d71/f87 x:0 0 0 2024-03-18T14:50:54.992 INFO:tasks.workunit.client.0.smithi012.stdout:6/374: mkdir d17/d38/d51/d6b 0 2024-03-18T14:50:54.994 INFO:tasks.workunit.client.0.smithi012.stdout:8/506: unlink d0/d15/d6e/d84/d54/f8d 0 2024-03-18T14:50:54.996 INFO:tasks.workunit.client.1.smithi100.stdout:9/31: dwrite f2 [0,4194304] 0 2024-03-18T14:50:54.996 INFO:tasks.workunit.client.1.smithi100.stdout:9/32: write f6 [242249,106895] 0 2024-03-18T14:50:55.004 INFO:tasks.workunit.client.0.smithi012.stdout:8/507: dread d0/d12/f46 [0,4194304] 0 2024-03-18T14:50:55.007 INFO:tasks.workunit.client.1.smithi100.stdout:9/33: dread f2 [0,4194304] 0 2024-03-18T14:50:55.008 INFO:tasks.workunit.client.1.smithi100.stdout:3/39: creat d3/f5 x:0 0 0 2024-03-18T14:50:55.010 INFO:tasks.workunit.client.1.smithi100.stdout:2/38: symlink d3/da/l11 0 2024-03-18T14:50:55.011 INFO:tasks.workunit.client.0.smithi012.stdout:4/388: symlink d6/d1a/d64/l84 0 2024-03-18T14:50:55.015 INFO:tasks.workunit.client.1.smithi100.stdout:4/26: mkdir d9 0 2024-03-18T14:50:55.016 INFO:tasks.workunit.client.0.smithi012.stdout:9/427: rename d3/d9/d19/d66/d21/l53 to d3/d9/d5c/l88 0 2024-03-18T14:50:55.017 INFO:tasks.workunit.client.1.smithi100.stdout:8/54: getdents d4/dc 0 2024-03-18T14:50:55.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/32: dwrite f4 [4194304,4194304] 0 2024-03-18T14:50:55.018 INFO:tasks.workunit.client.0.smithi012.stdout:8/508: mkdir d0/d15/d6e/d84/d54/da4 0 2024-03-18T14:50:55.020 INFO:tasks.workunit.client.1.smithi100.stdout:0/67: dwrite f2 [0,4194304] 0 2024-03-18T14:50:55.021 INFO:tasks.workunit.client.0.smithi012.stdout:3/332: rmdir dc/d3c/d4c/d50/d54 39 2024-03-18T14:50:55.023 INFO:tasks.workunit.client.1.smithi100.stdout:9/34: link f7 f9 0 2024-03-18T14:50:55.029 INFO:tasks.workunit.client.1.smithi100.stdout:2/39: rename d3/fb to d3/d6/f12 0 2024-03-18T14:50:55.030 INFO:tasks.workunit.client.0.smithi012.stdout:8/509: link d0/d12/f20 d0/d12/d33/d50/fa5 0 2024-03-18T14:50:55.031 INFO:tasks.workunit.client.1.smithi100.stdout:4/27: symlink d9/la 0 2024-03-18T14:50:55.032 INFO:tasks.workunit.client.0.smithi012.stdout:4/389: rmdir d6/d27 39 2024-03-18T14:50:55.034 INFO:tasks.workunit.client.1.smithi100.stdout:8/55: symlink d4/dc/dd/l13 0 2024-03-18T14:50:55.034 INFO:tasks.workunit.client.0.smithi012.stdout:3/333: dwrite dc/f2b [4194304,4194304] 0 2024-03-18T14:50:55.036 INFO:tasks.workunit.client.0.smithi012.stdout:6/375: rename c13 to d17/d38/d51/d63/c6c 0 2024-03-18T14:50:55.037 INFO:tasks.workunit.client.0.smithi012.stdout:3/334: dread dc/f24 [0,4194304] 0 2024-03-18T14:50:55.040 INFO:tasks.workunit.client.0.smithi012.stdout:9/428: link d3/d20/c5a d3/d9/d19/d66/d69/c89 0 2024-03-18T14:50:55.040 INFO:tasks.workunit.client.0.smithi012.stdout:9/429: fdatasync d3/d9/f1c 0 2024-03-18T14:50:55.041 INFO:tasks.workunit.client.0.smithi012.stdout:9/430: fdatasync d3/d9/d19/d66/d21/d35/d71/f87 0 2024-03-18T14:50:55.041 INFO:tasks.workunit.client.1.smithi100.stdout:8/56: write d4/dc/dd/f11 [3505550,9194] 0 2024-03-18T14:50:55.042 INFO:tasks.workunit.client.0.smithi012.stdout:8/510: mknod d0/d5a/ca6 0 2024-03-18T14:50:55.044 INFO:tasks.workunit.client.0.smithi012.stdout:4/390: creat d6/d19/f85 x:0 0 0 2024-03-18T14:50:55.049 INFO:tasks.workunit.client.1.smithi100.stdout:4/28: dwrite f0 [4194304,4194304] 0 2024-03-18T14:50:55.049 INFO:tasks.workunit.client.1.smithi100.stdout:4/29: fdatasync f8 0 2024-03-18T14:50:55.050 INFO:tasks.workunit.client.1.smithi100.stdout:9/35: mkdir da 0 2024-03-18T14:50:55.052 INFO:tasks.workunit.client.0.smithi012.stdout:6/376: rename f9 to d17/d19/f6d 0 2024-03-18T14:50:55.055 INFO:tasks.workunit.client.0.smithi012.stdout:3/335: link dc/f12 dc/d30/d35/f65 0 2024-03-18T14:50:55.057 INFO:tasks.workunit.client.1.smithi100.stdout:2/40: mkdir d3/d13 0 2024-03-18T14:50:55.058 INFO:tasks.workunit.client.0.smithi012.stdout:8/511: getdents d0/d12 0 2024-03-18T14:50:55.059 INFO:tasks.workunit.client.1.smithi100.stdout:3/40: truncate f1 1906511 0 2024-03-18T14:50:55.060 INFO:tasks.workunit.client.1.smithi100.stdout:6/33: getdents . 0 2024-03-18T14:50:55.062 INFO:tasks.workunit.client.1.smithi100.stdout:8/57: truncate f3 1010599 0 2024-03-18T14:50:55.063 INFO:tasks.workunit.client.1.smithi100.stdout:0/68: truncate f9 240736 0 2024-03-18T14:50:55.065 INFO:tasks.workunit.client.0.smithi012.stdout:6/377: mkdir d17/d19/d27/d6e 0 2024-03-18T14:50:55.066 INFO:tasks.workunit.client.1.smithi100.stdout:4/30: rmdir d9 39 2024-03-18T14:50:55.067 INFO:tasks.workunit.client.1.smithi100.stdout:6/34: read f4 [7048456,110417] 0 2024-03-18T14:50:55.067 INFO:tasks.workunit.client.1.smithi100.stdout:9/36: mkdir da/db 0 2024-03-18T14:50:55.067 INFO:tasks.workunit.client.1.smithi100.stdout:9/37: write f6 [21973,11507] 0 2024-03-18T14:50:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:3/336: getdents dc/d30/d3a 0 2024-03-18T14:50:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:3/337: chown dc/d30/d3a 11 1 2024-03-18T14:50:55.069 INFO:tasks.workunit.client.1.smithi100.stdout:2/41: symlink d3/d13/l14 0 2024-03-18T14:50:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:1/344: sync 2024-03-18T14:50:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:8/512: symlink d0/d12/d33/d50/la7 0 2024-03-18T14:50:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:8/513: chown d0/d5a/d19/d57/f68 837434 1 2024-03-18T14:50:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:8/514: fdatasync d0/d15/f43 0 2024-03-18T14:50:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:8/515: write d0/d15/d6e/d84/d2a/d3e/f5f [633650,21621] 0 2024-03-18T14:50:55.074 INFO:tasks.workunit.client.1.smithi100.stdout:4/31: creat d9/fb x:0 0 0 2024-03-18T14:50:55.076 INFO:tasks.workunit.client.1.smithi100.stdout:9/38: dread f6 [0,4194304] 0 2024-03-18T14:50:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:9/431: rename d3/d9/d19/d77/d52/l84 to d3/l8a 0 2024-03-18T14:50:55.077 INFO:tasks.workunit.client.1.smithi100.stdout:6/35: mkdir d6 0 2024-03-18T14:50:55.080 INFO:tasks.workunit.client.1.smithi100.stdout:9/39: rmdir da 39 2024-03-18T14:50:55.081 INFO:tasks.workunit.client.1.smithi100.stdout:9/40: write f4 [528700,68501] 0 2024-03-18T14:50:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:6/378: link d17/d19/d27/d2f/f62 d17/d19/d27/d58/f6f 0 2024-03-18T14:50:55.083 INFO:tasks.workunit.client.1.smithi100.stdout:6/36: mkdir d6/d7 0 2024-03-18T14:50:55.085 INFO:tasks.workunit.client.1.smithi100.stdout:7/56: sync 2024-03-18T14:50:55.085 INFO:tasks.workunit.client.1.smithi100.stdout:7/57: chown l2 6785 1 2024-03-18T14:50:55.085 INFO:tasks.workunit.client.0.smithi012.stdout:3/338: creat dc/d30/d35/f66 x:0 0 0 2024-03-18T14:50:55.086 INFO:tasks.workunit.client.0.smithi012.stdout:1/345: creat d8/db/d10/d31/d45/f6d x:0 0 0 2024-03-18T14:50:55.086 INFO:tasks.workunit.client.0.smithi012.stdout:1/346: creat d8/db/d10/d31/f6e x:0 0 0 2024-03-18T14:50:55.087 INFO:tasks.workunit.client.1.smithi100.stdout:0/69: dwrite f5 [0,4194304] 0 2024-03-18T14:50:55.088 INFO:tasks.workunit.client.1.smithi100.stdout:6/37: creat d6/d7/f8 x:0 0 0 2024-03-18T14:50:55.088 INFO:tasks.workunit.client.1.smithi100.stdout:6/38: write f4 [7036176,37199] 0 2024-03-18T14:50:55.088 INFO:tasks.workunit.client.1.smithi100.stdout:6/39: creat d6/d7/f9 x:0 0 0 2024-03-18T14:50:55.091 INFO:tasks.workunit.client.0.smithi012.stdout:4/391: rename d6/d1a/d30/d66/c68 to d6/d1a/d30/c86 0 2024-03-18T14:50:55.092 INFO:tasks.workunit.client.1.smithi100.stdout:7/58: mknod d5/d14/c1b 0 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/432: mkdir d3/d9/d19/d66/d21/d35/d48/d54/d8b 0 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/433: dread - d3/d49/f57 zero size 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/434: read - d3/d9/d19/d77/d52/f60 zero size 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/435: readlink d3/d9/d19/d77/d52/l55 0 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/436: readlink d3/d9/d19/d66/d21/d35/d48/l2d 0 2024-03-18T14:50:55.094 INFO:tasks.workunit.client.0.smithi012.stdout:9/437: fsync d3/d9/f39 0 2024-03-18T14:50:55.097 INFO:tasks.workunit.client.0.smithi012.stdout:6/379: symlink d17/d38/d3f/l70 0 2024-03-18T14:50:55.097 INFO:tasks.workunit.client.0.smithi012.stdout:6/380: write d17/d19/f20 [4869290,55439] 0 2024-03-18T14:50:55.097 INFO:tasks.workunit.client.0.smithi012.stdout:6/381: fdatasync f12 0 2024-03-18T14:50:55.097 INFO:tasks.workunit.client.1.smithi100.stdout:0/70: write f7 [3206376,107105] 0 2024-03-18T14:50:55.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/71: chown c0 2 1 2024-03-18T14:50:55.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/72: creat dd/f1b x:0 0 0 2024-03-18T14:50:55.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/73: creat dd/f1c x:0 0 0 2024-03-18T14:50:55.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/74: stat f5 0 2024-03-18T14:50:55.102 INFO:tasks.workunit.client.1.smithi100.stdout:9/41: dwrite f5 [0,4194304] 0 2024-03-18T14:50:55.105 INFO:tasks.workunit.client.0.smithi012.stdout:1/347: unlink d8/db/d10/c3a 0 2024-03-18T14:50:55.105 INFO:tasks.workunit.client.0.smithi012.stdout:1/348: creat d8/db/d10/d31/d45/f6f x:0 0 0 2024-03-18T14:50:55.106 INFO:tasks.workunit.client.0.smithi012.stdout:1/349: chown d8/d1e/f63 14 1 2024-03-18T14:50:55.106 INFO:tasks.workunit.client.0.smithi012.stdout:1/350: dread - d8/db/d10/d16/d1c/d2e/f4b zero size 2024-03-18T14:50:55.106 INFO:tasks.workunit.client.0.smithi012.stdout:1/351: fdatasync d8/d1e/d27/d44/f4f 0 2024-03-18T14:50:55.106 INFO:tasks.workunit.client.0.smithi012.stdout:1/352: chown d8/db/d10/d31/d43/f69 286 1 2024-03-18T14:50:55.108 INFO:tasks.workunit.client.1.smithi100.stdout:8/58: write d4/f6 [1113267,112433] 0 2024-03-18T14:50:55.109 INFO:tasks.workunit.client.1.smithi100.stdout:7/59: dwrite d5/fa [0,4194304] 0 2024-03-18T14:50:55.115 INFO:tasks.workunit.client.0.smithi012.stdout:4/392: link d6/d19/d39/f50 d6/d1a/d30/d46/d6f/d7c/f87 0 2024-03-18T14:50:55.115 INFO:tasks.workunit.client.0.smithi012.stdout:4/393: creat d6/d19/d6c/f88 x:0 0 0 2024-03-18T14:50:55.115 INFO:tasks.workunit.client.1.smithi100.stdout:5/50: sync 2024-03-18T14:50:55.116 INFO:tasks.workunit.client.1.smithi100.stdout:1/58: sync 2024-03-18T14:50:55.116 INFO:tasks.workunit.client.1.smithi100.stdout:3/41: rename f1 to d3/f6 0 2024-03-18T14:50:55.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/394: sync 2024-03-18T14:50:55.120 INFO:tasks.workunit.client.0.smithi012.stdout:5/421: sync 2024-03-18T14:50:55.120 INFO:tasks.workunit.client.0.smithi012.stdout:2/395: write d0/f7b [354812,67573] 0 2024-03-18T14:50:55.120 INFO:tasks.workunit.client.0.smithi012.stdout:5/422: creat d3/d1e/f8c x:0 0 0 2024-03-18T14:50:55.120 INFO:tasks.workunit.client.0.smithi012.stdout:5/423: readlink d3/d15/l44 0 2024-03-18T14:50:55.123 INFO:tasks.workunit.client.1.smithi100.stdout:2/42: rename d3/l5 to d3/da/de/l15 0 2024-03-18T14:50:55.123 INFO:tasks.workunit.client.1.smithi100.stdout:2/43: read - d3/d6/f12 zero size 2024-03-18T14:50:55.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/60: write d5/fa [3063851,51387] 0 2024-03-18T14:50:55.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/61: chown d5/ce 112 1 2024-03-18T14:50:55.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/62: stat d5/ce 0 2024-03-18T14:50:55.125 INFO:tasks.workunit.client.0.smithi012.stdout:2/396: dread d0/d10/d18/d4f/f60 [0,4194304] 0 2024-03-18T14:50:55.125 INFO:tasks.workunit.client.1.smithi100.stdout:8/59: write d4/f6 [945993,108637] 0 2024-03-18T14:50:55.126 INFO:tasks.workunit.client.1.smithi100.stdout:5/51: rename l9 to lc 0 2024-03-18T14:50:55.126 INFO:tasks.workunit.client.1.smithi100.stdout:5/52: chown f5 12411278 1 2024-03-18T14:50:55.129 INFO:tasks.workunit.client.0.smithi012.stdout:8/516: rename d0/d15/d78 to d0/d5a/d19/d63/da8 0 2024-03-18T14:50:55.129 INFO:tasks.workunit.client.1.smithi100.stdout:1/59: rmdir d8 39 2024-03-18T14:50:55.132 INFO:tasks.workunit.client.1.smithi100.stdout:4/32: rename c2 to d9/cc 0 2024-03-18T14:50:55.132 INFO:tasks.workunit.client.1.smithi100.stdout:6/40: rename d6 to d6/d7/da 22 2024-03-18T14:50:55.132 INFO:tasks.workunit.client.0.smithi012.stdout:5/424: mknod d3/d29/d36/c8d 0 2024-03-18T14:50:55.133 INFO:tasks.workunit.client.0.smithi012.stdout:2/397: read d0/d10/d18/d31/d4c/d59/f69 [5157306,118817] 0 2024-03-18T14:50:55.134 INFO:tasks.workunit.client.1.smithi100.stdout:2/44: creat d3/d13/f16 x:0 0 0 2024-03-18T14:50:55.134 INFO:tasks.workunit.client.1.smithi100.stdout:2/45: creat d3/f17 x:0 0 0 2024-03-18T14:50:55.134 INFO:tasks.workunit.client.1.smithi100.stdout:2/46: write d3/f17 [853257,58822] 0 2024-03-18T14:50:55.134 INFO:tasks.workunit.client.0.smithi012.stdout:9/438: dwrite d3/d9/d19/d66/d69/d7f/f82 [0,4194304] 0 2024-03-18T14:50:55.135 INFO:tasks.workunit.client.1.smithi100.stdout:3/42: mknod d3/c7 0 2024-03-18T14:50:55.136 INFO:tasks.workunit.client.0.smithi012.stdout:1/353: dwrite d8/d1e/d27/d44/f4f [0,4194304] 0 2024-03-18T14:50:55.136 INFO:tasks.workunit.client.0.smithi012.stdout:1/354: chown d8/db/d10/d16 349254290 1 2024-03-18T14:50:55.136 INFO:tasks.workunit.client.1.smithi100.stdout:0/75: dwrite dd/f1c [0,4194304] 0 2024-03-18T14:50:55.137 INFO:tasks.workunit.client.1.smithi100.stdout:0/76: stat dd/c15 0 2024-03-18T14:50:55.137 INFO:tasks.workunit.client.1.smithi100.stdout:0/77: creat dd/f1d x:0 0 0 2024-03-18T14:50:55.137 INFO:tasks.workunit.client.1.smithi100.stdout:3/43: dread d3/f6 [0,4194304] 0 2024-03-18T14:50:55.137 INFO:tasks.workunit.client.1.smithi100.stdout:3/44: read - d3/f5 zero size 2024-03-18T14:50:55.138 INFO:tasks.workunit.client.0.smithi012.stdout:8/517: unlink d0/d5a/d55/f37 0 2024-03-18T14:50:55.138 INFO:tasks.workunit.client.0.smithi012.stdout:8/518: fsync d0/d5a/d19/fa3 0 2024-03-18T14:50:55.140 INFO:tasks.workunit.client.1.smithi100.stdout:8/60: mknod d4/dc/dd/c14 0 2024-03-18T14:50:55.140 INFO:tasks.workunit.client.1.smithi100.stdout:8/61: creat d4/f15 x:0 0 0 2024-03-18T14:50:55.143 INFO:tasks.workunit.client.0.smithi012.stdout:5/425: symlink d3/d29/d36/l8e 0 2024-03-18T14:50:55.145 INFO:tasks.workunit.client.1.smithi100.stdout:8/62: dread f3 [0,4194304] 0 2024-03-18T14:50:55.147 INFO:tasks.workunit.client.0.smithi012.stdout:8/519: dread d0/d15/d6e/d84/f77 [0,4194304] 0 2024-03-18T14:50:55.147 INFO:tasks.workunit.client.0.smithi012.stdout:8/520: read - d0/d5a/fa2 zero size 2024-03-18T14:50:55.149 INFO:tasks.workunit.client.0.smithi012.stdout:1/355: dread d8/db/f15 [0,4194304] 0 2024-03-18T14:50:55.149 INFO:tasks.workunit.client.0.smithi012.stdout:1/356: chown d8/db/d10/d16/c22 513808 1 2024-03-18T14:50:55.150 INFO:tasks.workunit.client.1.smithi100.stdout:1/60: unlink d8/lc 0 2024-03-18T14:50:55.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/426: write d3/d29/d36/f45 [5617901,12412] 0 2024-03-18T14:50:55.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/427: write d3/f3a [2746593,84239] 0 2024-03-18T14:50:55.153 INFO:tasks.workunit.client.0.smithi012.stdout:2/398: unlink d0/d10/d18/d31/l52 0 2024-03-18T14:50:55.153 INFO:tasks.workunit.client.0.smithi012.stdout:2/399: truncate d0/d10/d18/d4f/f6f 411672 0 2024-03-18T14:50:55.153 INFO:tasks.workunit.client.0.smithi012.stdout:2/400: readlink d0/d10/d18/d31/l34 0 2024-03-18T14:50:55.155 INFO:tasks.workunit.client.1.smithi100.stdout:9/42: rename f5 to da/fc 0 2024-03-18T14:50:55.156 INFO:tasks.workunit.client.0.smithi012.stdout:4/394: getdents d6/d1a/d30/d66 0 2024-03-18T14:50:55.156 INFO:tasks.workunit.client.0.smithi012.stdout:4/395: write d6/d1a/d30/d46/d6f/d7c/f81 [4646711,52658] 0 2024-03-18T14:50:55.156 INFO:tasks.workunit.client.0.smithi012.stdout:4/396: getdents d6/d1a/d30/d66 0 2024-03-18T14:50:55.158 INFO:tasks.workunit.client.0.smithi012.stdout:6/382: rename d17/d18/l30 to d17/d38/d51/d63/l71 0 2024-03-18T14:50:55.158 INFO:tasks.workunit.client.0.smithi012.stdout:6/383: readlink d17/d19/d27/d2f/l32 0 2024-03-18T14:50:55.163 INFO:tasks.workunit.client.1.smithi100.stdout:6/41: fsync d6/d7/f9 0 2024-03-18T14:50:55.175 INFO:tasks.workunit.client.0.smithi012.stdout:9/439: dwrite d3/d49/f57 [0,4194304] 0 2024-03-18T14:50:55.175 INFO:tasks.workunit.client.0.smithi012.stdout:9/440: readlink d3/d9/d19/d66/l37 0 2024-03-18T14:50:55.175 INFO:tasks.workunit.client.0.smithi012.stdout:8/521: truncate d0/d12/f3c 1483971 0 2024-03-18T14:50:55.175 INFO:tasks.workunit.client.0.smithi012.stdout:8/522: chown d0/d5a/d19 1 1 2024-03-18T14:50:55.175 INFO:tasks.workunit.client.0.smithi012.stdout:8/523: readlink d0/d15/l88 0 2024-03-18T14:50:55.176 INFO:tasks.workunit.client.0.smithi012.stdout:1/357: rmdir d8/d1e/d61 0 2024-03-18T14:50:55.177 INFO:tasks.workunit.client.0.smithi012.stdout:1/358: getdents d8/db/d10/d16/d1c 0 2024-03-18T14:50:55.177 INFO:tasks.workunit.client.0.smithi012.stdout:1/359: read - d8/db/d10/d16/f5b zero size 2024-03-18T14:50:55.177 INFO:tasks.workunit.client.0.smithi012.stdout:1/360: chown d8/d1e/d27 2646983 1 2024-03-18T14:50:55.178 INFO:tasks.workunit.client.1.smithi100.stdout:0/78: mknod dd/c1e 0 2024-03-18T14:50:55.178 INFO:tasks.workunit.client.1.smithi100.stdout:0/79: creat dd/f1f x:0 0 0 2024-03-18T14:50:55.179 INFO:tasks.workunit.client.0.smithi012.stdout:5/428: mkdir d3/d29/d36/d3d/d73/d6b/d8f 0 2024-03-18T14:50:55.180 INFO:tasks.workunit.client.1.smithi100.stdout:6/42: dwrite f4 [4194304,4194304] 0 2024-03-18T14:50:55.181 INFO:tasks.workunit.client.1.smithi100.stdout:7/63: dwrite d5/fa [0,4194304] 0 2024-03-18T14:50:55.181 INFO:tasks.workunit.client.1.smithi100.stdout:7/64: readlink l2 0 2024-03-18T14:50:55.181 INFO:tasks.workunit.client.1.smithi100.stdout:4/33: dwrite f8 [0,4194304] 0 2024-03-18T14:50:55.181 INFO:tasks.workunit.client.1.smithi100.stdout:2/47: dwrite d3/d13/f16 [0,4194304] 0 2024-03-18T14:50:55.182 INFO:tasks.workunit.client.0.smithi012.stdout:4/397: mkdir d6/da/d89 0 2024-03-18T14:50:55.183 INFO:tasks.workunit.client.0.smithi012.stdout:5/429: dread d3/d29/d36/f6d [0,4194304] 0 2024-03-18T14:50:55.183 INFO:tasks.workunit.client.0.smithi012.stdout:5/430: stat d3/d29/d36/d3d/d86 0 2024-03-18T14:50:55.186 INFO:tasks.workunit.client.1.smithi100.stdout:8/63: symlink d4/dc/dd/de/l16 0 2024-03-18T14:50:55.186 INFO:tasks.workunit.client.1.smithi100.stdout:8/64: fdatasync d4/ff 0 2024-03-18T14:50:55.186 INFO:tasks.workunit.client.1.smithi100.stdout:8/65: write f3 [687900,109753] 0 2024-03-18T14:50:55.186 INFO:tasks.workunit.client.1.smithi100.stdout:8/66: chown d4/dc/c10 432436153 1 2024-03-18T14:50:55.186 INFO:tasks.workunit.client.1.smithi100.stdout:8/67: dread - d4/fa zero size 2024-03-18T14:50:55.191 INFO:tasks.workunit.client.0.smithi012.stdout:2/401: rename d0/f53 to d0/d10/d18/d4f/f7c 0 2024-03-18T14:50:55.191 INFO:tasks.workunit.client.0.smithi012.stdout:2/402: fdatasync d0/d10/d18/d4f/f60 0 2024-03-18T14:50:55.193 INFO:tasks.workunit.client.1.smithi100.stdout:1/61: mkdir d8/dd 0 2024-03-18T14:50:55.200 INFO:tasks.workunit.client.1.smithi100.stdout:9/43: mkdir da/dd 0 2024-03-18T14:50:55.202 INFO:tasks.workunit.client.0.smithi012.stdout:9/441: truncate d3/f41 102728 0 2024-03-18T14:50:55.207 INFO:tasks.workunit.client.0.smithi012.stdout:8/524: symlink d0/d15/la9 0 2024-03-18T14:50:55.210 INFO:tasks.workunit.client.0.smithi012.stdout:1/361: mkdir d8/db/d10/d31/d43/d5e/d65/d70 0 2024-03-18T14:50:55.215 INFO:tasks.workunit.client.0.smithi012.stdout:1/362: dread d8/db/f15 [0,4194304] 0 2024-03-18T14:50:55.215 INFO:tasks.workunit.client.0.smithi012.stdout:1/363: chown d8/db/fd 262427 1 2024-03-18T14:50:55.217 INFO:tasks.workunit.client.0.smithi012.stdout:9/442: dread d3/d20/f5f [0,4194304] 0 2024-03-18T14:50:55.217 INFO:tasks.workunit.client.1.smithi100.stdout:3/45: rmdir d3 39 2024-03-18T14:50:55.222 INFO:tasks.workunit.client.0.smithi012.stdout:2/403: mknod d0/d10/d18/d57/c7d 0 2024-03-18T14:50:55.222 INFO:tasks.workunit.client.0.smithi012.stdout:0/403: sync 2024-03-18T14:50:55.223 INFO:tasks.workunit.client.0.smithi012.stdout:7/423: sync 2024-03-18T14:50:55.223 INFO:tasks.workunit.client.0.smithi012.stdout:7/424: write dd/d13/d23/d2f/f32 [1492776,58556] 0 2024-03-18T14:50:55.226 INFO:tasks.workunit.client.0.smithi012.stdout:8/525: mkdir d0/d5a/d19/d57/daa 0 2024-03-18T14:50:55.230 INFO:tasks.workunit.client.0.smithi012.stdout:2/404: link d0/d10/d18/l26 d0/d10/d18/d31/d4c/d59/l7e 0 2024-03-18T14:50:55.231 INFO:tasks.workunit.client.1.smithi100.stdout:2/48: link d3/d6/c8 d3/c18 0 2024-03-18T14:50:55.234 INFO:tasks.workunit.client.1.smithi100.stdout:8/68: creat d4/dc/dd/de/f17 x:0 0 0 2024-03-18T14:50:55.236 INFO:tasks.workunit.client.0.smithi012.stdout:8/526: unlink d0/l3b 0 2024-03-18T14:50:55.237 INFO:tasks.workunit.client.1.smithi100.stdout:7/65: dwrite d5/f10 [0,4194304] 0 2024-03-18T14:50:55.240 INFO:tasks.workunit.client.1.smithi100.stdout:2/49: rename d3/d13/f16 to d3/da/de/f19 0 2024-03-18T14:50:55.241 INFO:tasks.workunit.client.0.smithi012.stdout:0/404: mkdir d1/d16/d19/d8d 0 2024-03-18T14:50:55.243 INFO:tasks.workunit.client.1.smithi100.stdout:0/80: dwrite f5 [0,4194304] 0 2024-03-18T14:50:55.243 INFO:tasks.workunit.client.1.smithi100.stdout:0/81: truncate dd/f1b 988270 0 2024-03-18T14:50:55.243 INFO:tasks.workunit.client.1.smithi100.stdout:8/69: symlink d4/l18 0 2024-03-18T14:50:55.243 INFO:tasks.workunit.client.1.smithi100.stdout:8/70: truncate d4/ff 1010824 0 2024-03-18T14:50:55.244 INFO:tasks.workunit.client.0.smithi012.stdout:8/527: truncate d0/d15/f28 4540556 0 2024-03-18T14:50:55.245 INFO:tasks.workunit.client.1.smithi100.stdout:3/46: mkdir d3/d8 0 2024-03-18T14:50:55.246 INFO:tasks.workunit.client.1.smithi100.stdout:3/47: stat d3/d8 0 2024-03-18T14:50:55.246 INFO:tasks.workunit.client.1.smithi100.stdout:3/48: truncate d3/f5 811755 0 2024-03-18T14:50:55.247 INFO:tasks.workunit.client.0.smithi012.stdout:0/405: mknod d1/d20/d23/d82/c8e 0 2024-03-18T14:50:55.248 INFO:tasks.workunit.client.1.smithi100.stdout:2/50: mkdir d3/d1a 0 2024-03-18T14:50:55.249 INFO:tasks.workunit.client.1.smithi100.stdout:2/51: write d3/d6/f9 [386316,37575] 0 2024-03-18T14:50:55.251 INFO:tasks.workunit.client.0.smithi012.stdout:0/406: creat d1/d16/d79/f8f x:0 0 0 2024-03-18T14:50:55.251 INFO:tasks.workunit.client.0.smithi012.stdout:0/407: truncate d1/f10 1400451 0 2024-03-18T14:50:55.251 INFO:tasks.workunit.client.0.smithi012.stdout:0/408: creat d1/d20/d23/d26/f90 x:0 0 0 2024-03-18T14:50:55.252 INFO:tasks.workunit.client.1.smithi100.stdout:0/82: mkdir dd/d20 0 2024-03-18T14:50:55.252 INFO:tasks.workunit.client.0.smithi012.stdout:5/431: dwrite d3/d1e/d33/d89/f71 [0,4194304] 0 2024-03-18T14:50:55.252 INFO:tasks.workunit.client.0.smithi012.stdout:5/432: creat d3/d15/f90 x:0 0 0 2024-03-18T14:50:55.253 INFO:tasks.workunit.client.1.smithi100.stdout:8/71: mkdir d4/dc/dd/de/d19 0 2024-03-18T14:50:55.255 INFO:tasks.workunit.client.1.smithi100.stdout:3/49: creat d3/f9 x:0 0 0 2024-03-18T14:50:55.256 INFO:tasks.workunit.client.0.smithi012.stdout:0/409: chown d1/d16/d19/c3d 19 1 2024-03-18T14:50:55.257 INFO:tasks.workunit.client.1.smithi100.stdout:1/62: dwrite f2 [0,4194304] 0 2024-03-18T14:50:55.258 INFO:tasks.workunit.client.1.smithi100.stdout:0/83: unlink dd/c13 0 2024-03-18T14:50:55.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/398: dwrite d6/da/d58/f59 [0,4194304] 0 2024-03-18T14:50:55.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/399: read d6/f28 [110777,115363] 0 2024-03-18T14:50:55.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/400: write d6/d19/d4f/f5e [4276668,38514] 0 2024-03-18T14:50:55.261 INFO:tasks.workunit.client.1.smithi100.stdout:8/72: symlink d4/dc/dd/l1a 0 2024-03-18T14:50:55.263 INFO:tasks.workunit.client.1.smithi100.stdout:3/50: truncate d3/f6 1300995 0 2024-03-18T14:50:55.264 INFO:tasks.workunit.client.0.smithi012.stdout:0/410: chown d1/d16/d19/f68 15 1 2024-03-18T14:50:55.266 INFO:tasks.workunit.client.1.smithi100.stdout:1/63: mkdir d8/de 0 2024-03-18T14:50:55.268 INFO:tasks.workunit.client.1.smithi100.stdout:8/73: dread d4/f6 [0,4194304] 0 2024-03-18T14:50:55.270 INFO:tasks.workunit.client.0.smithi012.stdout:4/401: symlink d6/d19/d39/l8a 0 2024-03-18T14:50:55.270 INFO:tasks.workunit.client.0.smithi012.stdout:4/402: write d6/d19/f51 [436248,75047] 0 2024-03-18T14:50:55.271 INFO:tasks.workunit.client.1.smithi100.stdout:9/44: dwrite f7 [0,4194304] 0 2024-03-18T14:50:55.271 INFO:tasks.workunit.client.1.smithi100.stdout:4/34: dwrite f0 [0,4194304] 0 2024-03-18T14:50:55.271 INFO:tasks.workunit.client.1.smithi100.stdout:4/35: dread - d9/fb zero size 2024-03-18T14:50:55.272 INFO:tasks.workunit.client.0.smithi012.stdout:0/411: symlink d1/d20/d23/d3b/d33/l91 0 2024-03-18T14:50:55.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/403: mkdir d6/d1a/d30/d46/d6f/d8b 0 2024-03-18T14:50:55.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/404: chown d6/d19/d4f 4636460 1 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/405: rmdir d6/d1a/d64 39 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/406: fdatasync d6/d19/d6c/f7a 0 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/407: chown d6/d1a/d30/d46/d6f/f71 5 1 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/408: stat d6/d27/c4a 0 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/409: readlink d6/l3a 0 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/410: write d6/f80 [25515,35691] 0 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/411: stat d6/d1a/d30/d46/d6f/d7c 0 2024-03-18T14:50:55.276 INFO:tasks.workunit.client.0.smithi012.stdout:4/412: truncate d6/d19/d39/f40 1677616 0 2024-03-18T14:50:55.277 INFO:tasks.workunit.client.1.smithi100.stdout:0/84: write dd/ff [1818561,70583] 0 2024-03-18T14:50:55.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/51: creat d3/fa x:0 0 0 2024-03-18T14:50:55.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/52: dread - d3/fa zero size 2024-03-18T14:50:55.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/53: write d3/f9 [290589,23413] 0 2024-03-18T14:50:55.281 INFO:tasks.workunit.client.0.smithi012.stdout:0/412: dread d1/d20/d7b/f81 [0,4194304] 0 2024-03-18T14:50:55.281 INFO:tasks.workunit.client.0.smithi012.stdout:0/413: write d1/d20/d23/d26/f53 [2951914,114770] 0 2024-03-18T14:50:55.282 INFO:tasks.workunit.client.0.smithi012.stdout:4/413: rmdir d6/d1a/d30/d66 0 2024-03-18T14:50:55.283 INFO:tasks.workunit.client.1.smithi100.stdout:4/36: dread f8 [0,4194304] 0 2024-03-18T14:50:55.284 INFO:tasks.workunit.client.1.smithi100.stdout:9/45: symlink da/dd/le 0 2024-03-18T14:50:55.284 INFO:tasks.workunit.client.1.smithi100.stdout:1/64: write f7 [1971385,68824] 0 2024-03-18T14:50:55.285 INFO:tasks.workunit.client.0.smithi012.stdout:9/443: dwrite d3/d9/f6c [0,4194304] 0 2024-03-18T14:50:55.285 INFO:tasks.workunit.client.0.smithi012.stdout:7/425: dwrite dd/d13/d23/d33/f78 [0,4194304] 0 2024-03-18T14:50:55.285 INFO:tasks.workunit.client.1.smithi100.stdout:0/85: creat dd/d20/f21 x:0 0 0 2024-03-18T14:50:55.286 INFO:tasks.workunit.client.1.smithi100.stdout:3/54: mknod d3/d8/cb 0 2024-03-18T14:50:55.288 INFO:tasks.workunit.client.1.smithi100.stdout:9/46: mknod da/cf 0 2024-03-18T14:50:55.288 INFO:tasks.workunit.client.1.smithi100.stdout:9/47: write f6 [360551,119875] 0 2024-03-18T14:50:55.290 INFO:tasks.workunit.client.0.smithi012.stdout:9/444: rename d3/d9/d19/d66/d21/d35/d48/d54 to d3/d9/d19/d66/d69/d7f/d8c 0 2024-03-18T14:50:55.290 INFO:tasks.workunit.client.0.smithi012.stdout:9/445: chown d3/f64 24 1 2024-03-18T14:50:55.293 INFO:tasks.workunit.client.1.smithi100.stdout:4/37: write f0 [4843277,68003] 0 2024-03-18T14:50:55.294 INFO:tasks.workunit.client.0.smithi012.stdout:2/405: dwrite d0/f55 [0,4194304] 0 2024-03-18T14:50:55.295 INFO:tasks.workunit.client.0.smithi012.stdout:8/528: dwrite d0/d15/f38 [0,4194304] 0 2024-03-18T14:50:55.295 INFO:tasks.workunit.client.0.smithi012.stdout:8/529: chown d0/d12/d33/d50/fa5 103 1 2024-03-18T14:50:55.295 INFO:tasks.workunit.client.0.smithi012.stdout:8/530: write d0/d12/f46 [4214183,64462] 0 2024-03-18T14:50:55.295 INFO:tasks.workunit.client.0.smithi012.stdout:7/426: creat dd/d13/d23/d76/d6f/f91 x:0 0 0 2024-03-18T14:50:55.296 INFO:tasks.workunit.client.1.smithi100.stdout:2/52: dwrite d3/da/de/ff [0,4194304] 0 2024-03-18T14:50:55.296 INFO:tasks.workunit.client.1.smithi100.stdout:2/53: creat d3/f1b x:0 0 0 2024-03-18T14:50:55.296 INFO:tasks.workunit.client.0.smithi012.stdout:8/531: write d0/f10 [1095981,22470] 0 2024-03-18T14:50:55.297 INFO:tasks.workunit.client.0.smithi012.stdout:8/532: chown d0/d5a/f8b 127489627 1 2024-03-18T14:50:55.297 INFO:tasks.workunit.client.0.smithi012.stdout:2/406: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:55.298 INFO:tasks.workunit.client.1.smithi100.stdout:1/65: write f2 [3901847,3244] 0 2024-03-18T14:50:55.298 INFO:tasks.workunit.client.1.smithi100.stdout:1/66: stat l3 0 2024-03-18T14:50:55.299 INFO:tasks.workunit.client.1.smithi100.stdout:9/48: rename f4 to da/dd/f10 0 2024-03-18T14:50:55.300 INFO:tasks.workunit.client.0.smithi012.stdout:0/414: write d1/d20/d23/d3b/f2c [2146540,92485] 0 2024-03-18T14:50:55.300 INFO:tasks.workunit.client.0.smithi012.stdout:1/364: dwrite d8/db/d10/d16/d1c/f2c [0,4194304] 0 2024-03-18T14:50:55.300 INFO:tasks.workunit.client.0.smithi012.stdout:0/415: chown d1/d16/d31 0 1 2024-03-18T14:50:55.300 INFO:tasks.workunit.client.0.smithi012.stdout:5/433: dwrite d3/d15/d5f/f70 [0,4194304] 0 2024-03-18T14:50:55.301 INFO:tasks.workunit.client.0.smithi012.stdout:8/533: dread d0/d12/f7c [0,4194304] 0 2024-03-18T14:50:55.301 INFO:tasks.workunit.client.0.smithi012.stdout:8/534: write d0/d12/f20 [3578096,109443] 0 2024-03-18T14:50:55.301 INFO:tasks.workunit.client.0.smithi012.stdout:8/535: write d0/d5a/d19/d57/f69 [35172,11864] 0 2024-03-18T14:50:55.306 INFO:tasks.workunit.client.0.smithi012.stdout:7/427: symlink dd/d13/d23/l92 0 2024-03-18T14:50:55.313 INFO:tasks.workunit.client.0.smithi012.stdout:2/407: write d0/d3c/d40/d47/f49 [518381,17617] 0 2024-03-18T14:50:55.313 INFO:tasks.workunit.client.0.smithi012.stdout:2/408: chown d0/c45 83111064 1 2024-03-18T14:50:55.313 INFO:tasks.workunit.client.0.smithi012.stdout:2/409: chown d0/d10/d18/d4f/f7c 74595418 1 2024-03-18T14:50:55.316 INFO:tasks.workunit.client.1.smithi100.stdout:1/67: creat d8/dd/ff x:0 0 0 2024-03-18T14:50:55.318 INFO:tasks.workunit.client.1.smithi100.stdout:0/86: dwrite dd/f16 [0,4194304] 0 2024-03-18T14:50:55.319 INFO:tasks.workunit.client.1.smithi100.stdout:4/38: rename c7 to d9/cd 0 2024-03-18T14:50:55.321 INFO:tasks.workunit.client.0.smithi012.stdout:7/428: dread dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:55.323 INFO:tasks.workunit.client.1.smithi100.stdout:9/49: unlink da/dd/le 0 2024-03-18T14:50:55.325 INFO:tasks.workunit.client.1.smithi100.stdout:2/54: getdents d3/d13 0 2024-03-18T14:50:55.325 INFO:tasks.workunit.client.1.smithi100.stdout:2/55: fdatasync d3/d6/f9 0 2024-03-18T14:50:55.327 INFO:tasks.workunit.client.1.smithi100.stdout:1/68: mkdir d8/dd/d10 0 2024-03-18T14:50:55.330 INFO:tasks.workunit.client.1.smithi100.stdout:3/55: dwrite d3/f9 [0,4194304] 0 2024-03-18T14:50:55.331 INFO:tasks.workunit.client.0.smithi012.stdout:9/446: dwrite d3/d9/d19/d66/d21/f26 [0,4194304] 0 2024-03-18T14:50:55.331 INFO:tasks.workunit.client.0.smithi012.stdout:9/447: write d3/d9/d5c/f85 [716196,67356] 0 2024-03-18T14:50:55.336 INFO:tasks.workunit.client.0.smithi012.stdout:1/365: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:50:55.336 INFO:tasks.workunit.client.1.smithi100.stdout:0/87: symlink dd/l22 0 2024-03-18T14:50:55.341 INFO:tasks.workunit.client.1.smithi100.stdout:0/88: truncate f3 3958403 0 2024-03-18T14:50:55.342 INFO:tasks.workunit.client.1.smithi100.stdout:2/56: rmdir d3/da/de 39 2024-03-18T14:50:55.343 INFO:tasks.workunit.client.1.smithi100.stdout:2/57: chown l1 5979951 1 2024-03-18T14:50:55.344 INFO:tasks.workunit.client.0.smithi012.stdout:0/416: link d1/d20/d23/d3b/d33/c42 d1/d20/c92 0 2024-03-18T14:50:55.345 INFO:tasks.workunit.client.0.smithi012.stdout:0/417: fdatasync d1/d20/d7b/f30 0 2024-03-18T14:50:55.347 INFO:tasks.workunit.client.1.smithi100.stdout:0/89: fdatasync dd/f1f 0 2024-03-18T14:50:55.347 INFO:tasks.workunit.client.1.smithi100.stdout:0/90: truncate f2 4812644 0 2024-03-18T14:50:55.347 INFO:tasks.workunit.client.1.smithi100.stdout:2/58: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:55.349 INFO:tasks.workunit.client.0.smithi012.stdout:5/434: link d3/d1e/l18 d3/d29/d36/l91 0 2024-03-18T14:50:55.351 INFO:tasks.workunit.client.1.smithi100.stdout:9/50: dwrite f9 [0,4194304] 0 2024-03-18T14:50:55.351 INFO:tasks.workunit.client.1.smithi100.stdout:3/56: dwrite d3/fa [0,4194304] 0 2024-03-18T14:50:55.352 INFO:tasks.workunit.client.1.smithi100.stdout:4/39: dwrite f8 [0,4194304] 0 2024-03-18T14:50:55.352 INFO:tasks.workunit.client.1.smithi100.stdout:4/40: fsync f0 0 2024-03-18T14:50:55.352 INFO:tasks.workunit.client.0.smithi012.stdout:8/536: truncate d0/d15/d6e/d84/d54/f72 3140990 0 2024-03-18T14:50:55.358 INFO:tasks.workunit.client.0.smithi012.stdout:2/410: rename d0/c19 to d0/d10/d18/d33/d71/c7f 0 2024-03-18T14:50:55.362 INFO:tasks.workunit.client.0.smithi012.stdout:7/429: symlink dd/l93 0 2024-03-18T14:50:55.363 INFO:tasks.workunit.client.1.smithi100.stdout:4/41: dread f8 [0,4194304] 0 2024-03-18T14:50:55.363 INFO:tasks.workunit.client.1.smithi100.stdout:4/42: creat d9/fe x:0 0 0 2024-03-18T14:50:55.363 INFO:tasks.workunit.client.1.smithi100.stdout:0/91: dread dd/f11 [0,4194304] 0 2024-03-18T14:50:55.372 INFO:tasks.workunit.client.1.smithi100.stdout:3/57: dwrite d3/f6 [0,4194304] 0 2024-03-18T14:50:55.372 INFO:tasks.workunit.client.0.smithi012.stdout:9/448: mknod d3/d9/d19/d66/d21/d35/d48/c8d 0 2024-03-18T14:50:55.372 INFO:tasks.workunit.client.1.smithi100.stdout:1/69: dwrite f7 [4194304,4194304] 0 2024-03-18T14:50:55.374 INFO:tasks.workunit.client.0.smithi012.stdout:2/411: dread d0/d10/d18/d31/d4c/f67 [0,4194304] 0 2024-03-18T14:50:55.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/43: symlink d9/lf 0 2024-03-18T14:50:55.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/44: creat d9/f10 x:0 0 0 2024-03-18T14:50:55.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/45: creat d9/f11 x:0 0 0 2024-03-18T14:50:55.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/46: chown d9/fb 5721 1 2024-03-18T14:50:55.379 INFO:tasks.workunit.client.1.smithi100.stdout:9/51: dwrite f7 [4194304,4194304] 0 2024-03-18T14:50:55.382 INFO:tasks.workunit.client.0.smithi012.stdout:1/366: mknod d8/db/d10/d56/c71 0 2024-03-18T14:50:55.384 INFO:tasks.workunit.client.1.smithi100.stdout:3/58: creat d3/d8/fc x:0 0 0 2024-03-18T14:50:55.384 INFO:tasks.workunit.client.1.smithi100.stdout:3/59: stat d3/f6 0 2024-03-18T14:50:55.384 INFO:tasks.workunit.client.1.smithi100.stdout:3/60: readlink - no filename 2024-03-18T14:50:55.384 INFO:tasks.workunit.client.1.smithi100.stdout:3/61: write d3/d8/fc [811276,6086] 0 2024-03-18T14:50:55.387 INFO:tasks.workunit.client.0.smithi012.stdout:9/449: dwrite d3/f64 [0,4194304] 0 2024-03-18T14:50:55.390 INFO:tasks.workunit.client.0.smithi012.stdout:9/450: write d3/d20/f31 [164772,53218] 0 2024-03-18T14:50:55.391 INFO:tasks.workunit.client.1.smithi100.stdout:1/70: mkdir d8/dd/d10/d11 0 2024-03-18T14:50:55.391 INFO:tasks.workunit.client.1.smithi100.stdout:1/71: dread - d8/dd/ff zero size 2024-03-18T14:50:55.393 INFO:tasks.workunit.client.0.smithi012.stdout:0/418: mkdir d1/d20/d23/d6f/d93 0 2024-03-18T14:50:55.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/537: rename d0/d47/l6f to d0/d5a/d55/lab 0 2024-03-18T14:50:55.400 INFO:tasks.workunit.client.1.smithi100.stdout:4/47: mknod d9/c12 0 2024-03-18T14:50:55.402 INFO:tasks.workunit.client.0.smithi012.stdout:7/430: symlink dd/d13/d23/d2f/d53/l94 0 2024-03-18T14:50:55.405 INFO:tasks.workunit.client.0.smithi012.stdout:0/419: dread d1/d20/d4e/f52 [0,4194304] 0 2024-03-18T14:50:55.405 INFO:tasks.workunit.client.0.smithi012.stdout:2/412: mkdir d0/d10/d18/d33/d80 0 2024-03-18T14:50:55.410 INFO:tasks.workunit.client.1.smithi100.stdout:3/62: dwrite d3/f5 [0,4194304] 0 2024-03-18T14:50:55.411 INFO:tasks.workunit.client.0.smithi012.stdout:9/451: rmdir d3/d9/d19/d66/d69/d7f/d8c 39 2024-03-18T14:50:55.414 INFO:tasks.workunit.client.1.smithi100.stdout:9/52: mknod da/c11 0 2024-03-18T14:50:55.414 INFO:tasks.workunit.client.0.smithi012.stdout:0/420: symlink d1/d20/d23/d26/l94 0 2024-03-18T14:50:55.414 INFO:tasks.workunit.client.0.smithi012.stdout:0/421: truncate d1/d16/d31/f3f 134751 0 2024-03-18T14:50:55.415 INFO:tasks.workunit.client.0.smithi012.stdout:1/367: dwrite d8/db/d10/d31/d43/f51 [0,4194304] 0 2024-03-18T14:50:55.416 INFO:tasks.workunit.client.1.smithi100.stdout:1/72: rename d8/cb to d8/de/c12 0 2024-03-18T14:50:55.417 INFO:tasks.workunit.client.0.smithi012.stdout:2/413: unlink d0/d10/d18/c25 0 2024-03-18T14:50:55.417 INFO:tasks.workunit.client.1.smithi100.stdout:4/48: dread f0 [0,4194304] 0 2024-03-18T14:50:55.420 INFO:tasks.workunit.client.1.smithi100.stdout:3/63: read d3/f6 [3438341,58534] 0 2024-03-18T14:50:55.420 INFO:tasks.workunit.client.1.smithi100.stdout:3/64: chown d3/d8/fc 918 1 2024-03-18T14:50:55.420 INFO:tasks.workunit.client.0.smithi012.stdout:0/422: mknod d1/d16/d6e/c95 0 2024-03-18T14:50:55.420 INFO:tasks.workunit.client.0.smithi012.stdout:3/339: sync 2024-03-18T14:50:55.424 INFO:tasks.workunit.client.1.smithi100.stdout:0/92: creat dd/f23 x:0 0 0 2024-03-18T14:50:55.427 INFO:tasks.workunit.client.0.smithi012.stdout:1/368: symlink d8/db/d10/d31/d43/l72 0 2024-03-18T14:50:55.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/384: sync 2024-03-18T14:50:55.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/414: truncate d0/d10/d18/f24 1480135 0 2024-03-18T14:50:55.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/415: readlink d0/l29 0 2024-03-18T14:50:55.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/416: write d0/d3c/d40/f5e [1071748,5117] 0 2024-03-18T14:50:55.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/417: fsync d0/d10/f58 0 2024-03-18T14:50:55.433 INFO:tasks.workunit.client.1.smithi100.stdout:4/49: mkdir d9/d13 0 2024-03-18T14:50:55.435 INFO:tasks.workunit.client.0.smithi012.stdout:9/452: getdents d3/d9/d19 0 2024-03-18T14:50:55.438 INFO:tasks.workunit.client.0.smithi012.stdout:1/369: write d8/f1a [260196,88072] 0 2024-03-18T14:50:55.438 INFO:tasks.workunit.client.0.smithi012.stdout:3/340: rename dc/f26 to dc/d3c/d4c/d5f/f67 0 2024-03-18T14:50:55.444 INFO:tasks.workunit.client.1.smithi100.stdout:4/50: rename d9/c12 to d9/c14 0 2024-03-18T14:50:55.444 INFO:tasks.workunit.client.1.smithi100.stdout:4/51: getdents d9/d13 0 2024-03-18T14:50:55.444 INFO:tasks.workunit.client.1.smithi100.stdout:4/52: write d9/f11 [997512,91337] 0 2024-03-18T14:50:55.445 INFO:tasks.workunit.client.1.smithi100.stdout:4/53: chown d9/la 282 1 2024-03-18T14:50:55.447 INFO:tasks.workunit.client.1.smithi100.stdout:4/54: symlink d9/l15 0 2024-03-18T14:50:55.448 INFO:tasks.workunit.client.0.smithi012.stdout:1/370: read d8/d1e/d27/d44/f59 [3439587,40222] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:2/418: write d0/f55 [3434953,100594] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:8/538: dwrite d0/d15/d6e/d84/d2a/fa1 [0,4194304] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:8/539: chown d0/d15/d6e/d84/f30 1 1 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:2/419: write d0/f7b [1177689,72053] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:2/420: readlink d0/d10/d18/d57/l74 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:8/540: write d0/d15/d6e/f7b [615157,48721] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:7/431: dwrite dd/d13/d26/f5e [0,4194304] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:0/423: dwrite d1/d20/f2e [4194304,4194304] 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:3/341: truncate dc/f44 2816964 0 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:3/342: chown dc/d3c/f4d 1338337 1 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:3/343: chown dc/d30/d3a/f4b 250 1 2024-03-18T14:50:55.451 INFO:tasks.workunit.client.0.smithi012.stdout:3/344: dread - dc/d3c/d59/f5a zero size 2024-03-18T14:50:55.452 INFO:tasks.workunit.client.0.smithi012.stdout:6/385: dwrite d17/f1b [0,4194304] 0 2024-03-18T14:50:55.452 INFO:tasks.workunit.client.0.smithi012.stdout:6/386: fdatasync d17/d26/d36/f4b 0 2024-03-18T14:50:55.452 INFO:tasks.workunit.client.0.smithi012.stdout:6/387: stat d17/d38 0 2024-03-18T14:50:55.452 INFO:tasks.workunit.client.0.smithi012.stdout:9/453: rmdir d3/d9/d19/d77/d52/d65 39 2024-03-18T14:50:55.456 INFO:tasks.workunit.client.1.smithi100.stdout:4/55: dread f0 [4194304,4194304] 0 2024-03-18T14:50:55.456 INFO:tasks.workunit.client.1.smithi100.stdout:4/56: write d9/f11 [1656797,3134] 0 2024-03-18T14:50:55.461 INFO:tasks.workunit.client.0.smithi012.stdout:6/388: write f4 [7115330,105524] 0 2024-03-18T14:50:55.462 INFO:tasks.workunit.client.1.smithi100.stdout:4/57: dread d9/f11 [0,4194304] 0 2024-03-18T14:50:55.462 INFO:tasks.workunit.client.1.smithi100.stdout:4/58: write f0 [5272121,52908] 0 2024-03-18T14:50:55.462 INFO:tasks.workunit.client.1.smithi100.stdout:4/59: chown d9/la 395060 1 2024-03-18T14:50:55.462 INFO:tasks.workunit.client.1.smithi100.stdout:4/60: rename d9 to d9/d16 22 2024-03-18T14:50:55.466 INFO:tasks.workunit.client.0.smithi012.stdout:8/541: link d0/d15/d4e/c65 d0/d5a/d19/d63/cac 0 2024-03-18T14:50:55.466 INFO:tasks.workunit.client.0.smithi012.stdout:8/542: getdents d0/d47/d59/d67 0 2024-03-18T14:50:55.466 INFO:tasks.workunit.client.0.smithi012.stdout:8/543: write d0/d5a/d19/d57/f69 [911793,30872] 0 2024-03-18T14:50:55.467 INFO:tasks.workunit.client.1.smithi100.stdout:4/61: mkdir d9/d17 0 2024-03-18T14:50:55.468 INFO:tasks.workunit.client.0.smithi012.stdout:2/421: mknod d0/d10/d18/d31/d4c/d59/c81 0 2024-03-18T14:50:55.470 INFO:tasks.workunit.client.1.smithi100.stdout:4/62: symlink d9/d17/l18 0 2024-03-18T14:50:55.474 INFO:tasks.workunit.client.1.smithi100.stdout:4/63: creat d9/d13/f19 x:0 0 0 2024-03-18T14:50:55.475 INFO:tasks.workunit.client.0.smithi012.stdout:3/345: rename dc/d30/d35/c36 to dc/d30/d3a/c68 0 2024-03-18T14:50:55.478 INFO:tasks.workunit.client.0.smithi012.stdout:9/454: creat d3/d9/d19/d66/d21/f8e x:0 0 0 2024-03-18T14:50:55.478 INFO:tasks.workunit.client.0.smithi012.stdout:9/455: creat d3/d9/d5c/f8f x:0 0 0 2024-03-18T14:50:55.478 INFO:tasks.workunit.client.0.smithi012.stdout:9/456: creat d3/d9/d19/d66/d21/f90 x:0 0 0 2024-03-18T14:50:55.480 INFO:tasks.workunit.client.0.smithi012.stdout:6/389: mkdir d17/d38/d51/d72 0 2024-03-18T14:50:55.480 INFO:tasks.workunit.client.0.smithi012.stdout:6/390: fsync ff 0 2024-03-18T14:50:55.481 INFO:tasks.workunit.client.0.smithi012.stdout:8/544: getdents d0/d5a/d19/d63/da8/d80 0 2024-03-18T14:50:55.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/432: getdents dd 0 2024-03-18T14:50:55.484 INFO:tasks.workunit.client.0.smithi012.stdout:1/371: dwrite d8/db/d10/d16/d1c/f2c [4194304,4194304] 0 2024-03-18T14:50:55.484 INFO:tasks.workunit.client.0.smithi012.stdout:1/372: stat f5 0 2024-03-18T14:50:55.485 INFO:tasks.workunit.client.0.smithi012.stdout:2/422: dwrite d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:50:55.485 INFO:tasks.workunit.client.0.smithi012.stdout:2/423: write d0/d10/d18/f73 [42394,66649] 0 2024-03-18T14:50:55.485 INFO:tasks.workunit.client.0.smithi012.stdout:2/424: chown d0/d10/d18/d31/d4c/d59/l7e 15 1 2024-03-18T14:50:55.487 INFO:tasks.workunit.client.1.smithi100.stdout:1/73: unlink d8/de/c12 0 2024-03-18T14:50:55.487 INFO:tasks.workunit.client.1.smithi100.stdout:1/74: chown f7 14431994 1 2024-03-18T14:50:55.487 INFO:tasks.workunit.client.1.smithi100.stdout:1/75: fsync d8/dd/ff 0 2024-03-18T14:50:55.488 INFO:tasks.workunit.client.0.smithi012.stdout:3/346: truncate dc/f32 2784354 0 2024-03-18T14:50:55.488 INFO:tasks.workunit.client.0.smithi012.stdout:3/347: chown dc/d30/c46 1 1 2024-03-18T14:50:55.491 INFO:tasks.workunit.client.1.smithi100.stdout:1/76: symlink d8/dd/d10/l13 0 2024-03-18T14:50:55.493 INFO:tasks.workunit.client.0.smithi012.stdout:1/373: dread d8/db/d10/d16/d1c/f4c [0,4194304] 0 2024-03-18T14:50:55.493 INFO:tasks.workunit.client.0.smithi012.stdout:9/457: symlink d3/d9/d19/d66/d21/d35/d48/l91 0 2024-03-18T14:50:55.496 INFO:tasks.workunit.client.1.smithi100.stdout:3/65: dwrite d3/f9 [4194304,4194304] 0 2024-03-18T14:50:55.498 INFO:tasks.workunit.client.0.smithi012.stdout:6/391: link d17/d26/c46 d17/d38/c73 0 2024-03-18T14:50:55.504 INFO:tasks.workunit.client.1.smithi100.stdout:5/53: sync 2024-03-18T14:50:55.504 INFO:tasks.workunit.client.0.smithi012.stdout:7/433: creat dd/d13/d23/d60/f95 x:0 0 0 2024-03-18T14:50:55.507 INFO:tasks.workunit.client.1.smithi100.stdout:9/53: truncate da/dd/f10 4767811 0 2024-03-18T14:50:55.507 INFO:tasks.workunit.client.1.smithi100.stdout:9/54: readlink l8 0 2024-03-18T14:50:55.509 INFO:tasks.workunit.client.1.smithi100.stdout:1/77: getdents d8/dd 0 2024-03-18T14:50:55.510 INFO:tasks.workunit.client.1.smithi100.stdout:1/78: readlink l3 0 2024-03-18T14:50:55.512 INFO:tasks.workunit.client.0.smithi012.stdout:3/348: dwrite dc/f22 [0,4194304] 0 2024-03-18T14:50:55.512 INFO:tasks.workunit.client.0.smithi012.stdout:9/458: dwrite d3/d9/d19/d66/f44 [0,4194304] 0 2024-03-18T14:50:55.516 INFO:tasks.workunit.client.0.smithi012.stdout:2/425: unlink d0/d10/d18/d31/c7a 0 2024-03-18T14:50:55.516 INFO:tasks.workunit.client.0.smithi012.stdout:2/426: fsync d0/f55 0 2024-03-18T14:50:55.521 INFO:tasks.workunit.client.0.smithi012.stdout:1/374: link d8/d1e/d27/d44/f59 d8/db/d10/f73 0 2024-03-18T14:50:55.522 INFO:tasks.workunit.client.0.smithi012.stdout:1/375: write d8/db/d10/d16/d1c/d2e/f4b [557282,86988] 0 2024-03-18T14:50:55.526 INFO:tasks.workunit.client.1.smithi100.stdout:3/66: mkdir d3/dd 0 2024-03-18T14:50:55.526 INFO:tasks.workunit.client.0.smithi012.stdout:6/392: link fe d17/d38/d51/d72/f74 0 2024-03-18T14:50:55.529 INFO:tasks.workunit.client.0.smithi012.stdout:7/434: dwrite dd/d13/d23/d33/f78 [0,4194304] 0 2024-03-18T14:50:55.529 INFO:tasks.workunit.client.0.smithi012.stdout:7/435: creat dd/d13/f96 x:0 0 0 2024-03-18T14:50:55.529 INFO:tasks.workunit.client.0.smithi012.stdout:8/545: rename d0/d12 to d0/d5a/d19/dad 0 2024-03-18T14:50:55.532 INFO:tasks.workunit.client.1.smithi100.stdout:1/79: creat d8/de/f14 x:0 0 0 2024-03-18T14:50:55.532 INFO:tasks.workunit.client.1.smithi100.stdout:1/80: read - d8/fa zero size 2024-03-18T14:50:55.534 INFO:tasks.workunit.client.1.smithi100.stdout:0/93: rmdir dd/d20 39 2024-03-18T14:50:55.535 INFO:tasks.workunit.client.0.smithi012.stdout:3/349: rmdir dc/d3c/d4c/d5f 39 2024-03-18T14:50:55.538 INFO:tasks.workunit.client.1.smithi100.stdout:1/81: link d8/dd/d10/l13 d8/l15 0 2024-03-18T14:50:55.539 INFO:tasks.workunit.client.1.smithi100.stdout:1/82: chown f2 124800 1 2024-03-18T14:50:55.539 INFO:tasks.workunit.client.1.smithi100.stdout:1/83: chown d8/de/f14 265390828 1 2024-03-18T14:50:55.540 INFO:tasks.workunit.client.0.smithi012.stdout:8/546: dread d0/d15/d6e/d84/d54/f72 [0,4194304] 0 2024-03-18T14:50:55.542 INFO:tasks.workunit.client.1.smithi100.stdout:0/94: creat dd/f24 x:0 0 0 2024-03-18T14:50:55.544 INFO:tasks.workunit.client.1.smithi100.stdout:0/95: fsync dd/ff 0 2024-03-18T14:50:55.544 INFO:tasks.workunit.client.1.smithi100.stdout:4/64: fsync d9/d13/f19 0 2024-03-18T14:50:55.545 INFO:tasks.workunit.client.1.smithi100.stdout:0/96: dread f9 [0,4194304] 0 2024-03-18T14:50:55.550 INFO:tasks.workunit.client.0.smithi012.stdout:1/376: creat d8/d1e/d27/f74 x:0 0 0 2024-03-18T14:50:55.550 INFO:tasks.workunit.client.0.smithi012.stdout:1/377: creat d8/db/d10/d31/f75 x:0 0 0 2024-03-18T14:50:55.551 INFO:tasks.workunit.client.1.smithi100.stdout:4/65: dread f8 [0,4194304] 0 2024-03-18T14:50:55.551 INFO:tasks.workunit.client.1.smithi100.stdout:4/66: creat d9/f1a x:0 0 0 2024-03-18T14:50:55.551 INFO:tasks.workunit.client.1.smithi100.stdout:4/67: dread - d9/d13/f19 zero size 2024-03-18T14:50:55.552 INFO:tasks.workunit.client.1.smithi100.stdout:0/97: creat dd/d20/f25 x:0 0 0 2024-03-18T14:50:55.554 INFO:tasks.workunit.client.0.smithi012.stdout:6/393: unlink d17/d19/d27/l37 0 2024-03-18T14:50:55.554 INFO:tasks.workunit.client.0.smithi012.stdout:6/394: fsync d17/d26/d36/f4b 0 2024-03-18T14:50:55.559 INFO:tasks.workunit.client.1.smithi100.stdout:4/68: link d9/f11 d9/f1b 0 2024-03-18T14:50:55.561 INFO:tasks.workunit.client.0.smithi012.stdout:7/436: rename dd/d13/c4e to dd/d13/d26/c97 0 2024-03-18T14:50:55.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/69: link f0 d9/d17/f1c 0 2024-03-18T14:50:55.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/70: dread - d9/f1a zero size 2024-03-18T14:50:55.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/350: mkdir dc/d3c/d4c/d50/d54/d69 0 2024-03-18T14:50:55.568 INFO:tasks.workunit.client.0.smithi012.stdout:6/395: creat d17/f75 x:0 0 0 2024-03-18T14:50:55.568 INFO:tasks.workunit.client.1.smithi100.stdout:6/43: sync 2024-03-18T14:50:55.569 INFO:tasks.workunit.client.1.smithi100.stdout:8/74: sync 2024-03-18T14:50:55.570 INFO:tasks.workunit.client.1.smithi100.stdout:8/75: write d4/f9 [1488431,131051] 0 2024-03-18T14:50:55.570 INFO:tasks.workunit.client.1.smithi100.stdout:7/66: sync 2024-03-18T14:50:55.570 INFO:tasks.workunit.client.1.smithi100.stdout:7/67: creat d5/f1c x:0 0 0 2024-03-18T14:50:55.570 INFO:tasks.workunit.client.1.smithi100.stdout:7/68: creat d5/d14/d16/f1d x:0 0 0 2024-03-18T14:50:55.571 INFO:tasks.workunit.client.0.smithi012.stdout:9/459: dwrite d3/d9/d19/d66/f2b [0,4194304] 0 2024-03-18T14:50:55.571 INFO:tasks.workunit.client.0.smithi012.stdout:2/427: dwrite d0/d10/f16 [0,4194304] 0 2024-03-18T14:50:55.571 INFO:tasks.workunit.client.0.smithi012.stdout:8/547: dwrite d0/d5a/d19/fa3 [0,4194304] 0 2024-03-18T14:50:55.571 INFO:tasks.workunit.client.0.smithi012.stdout:8/548: readlink d0/l2 0 2024-03-18T14:50:55.571 INFO:tasks.workunit.client.0.smithi012.stdout:8/549: chown d0/d5a/d19/d57/daa 447763736 1 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.1.smithi100.stdout:1/84: dwrite d8/de/f14 [0,4194304] 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.1.smithi100.stdout:1/85: creat d8/de/f16 x:0 0 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.1.smithi100.stdout:4/71: read f8 [1999838,86474] 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.1.smithi100.stdout:3/67: dwrite d3/f9 [4194304,4194304] 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.1.smithi100.stdout:5/54: dwrite f5 [4194304,4194304] 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.0.smithi012.stdout:1/378: rename d8/db/l20 to d8/db/d10/d31/d43/l76 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.0.smithi012.stdout:1/379: fdatasync d8/db/f25 0 2024-03-18T14:50:55.572 INFO:tasks.workunit.client.0.smithi012.stdout:3/351: truncate dc/f14 1742147 0 2024-03-18T14:50:55.573 INFO:tasks.workunit.client.0.smithi012.stdout:3/352: fdatasync dc/f14 0 2024-03-18T14:50:55.575 INFO:tasks.workunit.client.1.smithi100.stdout:8/76: dread f3 [0,4194304] 0 2024-03-18T14:50:55.576 INFO:tasks.workunit.client.0.smithi012.stdout:6/396: mknod d17/d26/c76 0 2024-03-18T14:50:55.581 INFO:tasks.workunit.client.1.smithi100.stdout:9/55: dread f9 [0,4194304] 0 2024-03-18T14:50:55.582 INFO:tasks.workunit.client.0.smithi012.stdout:6/397: dread d17/d19/f3e [0,4194304] 0 2024-03-18T14:50:55.586 INFO:tasks.workunit.client.1.smithi100.stdout:7/69: creat d5/d14/f1e x:0 0 0 2024-03-18T14:50:55.586 INFO:tasks.workunit.client.0.smithi012.stdout:9/460: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:50:55.586 INFO:tasks.workunit.client.0.smithi012.stdout:9/461: creat d3/d9/d19/d66/d69/d7f/f92 x:0 0 0 2024-03-18T14:50:55.586 INFO:tasks.workunit.client.0.smithi012.stdout:9/462: write d3/d9/d19/d66/d21/d35/d48/f67 [921099,39250] 0 2024-03-18T14:50:55.588 INFO:tasks.workunit.client.1.smithi100.stdout:3/68: write d3/f5 [2441375,10691] 0 2024-03-18T14:50:55.588 INFO:tasks.workunit.client.1.smithi100.stdout:3/69: creat d3/d8/fe x:0 0 0 2024-03-18T14:50:55.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/463: dread d3/f2e [0,4194304] 0 2024-03-18T14:50:55.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/464: read d3/f3b [118042,67297] 0 2024-03-18T14:50:55.590 INFO:tasks.workunit.client.0.smithi012.stdout:9/465: chown d3/d20/f31 7153272 1 2024-03-18T14:50:55.591 INFO:tasks.workunit.client.0.smithi012.stdout:2/428: mknod d0/d10/d18/d33/d71/c82 0 2024-03-18T14:50:55.595 INFO:tasks.workunit.client.1.smithi100.stdout:4/72: unlink d9/cc 0 2024-03-18T14:50:55.598 INFO:tasks.workunit.client.1.smithi100.stdout:8/77: symlink d4/dc/dd/de/d19/l1b 0 2024-03-18T14:50:55.598 INFO:tasks.workunit.client.1.smithi100.stdout:8/78: chown d4/dc/dd/de/f17 81985955 1 2024-03-18T14:50:55.599 INFO:tasks.workunit.client.1.smithi100.stdout:0/98: dwrite dd/f18 [0,4194304] 0 2024-03-18T14:50:55.600 INFO:tasks.workunit.client.0.smithi012.stdout:8/550: dwrite d0/d5a/fa2 [0,4194304] 0 2024-03-18T14:50:55.601 INFO:tasks.workunit.client.1.smithi100.stdout:9/56: rename da/c11 to da/c12 0 2024-03-18T14:50:55.602 INFO:tasks.workunit.client.0.smithi012.stdout:9/466: read d3/d49/f57 [3618099,116790] 0 2024-03-18T14:50:55.602 INFO:tasks.workunit.client.0.smithi012.stdout:9/467: write d3/d20/f31 [837309,16094] 0 2024-03-18T14:50:55.602 INFO:tasks.workunit.client.0.smithi012.stdout:9/468: dread - d3/d9/d19/d77/d52/d65/f86 zero size 2024-03-18T14:50:55.602 INFO:tasks.workunit.client.0.smithi012.stdout:1/380: dwrite d8/d1e/d27/f38 [0,4194304] 0 2024-03-18T14:50:55.603 INFO:tasks.workunit.client.1.smithi100.stdout:3/70: dread d3/f9 [0,4194304] 0 2024-03-18T14:50:55.603 INFO:tasks.workunit.client.1.smithi100.stdout:3/71: write d3/d8/fc [1654240,124136] 0 2024-03-18T14:50:55.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/437: rename dd/f5f to dd/d13/d23/d33/f98 0 2024-03-18T14:50:55.603 INFO:tasks.workunit.client.0.smithi012.stdout:9/469: rename d3/d9/d19/d66/d21 to d3/d9/d19/d66/d21/d35/d93 22 2024-03-18T14:50:55.605 INFO:tasks.workunit.client.1.smithi100.stdout:7/70: rename d5/c11 to d5/c1f 0 2024-03-18T14:50:55.605 INFO:tasks.workunit.client.1.smithi100.stdout:7/71: dread - d5/d14/f1e zero size 2024-03-18T14:50:55.605 INFO:tasks.workunit.client.1.smithi100.stdout:7/72: chown l2 18175415 1 2024-03-18T14:50:55.607 INFO:tasks.workunit.client.0.smithi012.stdout:6/398: rmdir d17/d38/d51/d6b 0 2024-03-18T14:50:55.612 INFO:tasks.workunit.client.0.smithi012.stdout:1/381: write d8/d1e/d27/f38 [2684006,59800] 0 2024-03-18T14:50:55.612 INFO:tasks.workunit.client.0.smithi012.stdout:1/382: chown d8/db/d10 13 1 2024-03-18T14:50:55.612 INFO:tasks.workunit.client.0.smithi012.stdout:1/383: readlink d8/db/d10/d31/d45/l57 0 2024-03-18T14:50:55.612 INFO:tasks.workunit.client.0.smithi012.stdout:1/384: fsync d8/fc 0 2024-03-18T14:50:55.616 INFO:tasks.workunit.client.0.smithi012.stdout:2/429: mknod d0/d10/d18/d33/d80/c83 0 2024-03-18T14:50:55.616 INFO:tasks.workunit.client.0.smithi012.stdout:2/430: write d0/d10/d18/d4f/f5f [762032,17727] 0 2024-03-18T14:50:55.620 INFO:tasks.workunit.client.0.smithi012.stdout:2/431: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:55.620 INFO:tasks.workunit.client.0.smithi012.stdout:3/353: link dc/d30/f51 dc/f6a 0 2024-03-18T14:50:55.624 INFO:tasks.workunit.client.1.smithi100.stdout:4/73: creat d9/d17/f1d x:0 0 0 2024-03-18T14:50:55.626 INFO:tasks.workunit.client.0.smithi012.stdout:8/551: symlink d0/lae 0 2024-03-18T14:50:55.626 INFO:tasks.workunit.client.0.smithi012.stdout:8/552: fsync d0/d15/d6e/d84/f30 0 2024-03-18T14:50:55.627 INFO:tasks.workunit.client.1.smithi100.stdout:8/79: dwrite d4/ff [0,4194304] 0 2024-03-18T14:50:55.632 INFO:tasks.workunit.client.1.smithi100.stdout:0/99: unlink f7 0 2024-03-18T14:50:55.633 INFO:tasks.workunit.client.1.smithi100.stdout:0/100: fsync f3 0 2024-03-18T14:50:55.634 INFO:tasks.workunit.client.1.smithi100.stdout:9/57: dwrite f7 [0,4194304] 0 2024-03-18T14:50:55.634 INFO:tasks.workunit.client.1.smithi100.stdout:9/58: write f9 [5695454,5205] 0 2024-03-18T14:50:55.634 INFO:tasks.workunit.client.0.smithi012.stdout:9/470: dwrite d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:55.635 INFO:tasks.workunit.client.0.smithi012.stdout:6/399: dwrite d17/d38/f5a [0,4194304] 0 2024-03-18T14:50:55.635 INFO:tasks.workunit.client.0.smithi012.stdout:8/553: dread d0/d15/d6e/d84/f6b [0,4194304] 0 2024-03-18T14:50:55.643 INFO:tasks.workunit.client.0.smithi012.stdout:1/385: rmdir d8/db/d10/d56/d6c 0 2024-03-18T14:50:55.644 INFO:tasks.workunit.client.0.smithi012.stdout:2/432: stat d0/d10/l2f 0 2024-03-18T14:50:55.646 INFO:tasks.workunit.client.1.smithi100.stdout:3/72: creat d3/ff x:0 0 0 2024-03-18T14:50:55.646 INFO:tasks.workunit.client.1.smithi100.stdout:3/73: write d3/f6 [3410322,125378] 0 2024-03-18T14:50:55.646 INFO:tasks.workunit.client.0.smithi012.stdout:3/354: symlink dc/d3c/l6b 0 2024-03-18T14:50:55.648 INFO:tasks.workunit.client.1.smithi100.stdout:7/73: link d5/ce d5/d14/d16/c20 0 2024-03-18T14:50:55.649 INFO:tasks.workunit.client.0.smithi012.stdout:8/554: dwrite d0/d5a/d19/f93 [0,4194304] 0 2024-03-18T14:50:55.649 INFO:tasks.workunit.client.0.smithi012.stdout:8/555: truncate d0/d5a/f9a 55228 0 2024-03-18T14:50:55.649 INFO:tasks.workunit.client.0.smithi012.stdout:9/471: mknod d3/d9/d19/d77/c94 0 2024-03-18T14:50:55.652 INFO:tasks.workunit.client.0.smithi012.stdout:3/355: dread dc/f43 [0,4194304] 0 2024-03-18T14:50:55.652 INFO:tasks.workunit.client.0.smithi012.stdout:3/356: creat dc/d3c/f6c x:0 0 0 2024-03-18T14:50:55.653 INFO:tasks.workunit.client.1.smithi100.stdout:3/74: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:55.653 INFO:tasks.workunit.client.0.smithi012.stdout:6/400: creat d17/d19/d27/d58/f77 x:0 0 0 2024-03-18T14:50:55.655 INFO:tasks.workunit.client.1.smithi100.stdout:5/55: link fb fd 0 2024-03-18T14:50:55.655 INFO:tasks.workunit.client.1.smithi100.stdout:1/86: rmdir d8 39 2024-03-18T14:50:55.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/438: getdents dd/d4f 0 2024-03-18T14:50:55.657 INFO:tasks.workunit.client.0.smithi012.stdout:7/439: creat dd/d13/d23/f99 x:0 0 0 2024-03-18T14:50:55.658 INFO:tasks.workunit.client.0.smithi012.stdout:9/472: write d3/d9/f39 [1985722,49771] 0 2024-03-18T14:50:55.659 INFO:tasks.workunit.client.0.smithi012.stdout:4/414: sync 2024-03-18T14:50:55.659 INFO:tasks.workunit.client.0.smithi012.stdout:4/415: write d6/d27/f67 [296338,52239] 0 2024-03-18T14:50:55.659 INFO:tasks.workunit.client.0.smithi012.stdout:5/435: sync 2024-03-18T14:50:55.659 INFO:tasks.workunit.client.0.smithi012.stdout:5/436: creat d3/d29/d36/d3d/d73/f92 x:0 0 0 2024-03-18T14:50:55.659 INFO:tasks.workunit.client.0.smithi012.stdout:5/437: creat d3/d1e/f93 x:0 0 0 2024-03-18T14:50:55.663 INFO:tasks.workunit.client.0.smithi012.stdout:1/386: rename d8/db/d10/d16/d54/f5c to d8/db/d10/d31/d45/f77 0 2024-03-18T14:50:55.663 INFO:tasks.workunit.client.0.smithi012.stdout:1/387: chown d8/db/d10/d4e 2 1 2024-03-18T14:50:55.666 INFO:tasks.workunit.client.0.smithi012.stdout:2/433: unlink d0/f21 0 2024-03-18T14:50:55.666 INFO:tasks.workunit.client.0.smithi012.stdout:2/434: write d0/fa [7236093,76847] 0 2024-03-18T14:50:55.669 INFO:tasks.workunit.client.1.smithi100.stdout:8/80: unlink d4/dc/dd/de/c12 0 2024-03-18T14:50:55.671 INFO:tasks.workunit.client.1.smithi100.stdout:9/59: creat da/db/f13 x:0 0 0 2024-03-18T14:50:55.672 INFO:tasks.workunit.client.1.smithi100.stdout:9/60: read f6 [266005,31120] 0 2024-03-18T14:50:55.672 INFO:tasks.workunit.client.1.smithi100.stdout:9/61: chown da/dd/f10 10726 1 2024-03-18T14:50:55.675 INFO:tasks.workunit.client.0.smithi012.stdout:7/440: dwrite dd/d13/f72 [0,4194304] 0 2024-03-18T14:50:55.675 INFO:tasks.workunit.client.0.smithi012.stdout:6/401: dwrite d17/f2c [0,4194304] 0 2024-03-18T14:50:55.682 INFO:tasks.workunit.client.0.smithi012.stdout:3/357: creat dc/d30/f6d x:0 0 0 2024-03-18T14:50:55.688 INFO:tasks.workunit.client.1.smithi100.stdout:7/74: mkdir d5/d14/d16/d21 0 2024-03-18T14:50:55.688 INFO:tasks.workunit.client.1.smithi100.stdout:7/75: write d5/fa [4912913,12724] 0 2024-03-18T14:50:55.689 INFO:tasks.workunit.client.1.smithi100.stdout:9/62: dwrite f6 [0,4194304] 0 2024-03-18T14:50:55.691 INFO:tasks.workunit.client.0.smithi012.stdout:9/473: mkdir d3/d9/d19/d77/d52/d95 0 2024-03-18T14:50:55.691 INFO:tasks.workunit.client.0.smithi012.stdout:9/474: creat d3/d9/d19/d66/d21/d35/d71/f96 x:0 0 0 2024-03-18T14:50:55.691 INFO:tasks.workunit.client.1.smithi100.stdout:4/74: getdents d9/d17 0 2024-03-18T14:50:55.692 INFO:tasks.workunit.client.0.smithi012.stdout:4/416: truncate d6/d19/d4f/f5e 1966826 0 2024-03-18T14:50:55.692 INFO:tasks.workunit.client.0.smithi012.stdout:0/424: sync 2024-03-18T14:50:55.693 INFO:tasks.workunit.client.0.smithi012.stdout:3/358: dread dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:55.694 INFO:tasks.workunit.client.1.smithi100.stdout:3/75: creat d3/f10 x:0 0 0 2024-03-18T14:50:55.694 INFO:tasks.workunit.client.0.smithi012.stdout:9/475: dread d3/d9/d5c/f85 [0,4194304] 0 2024-03-18T14:50:55.694 INFO:tasks.workunit.client.0.smithi012.stdout:9/476: chown d3/d20 154 1 2024-03-18T14:50:55.697 INFO:tasks.workunit.client.1.smithi100.stdout:5/56: chown l3 295079 1 2024-03-18T14:50:55.701 INFO:tasks.workunit.client.1.smithi100.stdout:1/87: rename l6 to d8/dd/d10/l17 0 2024-03-18T14:50:55.701 INFO:tasks.workunit.client.1.smithi100.stdout:7/76: dread d5/d14/f18 [0,4194304] 0 2024-03-18T14:50:55.702 INFO:tasks.workunit.client.1.smithi100.stdout:7/77: fdatasync d5/f1c 0 2024-03-18T14:50:55.702 INFO:tasks.workunit.client.1.smithi100.stdout:7/78: getdents d5/d14/d16/d21 0 2024-03-18T14:50:55.702 INFO:tasks.workunit.client.1.smithi100.stdout:7/79: write d5/d14/d16/f1d [553971,105763] 0 2024-03-18T14:50:55.702 INFO:tasks.workunit.client.1.smithi100.stdout:7/80: fsync d5/d14/d16/f1a 0 2024-03-18T14:50:55.702 INFO:tasks.workunit.client.0.smithi012.stdout:8/556: rename d0/d5a/d19/d63/da8 to d0/d15/d6e/d84/d54/d62/daf 0 2024-03-18T14:50:55.703 INFO:tasks.workunit.client.1.smithi100.stdout:2/59: sync 2024-03-18T14:50:55.704 INFO:tasks.workunit.client.1.smithi100.stdout:8/81: mknod d4/c1c 0 2024-03-18T14:50:55.706 INFO:tasks.workunit.client.0.smithi012.stdout:1/388: symlink d8/db/d10/d31/l78 0 2024-03-18T14:50:55.709 INFO:tasks.workunit.client.0.smithi012.stdout:2/435: getdents d0/d10/d18/d31/d4c 0 2024-03-18T14:50:55.713 INFO:tasks.workunit.client.1.smithi100.stdout:9/63: symlink da/db/l14 0 2024-03-18T14:50:55.714 INFO:tasks.workunit.client.0.smithi012.stdout:7/441: rmdir dd 39 2024-03-18T14:50:55.714 INFO:tasks.workunit.client.0.smithi012.stdout:1/389: write d8/db/d10/d16/f66 [2552372,81117] 0 2024-03-18T14:50:55.714 INFO:tasks.workunit.client.0.smithi012.stdout:1/390: dread - d8/d1e/f63 zero size 2024-03-18T14:50:55.714 INFO:tasks.workunit.client.0.smithi012.stdout:1/391: stat d8/db/l26 0 2024-03-18T14:50:55.717 INFO:tasks.workunit.client.0.smithi012.stdout:7/442: dread dd/d4f/f69 [0,4194304] 0 2024-03-18T14:50:55.717 INFO:tasks.workunit.client.0.smithi012.stdout:7/443: truncate dd/d13/d23/f99 471429 0 2024-03-18T14:50:55.718 INFO:tasks.workunit.client.0.smithi012.stdout:6/402: mknod d17/d38/c78 0 2024-03-18T14:50:55.724 INFO:tasks.workunit.client.0.smithi012.stdout:4/417: link d6/d19/d39/f40 d6/f8c 0 2024-03-18T14:50:55.724 INFO:tasks.workunit.client.0.smithi012.stdout:4/418: write d6/d19/f1c [1245659,105867] 0 2024-03-18T14:50:55.729 INFO:tasks.workunit.client.0.smithi012.stdout:9/477: unlink d3/d9/d5c/c7b 0 2024-03-18T14:50:55.729 INFO:tasks.workunit.client.0.smithi012.stdout:6/403: read d17/f1b [2236984,113304] 0 2024-03-18T14:50:55.730 INFO:tasks.workunit.client.1.smithi100.stdout:6/44: sync 2024-03-18T14:50:55.730 INFO:tasks.workunit.client.1.smithi100.stdout:6/45: dread - d6/d7/f9 zero size 2024-03-18T14:50:55.730 INFO:tasks.workunit.client.1.smithi100.stdout:3/76: creat d3/dd/f11 x:0 0 0 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/392: dwrite d8/d1e/d27/f48 [0,4194304] 0 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/393: write d8/db/d10/d4e/f53 [957293,60639] 0 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/394: fsync d8/d1e/f63 0 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/395: chown d8/db/d10/d16/d1c 227 1 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/396: chown d8/db/d10/d16/d1c/f23 3 1 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:5/438: getdents d3/d29 0 2024-03-18T14:50:55.732 INFO:tasks.workunit.client.0.smithi012.stdout:5/439: read - d3/d1e/f8c zero size 2024-03-18T14:50:55.735 INFO:tasks.workunit.client.0.smithi012.stdout:3/359: rename dc/d30/l5b to dc/d3c/d4c/l6e 0 2024-03-18T14:50:55.737 INFO:tasks.workunit.client.1.smithi100.stdout:1/88: creat d8/de/f18 x:0 0 0 2024-03-18T14:50:55.738 INFO:tasks.workunit.client.0.smithi012.stdout:8/557: rmdir d0/d15/d6e/d84/d2a 39 2024-03-18T14:50:55.738 INFO:tasks.workunit.client.0.smithi012.stdout:8/558: fdatasync d0/d5a/d19/dad/f4b 0 2024-03-18T14:50:55.741 INFO:tasks.workunit.client.0.smithi012.stdout:2/436: mknod d0/d10/d18/d31/d4c/c84 0 2024-03-18T14:50:55.742 INFO:tasks.workunit.client.0.smithi012.stdout:4/419: dwrite d6/d19/d39/f50 [0,4194304] 0 2024-03-18T14:50:55.744 INFO:tasks.workunit.client.0.smithi012.stdout:7/444: creat dd/d13/d23/d76/d6b/f9a x:0 0 0 2024-03-18T14:50:55.744 INFO:tasks.workunit.client.0.smithi012.stdout:7/445: creat dd/d13/d26/f9b x:0 0 0 2024-03-18T14:50:55.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/46: read f4 [726704,64436] 0 2024-03-18T14:50:55.745 INFO:tasks.workunit.client.1.smithi100.stdout:9/64: creat da/dd/f15 x:0 0 0 2024-03-18T14:50:55.745 INFO:tasks.workunit.client.1.smithi100.stdout:9/65: readlink da/db/l14 0 2024-03-18T14:50:55.751 INFO:tasks.workunit.client.0.smithi012.stdout:9/478: link d3/d9/d19/d66/c3f d3/d49/c97 0 2024-03-18T14:50:55.752 INFO:tasks.workunit.client.1.smithi100.stdout:4/75: rmdir d9 39 2024-03-18T14:50:55.752 INFO:tasks.workunit.client.1.smithi100.stdout:4/76: write d9/f1a [656701,36731] 0 2024-03-18T14:50:55.752 INFO:tasks.workunit.client.1.smithi100.stdout:4/77: write d9/f1a [963642,79833] 0 2024-03-18T14:50:55.753 INFO:tasks.workunit.client.1.smithi100.stdout:9/66: read f2 [3579845,47841] 0 2024-03-18T14:50:55.753 INFO:tasks.workunit.client.1.smithi100.stdout:9/67: read f2 [2636524,127370] 0 2024-03-18T14:50:55.753 INFO:tasks.workunit.client.1.smithi100.stdout:9/68: stat da 0 2024-03-18T14:50:55.753 INFO:tasks.workunit.client.0.smithi012.stdout:6/404: creat d17/d26/f79 x:0 0 0 2024-03-18T14:50:55.757 INFO:tasks.workunit.client.1.smithi100.stdout:0/101: sync 2024-03-18T14:50:55.758 INFO:tasks.workunit.client.1.smithi100.stdout:5/57: symlink le 0 2024-03-18T14:50:55.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/89: mkdir d8/dd/d10/d11/d19 0 2024-03-18T14:50:55.759 INFO:tasks.workunit.client.0.smithi012.stdout:0/425: write d1/d16/d31/f3f [607878,55555] 0 2024-03-18T14:50:55.760 INFO:tasks.workunit.client.0.smithi012.stdout:1/397: link d8/db/l3d d8/db/d10/l79 0 2024-03-18T14:50:55.761 INFO:tasks.workunit.client.1.smithi100.stdout:7/81: getdents d5/d14/d16 0 2024-03-18T14:50:55.762 INFO:tasks.workunit.client.0.smithi012.stdout:5/440: readlink d3/d1e/d33/l69 0 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/78: creat d9/d13/f1e x:0 0 0 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/79: truncate d9/d13/f1e 309693 0 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/80: dread - d9/fb zero size 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/81: truncate d9/fe 1010967 0 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/82: dread - d9/f10 zero size 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/83: write d9/f10 [1020022,54462] 0 2024-03-18T14:50:55.764 INFO:tasks.workunit.client.1.smithi100.stdout:4/84: read d9/f1b [1430789,42033] 0 2024-03-18T14:50:55.765 INFO:tasks.workunit.client.1.smithi100.stdout:6/47: dwrite f4 [4194304,4194304] 0 2024-03-18T14:50:55.765 INFO:tasks.workunit.client.1.smithi100.stdout:6/48: dread - d6/d7/f8 zero size 2024-03-18T14:50:55.765 INFO:tasks.workunit.client.1.smithi100.stdout:9/69: creat da/f16 x:0 0 0 2024-03-18T14:50:55.767 INFO:tasks.workunit.client.0.smithi012.stdout:8/559: mkdir d0/d5a/d19/d63/db0 0 2024-03-18T14:50:55.767 INFO:tasks.workunit.client.0.smithi012.stdout:8/560: read d0/d5a/d19/f9d [2524221,20477] 0 2024-03-18T14:50:55.767 INFO:tasks.workunit.client.0.smithi012.stdout:8/561: fdatasync d0/d5a/d19/dad/f3c 0 2024-03-18T14:50:55.767 INFO:tasks.workunit.client.0.smithi012.stdout:8/562: fsync d0/d5a/d19/f75 0 2024-03-18T14:50:55.767 INFO:tasks.workunit.client.0.smithi012.stdout:2/437: dwrite d0/d10/d18/d33/f62 [0,4194304] 0 2024-03-18T14:50:55.768 INFO:tasks.workunit.client.1.smithi100.stdout:3/77: link d3/d8/cb d3/c12 0 2024-03-18T14:50:55.769 INFO:tasks.workunit.client.0.smithi012.stdout:4/420: creat d6/da/d58/f8d x:0 0 0 2024-03-18T14:50:55.769 INFO:tasks.workunit.client.0.smithi012.stdout:4/421: chown d6/da/f16 54043092 1 2024-03-18T14:50:55.770 INFO:tasks.workunit.client.1.smithi100.stdout:9/70: dread f9 [4194304,4194304] 0 2024-03-18T14:50:55.770 INFO:tasks.workunit.client.1.smithi100.stdout:0/102: creat dd/f26 x:0 0 0 2024-03-18T14:50:55.774 INFO:tasks.workunit.client.0.smithi012.stdout:7/446: mkdir dd/d13/d23/d60/d9c 0 2024-03-18T14:50:55.775 INFO:tasks.workunit.client.1.smithi100.stdout:4/85: rmdir d9 39 2024-03-18T14:50:55.777 INFO:tasks.workunit.client.0.smithi012.stdout:9/479: symlink d3/d9/d19/d66/d6e/l98 0 2024-03-18T14:50:55.777 INFO:tasks.workunit.client.0.smithi012.stdout:9/480: write d3/d9/d19/d66/d21/f46 [1542693,86338] 0 2024-03-18T14:50:55.779 INFO:tasks.workunit.client.1.smithi100.stdout:6/49: creat d6/fb x:0 0 0 2024-03-18T14:50:55.779 INFO:tasks.workunit.client.1.smithi100.stdout:6/50: rename d6 to d6/d7/dc 22 2024-03-18T14:50:55.780 INFO:tasks.workunit.client.0.smithi012.stdout:7/447: dread dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:55.781 INFO:tasks.workunit.client.1.smithi100.stdout:4/86: read d9/fe [515854,53613] 0 2024-03-18T14:50:55.781 INFO:tasks.workunit.client.1.smithi100.stdout:9/71: stat da/c12 0 2024-03-18T14:50:55.782 INFO:tasks.workunit.client.0.smithi012.stdout:0/426: creat d1/d20/d23/d3b/d33/d64/f96 x:0 0 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.1.smithi100.stdout:1/90: dwrite d8/de/f14 [0,4194304] 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.1.smithi100.stdout:7/82: dwrite d5/fa [0,4194304] 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.1.smithi100.stdout:0/103: write dd/f1d [791993,110724] 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.1.smithi100.stdout:0/104: truncate f6 1228760 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.1.smithi100.stdout:0/105: creat dd/d20/f27 x:0 0 0 2024-03-18T14:50:55.783 INFO:tasks.workunit.client.0.smithi012.stdout:1/398: creat d8/f7a x:0 0 0 2024-03-18T14:50:55.784 INFO:tasks.workunit.client.0.smithi012.stdout:9/481: dread d3/d9/f17 [0,4194304] 0 2024-03-18T14:50:55.784 INFO:tasks.workunit.client.0.smithi012.stdout:9/482: chown d3/d9/f1c 73758115 1 2024-03-18T14:50:55.784 INFO:tasks.workunit.client.0.smithi012.stdout:9/483: dread - d3/d9/d5c/f8f zero size 2024-03-18T14:50:55.785 INFO:tasks.workunit.client.1.smithi100.stdout:5/58: truncate fa 1183205 0 2024-03-18T14:50:55.786 INFO:tasks.workunit.client.0.smithi012.stdout:5/441: creat d3/d29/d36/d3d/d86/f94 x:0 0 0 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.1.smithi100.stdout:4/87: creat d9/f1f x:0 0 0 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.1.smithi100.stdout:4/88: write d9/f11 [1032326,43955] 0 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.1.smithi100.stdout:4/89: read - d9/fb zero size 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.1.smithi100.stdout:4/90: write d9/fe [1694081,115270] 0 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.0.smithi012.stdout:3/360: unlink dc/f1f 0 2024-03-18T14:50:55.789 INFO:tasks.workunit.client.0.smithi012.stdout:3/361: chown dc/d30 5773 1 2024-03-18T14:50:55.791 INFO:tasks.workunit.client.1.smithi100.stdout:9/72: symlink da/db/l17 0 2024-03-18T14:50:55.791 INFO:tasks.workunit.client.1.smithi100.stdout:1/91: dread f2 [0,4194304] 0 2024-03-18T14:50:55.791 INFO:tasks.workunit.client.1.smithi100.stdout:1/92: chown d8/de/f16 11790 1 2024-03-18T14:50:55.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/563: symlink d0/d15/d6e/d84/d54/d9f/lb1 0 2024-03-18T14:50:55.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/564: creat d0/d15/d6e/d84/d54/fb2 x:0 0 0 2024-03-18T14:50:55.797 INFO:tasks.workunit.client.1.smithi100.stdout:7/83: unlink d5/f1c 0 2024-03-18T14:50:55.799 INFO:tasks.workunit.client.0.smithi012.stdout:4/422: mknod d6/c8e 0 2024-03-18T14:50:55.799 INFO:tasks.workunit.client.0.smithi012.stdout:4/423: chown d6/d19/d6c/f88 24175136 1 2024-03-18T14:50:55.800 INFO:tasks.workunit.client.1.smithi100.stdout:0/106: write dd/f1a [153566,41627] 0 2024-03-18T14:50:55.802 INFO:tasks.workunit.client.0.smithi012.stdout:6/405: read f11 [2786402,54271] 0 2024-03-18T14:50:55.803 INFO:tasks.workunit.client.0.smithi012.stdout:1/399: symlink d8/db/d10/d31/d43/d5e/d65/l7b 0 2024-03-18T14:50:55.803 INFO:tasks.workunit.client.0.smithi012.stdout:1/400: stat d8/db/d10/d31/d45/f6d 0 2024-03-18T14:50:55.803 INFO:tasks.workunit.client.0.smithi012.stdout:1/401: creat d8/d1e/f7c x:0 0 0 2024-03-18T14:50:55.806 INFO:tasks.workunit.client.1.smithi100.stdout:4/91: rename d9/c14 to d9/d13/c20 0 2024-03-18T14:50:55.807 INFO:tasks.workunit.client.0.smithi012.stdout:5/442: creat d3/d29/f95 x:0 0 0 2024-03-18T14:50:55.809 INFO:tasks.workunit.client.1.smithi100.stdout:9/73: mkdir da/dd/d18 0 2024-03-18T14:50:55.809 INFO:tasks.workunit.client.0.smithi012.stdout:8/565: dread d0/d5a/d19/dad/f2d [0,4194304] 0 2024-03-18T14:50:55.809 INFO:tasks.workunit.client.0.smithi012.stdout:3/362: mknod dc/d3c/c6f 0 2024-03-18T14:50:55.809 INFO:tasks.workunit.client.0.smithi012.stdout:3/363: creat dc/d30/d35/f70 x:0 0 0 2024-03-18T14:50:55.810 INFO:tasks.workunit.client.1.smithi100.stdout:5/59: symlink lf 0 2024-03-18T14:50:55.810 INFO:tasks.workunit.client.1.smithi100.stdout:0/107: dread f2 [0,4194304] 0 2024-03-18T14:50:55.811 INFO:tasks.workunit.client.1.smithi100.stdout:1/93: dread f7 [4194304,4194304] 0 2024-03-18T14:50:55.812 INFO:tasks.workunit.client.1.smithi100.stdout:3/78: dwrite d3/dd/f11 [0,4194304] 0 2024-03-18T14:50:55.813 INFO:tasks.workunit.client.1.smithi100.stdout:5/60: fsync f7 0 2024-03-18T14:50:55.814 INFO:tasks.workunit.client.0.smithi012.stdout:2/438: dread d0/d10/d18/d31/d4c/f67 [0,4194304] 0 2024-03-18T14:50:55.817 INFO:tasks.workunit.client.0.smithi012.stdout:1/402: rename d8/db/d10/d31/d43/f51 to d8/db/f7d 0 2024-03-18T14:50:55.818 INFO:tasks.workunit.client.1.smithi100.stdout:6/51: dwrite d6/d7/f9 [0,4194304] 0 2024-03-18T14:50:55.819 INFO:tasks.workunit.client.0.smithi012.stdout:8/566: dread d0/d5a/d19/dad/d33/d50/fa5 [0,4194304] 0 2024-03-18T14:50:55.820 INFO:tasks.workunit.client.0.smithi012.stdout:7/448: dwrite dd/d13/d23/d60/f6c [0,4194304] 0 2024-03-18T14:50:55.820 INFO:tasks.workunit.client.0.smithi012.stdout:0/427: dwrite d1/f9 [0,4194304] 0 2024-03-18T14:50:55.821 INFO:tasks.workunit.client.1.smithi100.stdout:0/108: unlink f9 0 2024-03-18T14:50:55.821 INFO:tasks.workunit.client.1.smithi100.stdout:0/109: creat dd/d20/f28 x:0 0 0 2024-03-18T14:50:55.821 INFO:tasks.workunit.client.0.smithi012.stdout:5/443: mknod d3/d84/c96 0 2024-03-18T14:50:55.821 INFO:tasks.workunit.client.0.smithi012.stdout:5/444: fdatasync d3/d29/d36/d3d/f3e 0 2024-03-18T14:50:55.825 INFO:tasks.workunit.client.1.smithi100.stdout:1/94: rename d8/fa to d8/f1a 0 2024-03-18T14:50:55.827 INFO:tasks.workunit.client.0.smithi012.stdout:2/439: dread d0/d3c/d40/d47/f49 [0,4194304] 0 2024-03-18T14:50:55.831 INFO:tasks.workunit.client.0.smithi012.stdout:3/364: symlink dc/d3c/d4c/d50/d54/l71 0 2024-03-18T14:50:55.836 INFO:tasks.workunit.client.1.smithi100.stdout:3/79: mkdir d3/d8/d13 0 2024-03-18T14:50:55.836 INFO:tasks.workunit.client.0.smithi012.stdout:0/428: dread d1/d20/d23/d26/f36 [0,4194304] 0 2024-03-18T14:50:55.837 INFO:tasks.workunit.client.0.smithi012.stdout:0/429: chown d1/d20/d23/d3b/d33/l73 661 1 2024-03-18T14:50:55.838 INFO:tasks.workunit.client.1.smithi100.stdout:5/61: write f2 [205068,18690] 0 2024-03-18T14:50:55.838 INFO:tasks.workunit.client.1.smithi100.stdout:5/62: chown fd 5 1 2024-03-18T14:50:55.838 INFO:tasks.workunit.client.1.smithi100.stdout:5/63: truncate f2 544666 0 2024-03-18T14:50:55.838 INFO:tasks.workunit.client.1.smithi100.stdout:5/64: fdatasync f5 0 2024-03-18T14:50:55.846 INFO:tasks.workunit.client.0.smithi012.stdout:1/403: mkdir d8/db/d10/d16/d6b/d7e 0 2024-03-18T14:50:55.847 INFO:tasks.workunit.client.0.smithi012.stdout:4/424: dwrite d6/d1a/d30/d46/d6f/d7c/f87 [0,4194304] 0 2024-03-18T14:50:55.847 INFO:tasks.workunit.client.0.smithi012.stdout:4/425: truncate d6/d19/f42 1161356 0 2024-03-18T14:50:55.848 INFO:tasks.workunit.client.0.smithi012.stdout:8/567: link d0/d5a/d5e/f82 d0/d5a/d19/dad/d33/d50/fb3 0 2024-03-18T14:50:55.853 INFO:tasks.workunit.client.0.smithi012.stdout:5/445: link d3/d15/d5f/c88 d3/d29/d35/c97 0 2024-03-18T14:50:55.853 INFO:tasks.workunit.client.0.smithi012.stdout:5/446: truncate d3/d29/d35/f5a 1618246 0 2024-03-18T14:50:55.855 INFO:tasks.workunit.client.0.smithi012.stdout:2/440: mkdir d0/d3c/d40/d47/d85 0 2024-03-18T14:50:55.857 INFO:tasks.workunit.client.0.smithi012.stdout:3/365: mkdir dc/d3c/d72 0 2024-03-18T14:50:55.857 INFO:tasks.workunit.client.0.smithi012.stdout:3/366: dread - dc/d3c/d4c/d50/f52 zero size 2024-03-18T14:50:55.858 INFO:tasks.workunit.client.1.smithi100.stdout:6/52: mknod d6/cd 0 2024-03-18T14:50:55.858 INFO:tasks.workunit.client.1.smithi100.stdout:6/53: creat d6/fe x:0 0 0 2024-03-18T14:50:55.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/430: creat d1/d20/d23/d7f/f97 x:0 0 0 2024-03-18T14:50:55.861 INFO:tasks.workunit.client.1.smithi100.stdout:9/74: dwrite da/db/f13 [0,4194304] 0 2024-03-18T14:50:55.861 INFO:tasks.workunit.client.1.smithi100.stdout:9/75: write da/dd/f10 [6139502,52690] 0 2024-03-18T14:50:55.861 INFO:tasks.workunit.client.1.smithi100.stdout:9/76: chown f9 3906821 1 2024-03-18T14:50:55.861 INFO:tasks.workunit.client.1.smithi100.stdout:7/84: dwrite d5/fd [0,4194304] 0 2024-03-18T14:50:55.861 INFO:tasks.workunit.client.0.smithi012.stdout:7/449: dwrite dd/d13/d23/d76/d6b/f9a [0,4194304] 0 2024-03-18T14:50:55.862 INFO:tasks.workunit.client.0.smithi012.stdout:6/406: dwrite d17/f75 [0,4194304] 0 2024-03-18T14:50:55.862 INFO:tasks.workunit.client.0.smithi012.stdout:6/407: creat d17/d26/d36/f7a x:0 0 0 2024-03-18T14:50:55.862 INFO:tasks.workunit.client.1.smithi100.stdout:1/95: dwrite f2 [0,4194304] 0 2024-03-18T14:50:55.865 INFO:tasks.workunit.client.1.smithi100.stdout:5/65: creat f10 x:0 0 0 2024-03-18T14:50:55.866 INFO:tasks.workunit.client.0.smithi012.stdout:4/426: truncate d6/d19/d39/f40 659816 0 2024-03-18T14:50:55.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/80: dwrite d3/f10 [0,4194304] 0 2024-03-18T14:50:55.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/81: creat d3/d8/f14 x:0 0 0 2024-03-18T14:50:55.869 INFO:tasks.workunit.client.0.smithi012.stdout:8/568: truncate d0/d5a/d19/f9d 713870 0 2024-03-18T14:50:55.874 INFO:tasks.workunit.client.0.smithi012.stdout:0/431: read d1/d20/d7b/f13 [7727856,77973] 0 2024-03-18T14:50:55.874 INFO:tasks.workunit.client.0.smithi012.stdout:5/447: link d3/d29/d35/f48 d3/d29/d36/d3d/d73/d6b/f98 0 2024-03-18T14:50:55.875 INFO:tasks.workunit.client.0.smithi012.stdout:5/448: fsync d3/d1e/d33/f74 0 2024-03-18T14:50:55.875 INFO:tasks.workunit.client.0.smithi012.stdout:5/449: chown d3/d1e/d33/d89/d59 1872064 1 2024-03-18T14:50:55.875 INFO:tasks.workunit.client.0.smithi012.stdout:5/450: read - d3/d29/f61 zero size 2024-03-18T14:50:55.876 INFO:tasks.workunit.client.1.smithi100.stdout:6/54: dwrite d6/fb [0,4194304] 0 2024-03-18T14:50:55.878 INFO:tasks.workunit.client.0.smithi012.stdout:2/441: rename d0/f55 to d0/d10/d18/d33/d77/f86 0 2024-03-18T14:50:55.879 INFO:tasks.workunit.client.0.smithi012.stdout:6/408: write d17/f2c [3391159,78477] 0 2024-03-18T14:50:55.882 INFO:tasks.workunit.client.0.smithi012.stdout:3/367: unlink dc/d30/c46 0 2024-03-18T14:50:55.885 INFO:tasks.workunit.client.1.smithi100.stdout:9/77: unlink da/cf 0 2024-03-18T14:50:55.887 INFO:tasks.workunit.client.0.smithi012.stdout:7/450: symlink dd/d13/d26/l9d 0 2024-03-18T14:50:55.890 INFO:tasks.workunit.client.1.smithi100.stdout:1/96: dwrite f5 [0,4194304] 0 2024-03-18T14:50:55.891 INFO:tasks.workunit.client.0.smithi012.stdout:8/569: dwrite d0/d5a/d19/dad/f71 [0,4194304] 0 2024-03-18T14:50:55.891 INFO:tasks.workunit.client.1.smithi100.stdout:3/82: dwrite d3/d8/f14 [0,4194304] 0 2024-03-18T14:50:55.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/451: unlink f2 0 2024-03-18T14:50:55.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/452: creat d3/d29/d36/d3d/d73/f99 x:0 0 0 2024-03-18T14:50:55.903 INFO:tasks.workunit.client.0.smithi012.stdout:2/442: dread d0/d10/d18/d31/d4c/f51 [0,4194304] 0 2024-03-18T14:50:55.907 INFO:tasks.workunit.client.0.smithi012.stdout:6/409: link d17/d19/c25 d17/d26/c7b 0 2024-03-18T14:50:55.907 INFO:tasks.workunit.client.0.smithi012.stdout:6/410: stat d17/d33/l3a 0 2024-03-18T14:50:55.907 INFO:tasks.workunit.client.0.smithi012.stdout:6/411: creat d17/d19/d27/d58/f7c x:0 0 0 2024-03-18T14:50:55.908 INFO:tasks.workunit.client.1.smithi100.stdout:6/55: mkdir d6/d7/df 0 2024-03-18T14:50:55.908 INFO:tasks.workunit.client.1.smithi100.stdout:6/56: readlink l5 0 2024-03-18T14:50:55.910 INFO:tasks.workunit.client.0.smithi012.stdout:3/368: mknod dc/d3c/d4c/c73 0 2024-03-18T14:50:55.914 INFO:tasks.workunit.client.0.smithi012.stdout:1/404: getdents d8/db/d10/d16/d6b 0 2024-03-18T14:50:55.914 INFO:tasks.workunit.client.0.smithi012.stdout:6/412: dread d17/f1b [0,4194304] 0 2024-03-18T14:50:55.915 INFO:tasks.workunit.client.0.smithi012.stdout:2/443: dread d0/f8 [0,4194304] 0 2024-03-18T14:50:55.916 INFO:tasks.workunit.client.0.smithi012.stdout:7/451: link dd/d4f/c71 dd/d13/d23/d60/c9e 0 2024-03-18T14:50:55.916 INFO:tasks.workunit.client.0.smithi012.stdout:7/452: chown dd/d13/d26/c97 192654533 1 2024-03-18T14:50:55.916 INFO:tasks.workunit.client.0.smithi012.stdout:2/444: write d0/d10/d18/d4f/f7c [4887247,101136] 0 2024-03-18T14:50:55.916 INFO:tasks.workunit.client.0.smithi012.stdout:2/445: write d0/f7b [2225131,117476] 0 2024-03-18T14:50:55.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/78: mknod da/db/c19 0 2024-03-18T14:50:55.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/79: creat da/db/f1a x:0 0 0 2024-03-18T14:50:55.918 INFO:tasks.workunit.client.0.smithi012.stdout:8/570: dwrite d0/d5a/d19/dad/f7c [0,4194304] 0 2024-03-18T14:50:55.918 INFO:tasks.workunit.client.0.smithi012.stdout:8/571: stat d0/d5a/d19/d57 0 2024-03-18T14:50:55.919 INFO:tasks.workunit.client.0.smithi012.stdout:5/453: dwrite d3/d1e/d33/d89/f71 [0,4194304] 0 2024-03-18T14:50:55.924 INFO:tasks.workunit.client.1.smithi100.stdout:8/82: sync 2024-03-18T14:50:55.924 INFO:tasks.workunit.client.1.smithi100.stdout:2/60: sync 2024-03-18T14:50:55.926 INFO:tasks.workunit.client.1.smithi100.stdout:1/97: creat d8/dd/d10/d11/f1b x:0 0 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:8/83: write d4/f9 [627010,83024] 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/61: write d3/f17 [107197,94378] 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:8/84: fsync d4/f6 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/62: truncate d3/fc 266387 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/63: chown d3/d6 621 1 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/64: truncate d3/f1b 538300 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/65: readlink l1 0 2024-03-18T14:50:55.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/66: fsync d3/da/de/f19 0 2024-03-18T14:50:55.928 INFO:tasks.workunit.client.0.smithi012.stdout:0/432: truncate d1/d20/d23/d26/f53 855758 0 2024-03-18T14:50:55.928 INFO:tasks.workunit.client.1.smithi100.stdout:9/80: read f9 [270269,44402] 0 2024-03-18T14:50:55.928 INFO:tasks.workunit.client.1.smithi100.stdout:9/81: chown da/db 189826266 1 2024-03-18T14:50:55.929 INFO:tasks.workunit.client.1.smithi100.stdout:8/85: dread f3 [0,4194304] 0 2024-03-18T14:50:55.930 INFO:tasks.workunit.client.1.smithi100.stdout:8/86: write f1 [3557210,23640] 0 2024-03-18T14:50:55.930 INFO:tasks.workunit.client.1.smithi100.stdout:7/85: getdents d5 0 2024-03-18T14:50:55.933 INFO:tasks.workunit.client.1.smithi100.stdout:3/83: mknod d3/dd/c15 0 2024-03-18T14:50:55.934 INFO:tasks.workunit.client.1.smithi100.stdout:7/86: dread d5/d14/d16/f1d [0,4194304] 0 2024-03-18T14:50:55.935 INFO:tasks.workunit.client.1.smithi100.stdout:0/110: rename dd/c1e to dd/c29 0 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.1.smithi100.stdout:7/87: dread d5/d14/d16/f1d [0,4194304] 0 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.0.smithi012.stdout:3/369: dwrite f6 [0,4194304] 0 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.0.smithi012.stdout:3/370: readlink dc/d3c/l6b 0 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.0.smithi012.stdout:3/371: chown dc/d3c/d4c/d50/d54/d69 38992 1 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.0.smithi012.stdout:3/372: creat dc/d3c/f74 x:0 0 0 2024-03-18T14:50:55.937 INFO:tasks.workunit.client.0.smithi012.stdout:3/373: creat dc/d30/d35/f75 x:0 0 0 2024-03-18T14:50:55.938 INFO:tasks.workunit.client.0.smithi012.stdout:6/413: dread d17/d18/f23 [0,4194304] 0 2024-03-18T14:50:55.938 INFO:tasks.workunit.client.1.smithi100.stdout:6/57: mkdir d6/d10 0 2024-03-18T14:50:55.940 INFO:tasks.workunit.client.0.smithi012.stdout:3/374: dread dc/f24 [0,4194304] 0 2024-03-18T14:50:55.940 INFO:tasks.workunit.client.0.smithi012.stdout:3/375: truncate dc/d3c/d59/f5a 697587 0 2024-03-18T14:50:55.945 INFO:tasks.workunit.client.1.smithi100.stdout:1/98: unlink d8/dd/d10/l17 0 2024-03-18T14:50:55.947 INFO:tasks.workunit.client.0.smithi012.stdout:4/427: write d6/d19/d39/f40 [1089140,49166] 0 2024-03-18T14:50:55.948 INFO:tasks.workunit.client.0.smithi012.stdout:1/405: dwrite d8/d1e/d27/f74 [0,4194304] 0 2024-03-18T14:50:55.948 INFO:tasks.workunit.client.0.smithi012.stdout:1/406: dread - d8/db/d10/d31/f75 zero size 2024-03-18T14:50:55.948 INFO:tasks.workunit.client.0.smithi012.stdout:1/407: chown d8/db/d10/f34 527424 1 2024-03-18T14:50:55.948 INFO:tasks.workunit.client.0.smithi012.stdout:1/408: dread - d8/db/d10/d31/d45/f6d zero size 2024-03-18T14:50:55.949 INFO:tasks.workunit.client.1.smithi100.stdout:2/67: creat d3/da/de/f1c x:0 0 0 2024-03-18T14:50:55.953 INFO:tasks.workunit.client.1.smithi100.stdout:9/82: symlink da/l1b 0 2024-03-18T14:50:55.955 INFO:tasks.workunit.client.0.smithi012.stdout:5/454: dwrite d3/d15/d5f/f70 [0,4194304] 0 2024-03-18T14:50:55.955 INFO:tasks.workunit.client.0.smithi012.stdout:5/455: creat d3/d29/f9a x:0 0 0 2024-03-18T14:50:55.956 INFO:tasks.workunit.client.0.smithi012.stdout:8/572: mknod d0/d47/d59/d86/d87/cb4 0 2024-03-18T14:50:55.956 INFO:tasks.workunit.client.0.smithi012.stdout:0/433: dwrite d1/d20/d7b/f30 [0,4194304] 0 2024-03-18T14:50:55.957 INFO:tasks.workunit.client.1.smithi100.stdout:8/87: symlink d4/dc/dd/de/d19/l1d 0 2024-03-18T14:50:55.961 INFO:tasks.workunit.client.1.smithi100.stdout:7/88: unlink d5/c1f 0 2024-03-18T14:50:55.962 INFO:tasks.workunit.client.1.smithi100.stdout:7/89: getdents d5/d14/d16 0 2024-03-18T14:50:55.962 INFO:tasks.workunit.client.0.smithi012.stdout:0/434: dread d1/f3 [0,4194304] 0 2024-03-18T14:50:55.965 INFO:tasks.workunit.client.1.smithi100.stdout:1/99: rmdir d8/dd/d10/d11/d19 0 2024-03-18T14:50:55.967 INFO:tasks.workunit.client.1.smithi100.stdout:3/84: dwrite d3/d8/fc [0,4194304] 0 2024-03-18T14:50:55.968 INFO:tasks.workunit.client.1.smithi100.stdout:2/68: rename d3/d6/f12 to d3/d13/f1d 0 2024-03-18T14:50:55.968 INFO:tasks.workunit.client.1.smithi100.stdout:2/69: write d3/f10 [581367,109049] 0 2024-03-18T14:50:55.969 INFO:tasks.workunit.client.1.smithi100.stdout:2/70: write d3/f10 [1287918,77188] 0 2024-03-18T14:50:55.969 INFO:tasks.workunit.client.0.smithi012.stdout:6/414: mkdir d17/d38/d51/d66/d7d 0 2024-03-18T14:50:55.969 INFO:tasks.workunit.client.0.smithi012.stdout:6/415: readlink d17/d26/d36/l49 0 2024-03-18T14:50:55.969 INFO:tasks.workunit.client.0.smithi012.stdout:6/416: chown d17/d26/d36/l40 1 1 2024-03-18T14:50:55.970 INFO:tasks.workunit.client.0.smithi012.stdout:6/417: creat d17/d26/d36/d4d/f7e x:0 0 0 2024-03-18T14:50:55.970 INFO:tasks.workunit.client.0.smithi012.stdout:6/418: write d17/d19/d27/d2f/f43 [498447,86111] 0 2024-03-18T14:50:55.970 INFO:tasks.workunit.client.1.smithi100.stdout:9/83: symlink da/dd/d18/l1c 0 2024-03-18T14:50:55.971 INFO:tasks.workunit.client.0.smithi012.stdout:3/376: stat dc/d3c/d4c/d5f/c60 0 2024-03-18T14:50:55.971 INFO:tasks.workunit.client.1.smithi100.stdout:6/58: dwrite d6/d7/f8 [0,4194304] 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:4/428: dwrite d6/da/fd [0,4194304] 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:4/429: truncate d6/d1a/d30/d46/d6f/f71 1390734 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:4/430: write d6/da/f13 [1506658,44201] 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/446: getdents d0/d3c/d40/d47 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/447: readlink d0/d10/d18/d57/l5d 0 2024-03-18T14:50:55.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/448: readlink d0/d10/d18/l2b 0 2024-03-18T14:50:55.974 INFO:tasks.workunit.client.0.smithi012.stdout:2/449: truncate d0/f8 4404859 0 2024-03-18T14:50:55.985 INFO:tasks.workunit.client.0.smithi012.stdout:1/409: unlink d8/db/d10/d31/d43/l72 0 2024-03-18T14:50:55.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/88: creat d4/dc/dd/de/d19/f1e x:0 0 0 2024-03-18T14:50:55.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/89: read d4/f6 [1008664,44778] 0 2024-03-18T14:50:55.992 INFO:tasks.workunit.client.0.smithi012.stdout:5/456: rename d3/d1e/d33/d89 to d3/d29/d36/d9b 0 2024-03-18T14:50:55.992 INFO:tasks.workunit.client.0.smithi012.stdout:5/457: fsync d3/d29/d36/d3d/f6e 0 2024-03-18T14:50:55.994 INFO:tasks.workunit.client.1.smithi100.stdout:1/100: mknod d8/dd/c1c 0 2024-03-18T14:50:55.994 INFO:tasks.workunit.client.1.smithi100.stdout:1/101: fsync d8/dd/ff 0 2024-03-18T14:50:55.995 INFO:tasks.workunit.client.0.smithi012.stdout:8/573: mknod d0/d5a/d19/dad/cb5 0 2024-03-18T14:50:55.998 INFO:tasks.workunit.client.0.smithi012.stdout:5/458: dread d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:56.001 INFO:tasks.workunit.client.1.smithi100.stdout:9/84: rmdir da/dd/d18 39 2024-03-18T14:50:56.003 INFO:tasks.workunit.client.1.smithi100.stdout:2/71: dwrite d3/da/de/ff [0,4194304] 0 2024-03-18T14:50:56.004 INFO:tasks.workunit.client.0.smithi012.stdout:0/435: dwrite d1/d20/d23/d3b/d33/f6d [0,4194304] 0 2024-03-18T14:50:56.004 INFO:tasks.workunit.client.1.smithi100.stdout:3/85: dwrite d3/f5 [0,4194304] 0 2024-03-18T14:50:56.004 INFO:tasks.workunit.client.0.smithi012.stdout:4/431: dwrite d6/d19/d39/f6d [0,4194304] 0 2024-03-18T14:50:56.006 INFO:tasks.workunit.client.0.smithi012.stdout:6/419: link d17/d38/d3f/l61 d17/d19/d27/d58/l7f 0 2024-03-18T14:50:56.012 INFO:tasks.workunit.client.1.smithi100.stdout:8/90: mkdir d4/dc/d1f 0 2024-03-18T14:50:56.014 INFO:tasks.workunit.client.1.smithi100.stdout:1/102: mknod d8/dd/c1d 0 2024-03-18T14:50:56.016 INFO:tasks.workunit.client.0.smithi012.stdout:2/450: creat d0/d10/d18/d33/d71/f87 x:0 0 0 2024-03-18T14:50:56.016 INFO:tasks.workunit.client.0.smithi012.stdout:2/451: write d0/f79 [853778,8775] 0 2024-03-18T14:50:56.017 INFO:tasks.workunit.client.0.smithi012.stdout:2/452: write d0/d10/d18/d33/d77/f86 [3700078,11782] 0 2024-03-18T14:50:56.018 INFO:tasks.workunit.client.1.smithi100.stdout:9/85: link da/fc da/f1d 0 2024-03-18T14:50:56.019 INFO:tasks.workunit.client.1.smithi100.stdout:1/103: dread f7 [4194304,4194304] 0 2024-03-18T14:50:56.021 INFO:tasks.workunit.client.0.smithi012.stdout:1/410: unlink d8/db/d10/d31/f75 0 2024-03-18T14:50:56.026 INFO:tasks.workunit.client.1.smithi100.stdout:2/72: link d3/d6/cd d3/d13/c1e 0 2024-03-18T14:50:56.034 INFO:tasks.workunit.client.0.smithi012.stdout:8/574: dwrite d0/d15/d6e/d84/f6b [0,4194304] 0 2024-03-18T14:50:56.034 INFO:tasks.workunit.client.0.smithi012.stdout:5/459: dwrite d3/d29/d36/d3d/d73/f76 [0,4194304] 0 2024-03-18T14:50:56.037 INFO:tasks.workunit.client.0.smithi012.stdout:2/453: dwrite d0/d10/d18/f5b [0,4194304] 0 2024-03-18T14:50:56.037 INFO:tasks.workunit.client.0.smithi012.stdout:2/454: readlink d0/d10/l4d 0 2024-03-18T14:50:56.038 INFO:tasks.workunit.client.1.smithi100.stdout:6/59: truncate d6/fb 4180739 0 2024-03-18T14:50:56.038 INFO:tasks.workunit.client.1.smithi100.stdout:6/60: chown c2 1 1 2024-03-18T14:50:56.038 INFO:tasks.workunit.client.1.smithi100.stdout:6/61: fdatasync d6/fe 0 2024-03-18T14:50:56.038 INFO:tasks.workunit.client.1.smithi100.stdout:6/62: stat c2 0 2024-03-18T14:50:56.040 INFO:tasks.workunit.client.1.smithi100.stdout:8/91: unlink d4/dc/dd/c14 0 2024-03-18T14:50:56.051 INFO:tasks.workunit.client.0.smithi012.stdout:0/436: link d1/c7 d1/d20/d23/d88/c98 0 2024-03-18T14:50:56.054 INFO:tasks.workunit.client.1.smithi100.stdout:3/86: rename d3/d8 to d3/d16 0 2024-03-18T14:50:56.054 INFO:tasks.workunit.client.1.smithi100.stdout:3/87: chown d3/d16/fc 26 1 2024-03-18T14:50:56.056 INFO:tasks.workunit.client.0.smithi012.stdout:6/420: creat d17/d38/d5b/f80 x:0 0 0 2024-03-18T14:50:56.057 INFO:tasks.workunit.client.0.smithi012.stdout:3/377: symlink dc/l76 0 2024-03-18T14:50:56.057 INFO:tasks.workunit.client.0.smithi012.stdout:3/378: chown dc/d3c 8801 1 2024-03-18T14:50:56.057 INFO:tasks.workunit.client.0.smithi012.stdout:3/379: readlink dc/d3c/d4c/l6e 0 2024-03-18T14:50:56.057 INFO:tasks.workunit.client.0.smithi012.stdout:3/380: fsync dc/d30/f51 0 2024-03-18T14:50:56.058 INFO:tasks.workunit.client.0.smithi012.stdout:1/411: creat d8/db/d10/f7f x:0 0 0 2024-03-18T14:50:56.059 INFO:tasks.workunit.client.0.smithi012.stdout:8/575: symlink d0/d15/d6e/d84/d2a/d3e/lb6 0 2024-03-18T14:50:56.059 INFO:tasks.workunit.client.0.smithi012.stdout:8/576: write d0/d5a/d19/dad/d33/f5c [2838587,11419] 0 2024-03-18T14:50:56.076 INFO:tasks.workunit.client.1.smithi100.stdout:6/63: dread f4 [4194304,4194304] 0 2024-03-18T14:50:56.081 INFO:tasks.workunit.client.0.smithi012.stdout:5/460: write d3/d29/d36/d3d/f7f [2932709,116450] 0 2024-03-18T14:50:56.081 INFO:tasks.workunit.client.0.smithi012.stdout:4/432: dwrite d6/f8c [0,4194304] 0 2024-03-18T14:50:56.081 INFO:tasks.workunit.client.0.smithi012.stdout:4/433: write d6/d19/f3e [595895,45920] 0 2024-03-18T14:50:56.081 INFO:tasks.workunit.client.0.smithi012.stdout:5/461: creat d3/d84/f9c x:0 0 0 2024-03-18T14:50:56.081 INFO:tasks.workunit.client.0.smithi012.stdout:4/434: write d6/d1a/d30/f47 [557211,10628] 0 2024-03-18T14:50:56.082 INFO:tasks.workunit.client.1.smithi100.stdout:3/88: write d3/fa [1403125,64569] 0 2024-03-18T14:50:56.087 INFO:tasks.workunit.client.1.smithi100.stdout:6/64: write d6/d7/f9 [3946937,3518] 0 2024-03-18T14:50:56.089 INFO:tasks.workunit.client.1.smithi100.stdout:2/73: dwrite d3/f17 [0,4194304] 0 2024-03-18T14:50:56.092 INFO:tasks.workunit.client.0.smithi012.stdout:5/462: read d3/d29/d36/f45 [1219117,35513] 0 2024-03-18T14:50:56.092 INFO:tasks.workunit.client.0.smithi012.stdout:5/463: dread - d3/d29/f61 zero size 2024-03-18T14:50:56.096 INFO:tasks.workunit.client.0.smithi012.stdout:6/421: dwrite d17/d33/f48 [0,4194304] 0 2024-03-18T14:50:56.097 INFO:tasks.workunit.client.1.smithi100.stdout:8/92: dwrite f3 [0,4194304] 0 2024-03-18T14:50:56.105 INFO:tasks.workunit.client.0.smithi012.stdout:0/437: mkdir d1/d20/d23/d3b/d33/d51/d99 0 2024-03-18T14:50:56.107 INFO:tasks.workunit.client.0.smithi012.stdout:3/381: getdents dc/d3c/d4c/d5f 0 2024-03-18T14:50:56.108 INFO:tasks.workunit.client.1.smithi100.stdout:2/74: dwrite f0 [0,4194304] 0 2024-03-18T14:50:56.108 INFO:tasks.workunit.client.0.smithi012.stdout:6/422: dread d17/d19/f2b [0,4194304] 0 2024-03-18T14:50:56.108 INFO:tasks.workunit.client.0.smithi012.stdout:6/423: write d17/d26/d36/f4b [1012740,67013] 0 2024-03-18T14:50:56.108 INFO:tasks.workunit.client.0.smithi012.stdout:6/424: creat d17/d26/d36/d4d/f81 x:0 0 0 2024-03-18T14:50:56.108 INFO:tasks.workunit.client.0.smithi012.stdout:6/425: truncate d17/d38/d51/f53 617791 0 2024-03-18T14:50:56.109 INFO:tasks.workunit.client.1.smithi100.stdout:9/86: getdents da 0 2024-03-18T14:50:56.112 INFO:tasks.workunit.client.0.smithi012.stdout:1/412: truncate d8/db/d10/f73 3810976 0 2024-03-18T14:50:56.114 INFO:tasks.workunit.client.1.smithi100.stdout:3/89: unlink d3/d16/fe 0 2024-03-18T14:50:56.117 INFO:tasks.workunit.client.0.smithi012.stdout:6/426: write d17/f1a [2642692,84234] 0 2024-03-18T14:50:56.119 INFO:tasks.workunit.client.0.smithi012.stdout:8/577: symlink d0/d5a/lb7 0 2024-03-18T14:50:56.121 INFO:tasks.workunit.client.1.smithi100.stdout:6/65: symlink d6/d7/l11 0 2024-03-18T14:50:56.123 INFO:tasks.workunit.client.0.smithi012.stdout:8/578: read d0/d15/d6e/f7b [13286,37237] 0 2024-03-18T14:50:56.123 INFO:tasks.workunit.client.0.smithi012.stdout:8/579: readlink d0/d5a/d19/l26 0 2024-03-18T14:50:56.123 INFO:tasks.workunit.client.0.smithi012.stdout:8/580: chown d0/d5a/d19/d57/d9e 29368869 1 2024-03-18T14:50:56.124 INFO:tasks.workunit.client.1.smithi100.stdout:3/90: write d3/f10 [3617715,16782] 0 2024-03-18T14:50:56.124 INFO:tasks.workunit.client.1.smithi100.stdout:3/91: stat d3/c7 0 2024-03-18T14:50:56.138 INFO:tasks.workunit.client.1.smithi100.stdout:8/93: creat d4/dc/d1f/f20 x:0 0 0 2024-03-18T14:50:56.142 INFO:tasks.workunit.client.1.smithi100.stdout:9/87: dwrite f6 [0,4194304] 0 2024-03-18T14:50:56.142 INFO:tasks.workunit.client.1.smithi100.stdout:9/88: fsync da/db/f1a 0 2024-03-18T14:50:56.142 INFO:tasks.workunit.client.1.smithi100.stdout:9/89: dread - da/f16 zero size 2024-03-18T14:50:56.143 INFO:tasks.workunit.client.0.smithi012.stdout:1/413: dwrite d8/d1e/d27/f74 [0,4194304] 0 2024-03-18T14:50:56.145 INFO:tasks.workunit.client.0.smithi012.stdout:8/581: dwrite d0/d5a/d19/d57/f68 [4194304,4194304] 0 2024-03-18T14:50:56.147 INFO:tasks.workunit.client.1.smithi100.stdout:6/66: mkdir d6/d12 0 2024-03-18T14:50:56.148 INFO:tasks.workunit.client.0.smithi012.stdout:1/414: write d8/db/d10/d16/d1c/f4c [3204696,27516] 0 2024-03-18T14:50:56.148 INFO:tasks.workunit.client.0.smithi012.stdout:1/415: chown d8/db/d10/c50 53947 1 2024-03-18T14:50:56.148 INFO:tasks.workunit.client.0.smithi012.stdout:1/416: stat d8/db/d10/d31/f68 0 2024-03-18T14:50:56.148 INFO:tasks.workunit.client.0.smithi012.stdout:1/417: chown d8/db/d10/f7f 56 1 2024-03-18T14:50:56.152 INFO:tasks.workunit.client.1.smithi100.stdout:9/90: dread da/f1d [0,4194304] 0 2024-03-18T14:50:56.154 INFO:tasks.workunit.client.1.smithi100.stdout:3/92: mkdir d3/d16/d13/d17 0 2024-03-18T14:50:56.158 INFO:tasks.workunit.client.1.smithi100.stdout:8/94: mkdir d4/dc/dd/de/d21 0 2024-03-18T14:50:56.162 INFO:tasks.workunit.client.1.smithi100.stdout:5/66: sync 2024-03-18T14:50:56.162 INFO:tasks.workunit.client.1.smithi100.stdout:4/92: sync 2024-03-18T14:50:56.163 INFO:tasks.workunit.client.1.smithi100.stdout:0/111: sync 2024-03-18T14:50:56.166 INFO:tasks.workunit.client.1.smithi100.stdout:0/112: write dd/f1d [254002,129293] 0 2024-03-18T14:50:56.166 INFO:tasks.workunit.client.1.smithi100.stdout:0/113: truncate f6 1782529 0 2024-03-18T14:50:56.168 INFO:tasks.workunit.client.1.smithi100.stdout:9/91: dwrite da/f16 [0,4194304] 0 2024-03-18T14:50:56.169 INFO:tasks.workunit.client.1.smithi100.stdout:6/67: link d6/fb d6/d10/f13 0 2024-03-18T14:50:56.171 INFO:tasks.workunit.client.1.smithi100.stdout:3/93: symlink d3/d16/l18 0 2024-03-18T14:50:56.171 INFO:tasks.workunit.client.1.smithi100.stdout:3/94: chown d3/d16/f14 39 1 2024-03-18T14:50:56.177 INFO:tasks.workunit.client.1.smithi100.stdout:4/93: mknod d9/c21 0 2024-03-18T14:50:56.181 INFO:tasks.workunit.client.1.smithi100.stdout:4/94: dread d9/fe [0,4194304] 0 2024-03-18T14:50:56.193 INFO:tasks.workunit.client.1.smithi100.stdout:9/92: rename da/dd/d18/l1c to da/l1e 0 2024-03-18T14:50:56.197 INFO:tasks.workunit.client.1.smithi100.stdout:6/68: creat d6/d10/f14 x:0 0 0 2024-03-18T14:50:56.200 INFO:tasks.workunit.client.1.smithi100.stdout:3/95: symlink d3/dd/l19 0 2024-03-18T14:50:56.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/435: mknod d6/da/d58/c8f 0 2024-03-18T14:50:56.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/436: read d6/f21 [202335,11504] 0 2024-03-18T14:50:56.203 INFO:tasks.workunit.client.1.smithi100.stdout:5/67: rename f5 to f11 0 2024-03-18T14:50:56.204 INFO:tasks.workunit.client.1.smithi100.stdout:5/68: dread - f10 zero size 2024-03-18T14:50:56.204 INFO:tasks.workunit.client.1.smithi100.stdout:6/69: read d6/fb [684684,50981] 0 2024-03-18T14:50:56.205 INFO:tasks.workunit.client.1.smithi100.stdout:0/114: rmdir dd/d20 39 2024-03-18T14:50:56.207 INFO:tasks.workunit.client.1.smithi100.stdout:4/95: link d9/f1f d9/f22 0 2024-03-18T14:50:56.207 INFO:tasks.workunit.client.1.smithi100.stdout:4/96: write d9/f1f [387706,51114] 0 2024-03-18T14:50:56.208 INFO:tasks.workunit.client.1.smithi100.stdout:5/69: creat f12 x:0 0 0 2024-03-18T14:50:56.208 INFO:tasks.workunit.client.1.smithi100.stdout:3/96: dread d3/d16/f14 [0,4194304] 0 2024-03-18T14:50:56.209 INFO:tasks.workunit.client.1.smithi100.stdout:9/93: dwrite da/db/f1a [0,4194304] 0 2024-03-18T14:50:56.212 INFO:tasks.workunit.client.1.smithi100.stdout:5/70: dread f2 [0,4194304] 0 2024-03-18T14:50:56.214 INFO:tasks.workunit.client.1.smithi100.stdout:4/97: creat d9/d17/f23 x:0 0 0 2024-03-18T14:50:56.214 INFO:tasks.workunit.client.1.smithi100.stdout:4/98: readlink l5 0 2024-03-18T14:50:56.215 INFO:tasks.workunit.client.1.smithi100.stdout:4/99: truncate d9/d17/f23 722227 0 2024-03-18T14:50:56.215 INFO:tasks.workunit.client.1.smithi100.stdout:4/100: creat d9/d17/f24 x:0 0 0 2024-03-18T14:50:56.216 INFO:tasks.workunit.client.1.smithi100.stdout:3/97: truncate d3/d16/f14 2878460 0 2024-03-18T14:50:56.235 INFO:tasks.workunit.client.1.smithi100.stdout:0/115: rename dd/c15 to dd/c2a 0 2024-03-18T14:50:56.235 INFO:tasks.workunit.client.1.smithi100.stdout:4/101: dwrite f8 [0,4194304] 0 2024-03-18T14:50:56.236 INFO:tasks.workunit.client.1.smithi100.stdout:3/98: dwrite d3/f10 [0,4194304] 0 2024-03-18T14:50:56.237 INFO:tasks.workunit.client.0.smithi012.stdout:2/455: rename d0/d10/l28 to d0/d10/d18/d33/d80/l88 0 2024-03-18T14:50:56.238 INFO:tasks.workunit.client.1.smithi100.stdout:5/71: link le l13 0 2024-03-18T14:50:56.239 INFO:tasks.workunit.client.0.smithi012.stdout:0/438: fsync d1/d16/d19/f56 0 2024-03-18T14:50:56.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:55 smithi012 ceph-mon[29182]: pgmap v245: 65 pgs: 65 active+clean; 939 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 80 MiB/s rd, 105 MiB/s wr, 542 op/s 2024-03-18T14:50:56.244 INFO:tasks.workunit.client.1.smithi100.stdout:4/102: mkdir d9/d13/d25 0 2024-03-18T14:50:56.250 INFO:tasks.workunit.client.0.smithi012.stdout:6/427: mkdir d17/d26/d36/d4d/d82 0 2024-03-18T14:50:56.250 INFO:tasks.workunit.client.0.smithi012.stdout:6/428: dread - d17/d19/d27/f54 zero size 2024-03-18T14:50:56.251 INFO:tasks.workunit.client.1.smithi100.stdout:5/72: truncate f11 481131 0 2024-03-18T14:50:56.251 INFO:tasks.workunit.client.1.smithi100.stdout:5/73: stat fd 0 2024-03-18T14:50:56.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/116: mkdir dd/d2b 0 2024-03-18T14:50:56.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/117: fdatasync dd/f24 0 2024-03-18T14:50:56.257 INFO:tasks.workunit.client.1.smithi100.stdout:7/90: sync 2024-03-18T14:50:56.259 INFO:tasks.workunit.client.0.smithi012.stdout:8/582: rename d0/d15/d6e/d84/d54/d62/daf/d80/l89 to d0/d15/d6e/d84/d54/d62/daf/d80/lb8 0 2024-03-18T14:50:56.259 INFO:tasks.workunit.client.0.smithi012.stdout:8/583: fsync d0/d5a/d19/dad/f4d 0 2024-03-18T14:50:56.259 INFO:tasks.workunit.client.0.smithi012.stdout:8/584: chown d0/d15/d6e/d84/d2a/c7e 255952475 1 2024-03-18T14:50:56.264 INFO:tasks.workunit.client.0.smithi012.stdout:1/418: unlink d8/db/d10/d16/d1c/c21 0 2024-03-18T14:50:56.266 INFO:tasks.workunit.client.0.smithi012.stdout:4/437: mknod d6/da/d89/c90 0 2024-03-18T14:50:56.270 INFO:tasks.workunit.client.1.smithi100.stdout:0/118: creat dd/d20/f2c x:0 0 0 2024-03-18T14:50:56.271 INFO:tasks.workunit.client.1.smithi100.stdout:0/119: creat dd/d20/f2d x:0 0 0 2024-03-18T14:50:56.272 INFO:tasks.workunit.client.1.smithi100.stdout:7/91: mkdir d5/d22 0 2024-03-18T14:50:56.276 INFO:tasks.workunit.client.1.smithi100.stdout:5/74: rename f0 to f14 0 2024-03-18T14:50:56.277 INFO:tasks.workunit.client.0.smithi012.stdout:6/429: symlink d17/d26/d36/l83 0 2024-03-18T14:50:56.277 INFO:tasks.workunit.client.0.smithi012.stdout:6/430: write d17/d33/f48 [4954148,1655] 0 2024-03-18T14:50:56.281 INFO:tasks.workunit.client.1.smithi100.stdout:0/120: write dd/f18 [2311812,20464] 0 2024-03-18T14:50:56.282 INFO:tasks.workunit.client.1.smithi100.stdout:7/92: link d5/d14/d16/c19 d5/d22/c23 0 2024-03-18T14:50:56.282 INFO:tasks.workunit.client.1.smithi100.stdout:7/93: stat l2 0 2024-03-18T14:50:56.288 INFO:tasks.workunit.client.0.smithi012.stdout:1/419: dwrite d8/db/d10/d16/d1c/f2c [8388608,4194304] 0 2024-03-18T14:50:56.288 INFO:tasks.workunit.client.0.smithi012.stdout:8/585: dread d0/f7 [0,4194304] 0 2024-03-18T14:50:56.290 INFO:tasks.workunit.client.1.smithi100.stdout:7/94: mkdir d5/d22/d24 0 2024-03-18T14:50:56.291 INFO:tasks.workunit.client.0.smithi012.stdout:8/586: rmdir d0/d15/d6e/d84/d54/d62/daf 39 2024-03-18T14:50:56.294 INFO:tasks.workunit.client.1.smithi100.stdout:5/75: rename fd to f15 0 2024-03-18T14:50:56.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:55 smithi100 ceph-mon[37807]: pgmap v245: 65 pgs: 65 active+clean; 939 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 80 MiB/s rd, 105 MiB/s wr, 542 op/s 2024-03-18T14:50:56.296 INFO:tasks.workunit.client.0.smithi012.stdout:8/587: rename d0/d5a/d19/dad/f7c to d0/d5a/d19/d57/daa/fb9 0 2024-03-18T14:50:56.296 INFO:tasks.workunit.client.0.smithi012.stdout:8/588: creat d0/d47/fba x:0 0 0 2024-03-18T14:50:56.297 INFO:tasks.workunit.client.1.smithi100.stdout:7/95: mknod d5/c25 0 2024-03-18T14:50:56.302 INFO:tasks.workunit.client.1.smithi100.stdout:7/96: symlink d5/d14/d16/l26 0 2024-03-18T14:50:56.302 INFO:tasks.workunit.client.0.smithi012.stdout:6/431: dwrite d17/d26/d36/d4d/f5c [0,4194304] 0 2024-03-18T14:50:56.303 INFO:tasks.workunit.client.1.smithi100.stdout:0/121: dwrite dd/f24 [0,4194304] 0 2024-03-18T14:50:56.303 INFO:tasks.workunit.client.1.smithi100.stdout:0/122: chown dd/d20/f27 255867 1 2024-03-18T14:50:56.304 INFO:tasks.workunit.client.0.smithi012.stdout:8/589: symlink d0/d5a/d19/lbb 0 2024-03-18T14:50:56.306 INFO:tasks.workunit.client.1.smithi100.stdout:5/76: symlink l16 0 2024-03-18T14:50:56.313 INFO:tasks.workunit.client.0.smithi012.stdout:8/590: dread d0/d15/d6e/d84/d2a/fa1 [0,4194304] 0 2024-03-18T14:50:56.317 INFO:tasks.workunit.client.0.smithi012.stdout:6/432: rmdir d17/d38/d5b 39 2024-03-18T14:50:56.319 INFO:tasks.workunit.client.1.smithi100.stdout:7/97: dwrite d5/fd [0,4194304] 0 2024-03-18T14:50:56.321 INFO:tasks.workunit.client.1.smithi100.stdout:5/77: creat f17 x:0 0 0 2024-03-18T14:50:56.321 INFO:tasks.workunit.client.1.smithi100.stdout:5/78: chown f2 806 1 2024-03-18T14:50:56.321 INFO:tasks.workunit.client.0.smithi012.stdout:6/433: mkdir d17/d19/d27/d6e/d84 0 2024-03-18T14:50:56.325 INFO:tasks.workunit.client.1.smithi100.stdout:5/79: dread fa [0,4194304] 0 2024-03-18T14:50:56.326 INFO:tasks.workunit.client.0.smithi012.stdout:6/434: write fb [575560,7972] 0 2024-03-18T14:50:56.327 INFO:tasks.workunit.client.0.smithi012.stdout:6/435: dread - d17/d19/d27/f68 zero size 2024-03-18T14:50:56.327 INFO:tasks.workunit.client.0.smithi012.stdout:6/436: creat d17/d38/d51/f85 x:0 0 0 2024-03-18T14:50:56.327 INFO:tasks.workunit.client.1.smithi100.stdout:7/98: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:56.327 INFO:tasks.workunit.client.1.smithi100.stdout:0/123: link dd/f1b dd/f2e 0 2024-03-18T14:50:56.332 INFO:tasks.workunit.client.0.smithi012.stdout:8/591: dwrite d0/d5a/d19/dad/f4d [0,4194304] 0 2024-03-18T14:50:56.338 INFO:tasks.workunit.client.0.smithi012.stdout:8/592: symlink d0/d15/d6e/d84/d54/lbc 0 2024-03-18T14:50:56.338 INFO:tasks.workunit.client.0.smithi012.stdout:8/593: chown d0/d15/d6e/d84/c2b 153157 1 2024-03-18T14:50:56.340 INFO:tasks.workunit.client.0.smithi012.stdout:3/382: symlink dc/d30/l77 0 2024-03-18T14:50:56.344 INFO:tasks.workunit.client.1.smithi100.stdout:9/94: rmdir da/dd/d18 0 2024-03-18T14:50:56.348 INFO:tasks.workunit.client.1.smithi100.stdout:0/124: dwrite f2 [4194304,4194304] 0 2024-03-18T14:50:56.348 INFO:tasks.workunit.client.0.smithi012.stdout:3/383: write dc/f44 [2542587,23291] 0 2024-03-18T14:50:56.351 INFO:tasks.workunit.client.0.smithi012.stdout:6/437: dwrite f14 [4194304,4194304] 0 2024-03-18T14:50:56.351 INFO:tasks.workunit.client.0.smithi012.stdout:6/438: write d17/d19/d27/f54 [650509,71396] 0 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/439: fdatasync d17/d19/f3e 0 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/440: chown f4 4029378 1 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/441: dread - d17/d38/d51/f85 zero size 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/442: chown d17/d26/c31 114 1 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/443: creat d17/d19/d27/d58/f86 x:0 0 0 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/444: truncate d17/d33/f64 903053 0 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.0.smithi012.stdout:6/445: truncate d17/d26/d36/d4d/f7e 121874 0 2024-03-18T14:50:56.352 INFO:tasks.workunit.client.1.smithi100.stdout:0/125: link dd/c19 dd/d20/c2f 0 2024-03-18T14:50:56.354 INFO:tasks.workunit.client.0.smithi012.stdout:6/446: creat d17/d38/d5b/f87 x:0 0 0 2024-03-18T14:50:56.355 INFO:tasks.workunit.client.0.smithi012.stdout:6/447: creat d17/d33/f88 x:0 0 0 2024-03-18T14:50:56.355 INFO:tasks.workunit.client.0.smithi012.stdout:6/448: dread - d17/d19/d27/d58/f77 zero size 2024-03-18T14:50:56.355 INFO:tasks.workunit.client.0.smithi012.stdout:6/449: readlink d17/d26/l69 0 2024-03-18T14:50:56.355 INFO:tasks.workunit.client.1.smithi100.stdout:0/126: mknod dd/d2b/c30 0 2024-03-18T14:50:56.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/384: write f6 [3988701,68647] 0 2024-03-18T14:50:56.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/385: readlink dc/d30/d35/l34 0 2024-03-18T14:50:56.359 INFO:tasks.workunit.client.0.smithi012.stdout:3/386: fsync dc/f11 0 2024-03-18T14:50:56.359 INFO:tasks.workunit.client.0.smithi012.stdout:3/387: chown dc/d3c/d4c/d50/d54/l71 63 1 2024-03-18T14:50:56.359 INFO:tasks.workunit.client.0.smithi012.stdout:3/388: write dc/d3c/f6c [647389,64945] 0 2024-03-18T14:50:56.361 INFO:tasks.workunit.client.0.smithi012.stdout:8/594: dwrite d0/d15/d6e/d84/d2a/fa1 [0,4194304] 0 2024-03-18T14:50:56.362 INFO:tasks.workunit.client.0.smithi012.stdout:3/389: rename dc/d3c/l6b to dc/d3c/d4c/d50/l78 0 2024-03-18T14:50:56.363 INFO:tasks.workunit.client.1.smithi100.stdout:1/104: sync 2024-03-18T14:50:56.368 INFO:tasks.workunit.client.0.smithi012.stdout:3/390: creat dc/d3c/d4c/d50/d64/f79 x:0 0 0 2024-03-18T14:50:56.369 INFO:tasks.workunit.client.0.smithi012.stdout:8/595: truncate d0/d15/d6e/d84/f6b 3738452 0 2024-03-18T14:50:56.370 INFO:tasks.workunit.client.0.smithi012.stdout:8/596: chown d0/d5a/d19/d57/f69 11 1 2024-03-18T14:50:56.371 INFO:tasks.workunit.client.1.smithi100.stdout:1/105: rename d8/dd/d10 to d8/d1e 0 2024-03-18T14:50:56.371 INFO:tasks.workunit.client.1.smithi100.stdout:1/106: chown d8/d1e/l13 15857048 1 2024-03-18T14:50:56.374 INFO:tasks.workunit.client.1.smithi100.stdout:1/107: rename f2 to d8/dd/f1f 0 2024-03-18T14:50:56.378 INFO:tasks.workunit.client.0.smithi012.stdout:3/391: dread dc/d30/d3a/f5e [0,4194304] 0 2024-03-18T14:50:56.381 INFO:tasks.workunit.client.0.smithi012.stdout:8/597: dread d0/d15/d6e/d84/d54/f85 [0,4194304] 0 2024-03-18T14:50:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:8/598: mknod d0/d5a/cbd 0 2024-03-18T14:50:56.382 INFO:tasks.workunit.client.0.smithi012.stdout:8/599: mkdir d0/d5a/d19/d63/dbe 0 2024-03-18T14:50:56.383 INFO:tasks.workunit.client.0.smithi012.stdout:8/600: symlink d0/d5a/d19/dad/lbf 0 2024-03-18T14:50:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:8/601: mknod d0/d47/d59/cc0 0 2024-03-18T14:50:56.384 INFO:tasks.workunit.client.0.smithi012.stdout:8/602: getdents d0/d5a/d19/dad/d33/d50/d6a 0 2024-03-18T14:50:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:8/603: fdatasync d0/d47/f4f 0 2024-03-18T14:50:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:8/604: rmdir d0/d15/d6e/d84/d54/d62/daf/d80 39 2024-03-18T14:50:56.385 INFO:tasks.workunit.client.0.smithi012.stdout:8/605: write d0/f6 [1448431,130483] 0 2024-03-18T14:50:56.394 INFO:tasks.workunit.client.1.smithi100.stdout:5/80: dread f11 [0,4194304] 0 2024-03-18T14:50:56.394 INFO:tasks.workunit.client.1.smithi100.stdout:5/81: write f11 [1461621,106079] 0 2024-03-18T14:50:56.396 INFO:tasks.workunit.client.1.smithi100.stdout:0/127: fsync dd/d20/f2d 0 2024-03-18T14:50:56.400 INFO:tasks.workunit.client.1.smithi100.stdout:0/128: mkdir dd/d31 0 2024-03-18T14:50:56.405 INFO:tasks.workunit.client.1.smithi100.stdout:0/129: rename dd/f23 to dd/d20/f32 0 2024-03-18T14:50:56.406 INFO:tasks.workunit.client.1.smithi100.stdout:0/130: read - dd/d20/f2c zero size 2024-03-18T14:50:56.408 INFO:tasks.workunit.client.1.smithi100.stdout:0/131: unlink dd/f24 0 2024-03-18T14:50:56.412 INFO:tasks.workunit.client.1.smithi100.stdout:0/132: unlink f5 0 2024-03-18T14:50:56.412 INFO:tasks.workunit.client.1.smithi100.stdout:5/82: dwrite f11 [0,4194304] 0 2024-03-18T14:50:56.413 INFO:tasks.workunit.client.0.smithi012.stdout:5/464: mkdir d3/d29/d9d 0 2024-03-18T14:50:56.417 INFO:tasks.workunit.client.0.smithi012.stdout:5/465: rename d3/d15/f83 to d3/f9e 0 2024-03-18T14:50:56.420 INFO:tasks.workunit.client.1.smithi100.stdout:0/133: dread dd/ff [0,4194304] 0 2024-03-18T14:50:56.420 INFO:tasks.workunit.client.1.smithi100.stdout:0/134: stat dd/d20/f2c 0 2024-03-18T14:50:56.420 INFO:tasks.workunit.client.0.smithi012.stdout:2/456: symlink d0/d10/d18/l89 0 2024-03-18T14:50:56.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/83: fsync f17 0 2024-03-18T14:50:56.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/84: truncate f12 255580 0 2024-03-18T14:50:56.424 INFO:tasks.workunit.client.0.smithi012.stdout:2/457: creat d0/d10/d18/d31/d4c/d59/f8a x:0 0 0 2024-03-18T14:50:56.426 INFO:tasks.workunit.client.0.smithi012.stdout:2/458: symlink d0/d3c/d40/d47/d85/l8b 0 2024-03-18T14:50:56.428 INFO:tasks.workunit.client.1.smithi100.stdout:0/135: mknod dd/c33 0 2024-03-18T14:50:56.428 INFO:tasks.workunit.client.1.smithi100.stdout:0/136: dread - dd/f26 zero size 2024-03-18T14:50:56.428 INFO:tasks.workunit.client.1.smithi100.stdout:0/137: dread - dd/d20/f21 zero size 2024-03-18T14:50:56.429 INFO:tasks.workunit.client.1.smithi100.stdout:5/85: rename f11 to f18 0 2024-03-18T14:50:56.433 INFO:tasks.workunit.client.0.smithi012.stdout:4/438: rename d6/da/f16 to d6/da/f91 0 2024-03-18T14:50:56.433 INFO:tasks.workunit.client.0.smithi012.stdout:4/439: chown d6/f8c 118954 1 2024-03-18T14:50:56.433 INFO:tasks.workunit.client.0.smithi012.stdout:4/440: write d6/d1a/d64/f78 [471769,34733] 0 2024-03-18T14:50:56.436 INFO:tasks.workunit.client.0.smithi012.stdout:6/450: getdents d17/d19/d27/d58 0 2024-03-18T14:50:56.437 INFO:tasks.workunit.client.1.smithi100.stdout:5/86: write fb [617144,121126] 0 2024-03-18T14:50:56.437 INFO:tasks.workunit.client.1.smithi100.stdout:5/87: chown fb 1960 1 2024-03-18T14:50:56.437 INFO:tasks.workunit.client.0.smithi012.stdout:2/459: dread d0/d10/f32 [0,4194304] 0 2024-03-18T14:50:56.438 INFO:tasks.workunit.client.1.smithi100.stdout:8/95: sync 2024-03-18T14:50:56.438 INFO:tasks.workunit.client.1.smithi100.stdout:2/75: sync 2024-03-18T14:50:56.438 INFO:tasks.workunit.client.1.smithi100.stdout:6/70: sync 2024-03-18T14:50:56.438 INFO:tasks.workunit.client.1.smithi100.stdout:6/71: creat d6/f15 x:0 0 0 2024-03-18T14:50:56.439 INFO:tasks.workunit.client.1.smithi100.stdout:6/72: fsync f4 0 2024-03-18T14:50:56.439 INFO:tasks.workunit.client.1.smithi100.stdout:8/96: write d4/dc/dd/de/f17 [397660,108207] 0 2024-03-18T14:50:56.439 INFO:tasks.workunit.client.0.smithi012.stdout:2/460: getdents d0/d10/d18/d31/d4c 0 2024-03-18T14:50:56.441 INFO:tasks.workunit.client.1.smithi100.stdout:2/76: dread d3/f10 [0,4194304] 0 2024-03-18T14:50:56.441 INFO:tasks.workunit.client.1.smithi100.stdout:2/77: fsync d3/da/de/ff 0 2024-03-18T14:50:56.441 INFO:tasks.workunit.client.0.smithi012.stdout:2/461: dread d0/d10/d18/d33/f37 [0,4194304] 0 2024-03-18T14:50:56.442 INFO:tasks.workunit.client.0.smithi012.stdout:2/462: creat d0/d10/d18/d31/d4c/d59/f8c x:0 0 0 2024-03-18T14:50:56.443 INFO:tasks.workunit.client.1.smithi100.stdout:6/73: mknod d6/c16 0 2024-03-18T14:50:56.443 INFO:tasks.workunit.client.1.smithi100.stdout:2/78: dread d3/f10 [0,4194304] 0 2024-03-18T14:50:56.443 INFO:tasks.workunit.client.1.smithi100.stdout:2/79: chown d3/d13/l14 324 1 2024-03-18T14:50:56.444 INFO:tasks.workunit.client.0.smithi012.stdout:6/451: rmdir d17/d38/d51 39 2024-03-18T14:50:56.444 INFO:tasks.workunit.client.0.smithi012.stdout:6/452: dread - d17/d26/d36/f7a zero size 2024-03-18T14:50:56.445 INFO:tasks.workunit.client.1.smithi100.stdout:8/97: symlink d4/dc/dd/de/d19/l22 0 2024-03-18T14:50:56.445 INFO:tasks.workunit.client.0.smithi012.stdout:3/392: rmdir dc/d3c/d4c/d50 39 2024-03-18T14:50:56.448 INFO:tasks.workunit.client.1.smithi100.stdout:8/98: dread d4/f9 [0,4194304] 0 2024-03-18T14:50:56.449 INFO:tasks.workunit.client.1.smithi100.stdout:5/88: mknod c19 0 2024-03-18T14:50:56.451 INFO:tasks.workunit.client.1.smithi100.stdout:1/108: unlink d8/dd/f1f 0 2024-03-18T14:50:56.452 INFO:tasks.workunit.client.1.smithi100.stdout:1/109: readlink d8/l15 0 2024-03-18T14:50:56.452 INFO:tasks.workunit.client.1.smithi100.stdout:1/110: truncate d8/d1e/d11/f1b 683452 0 2024-03-18T14:50:56.452 INFO:tasks.workunit.client.1.smithi100.stdout:0/138: dwrite fa [0,4194304] 0 2024-03-18T14:50:56.453 INFO:tasks.workunit.client.0.smithi012.stdout:2/463: symlink d0/d10/d18/d57/l8d 0 2024-03-18T14:50:56.453 INFO:tasks.workunit.client.0.smithi012.stdout:2/464: dread - d0/d10/d18/d33/d71/f87 zero size 2024-03-18T14:50:56.455 INFO:tasks.workunit.client.1.smithi100.stdout:6/74: mknod d6/c17 0 2024-03-18T14:50:56.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/453: symlink d17/d33/l89 0 2024-03-18T14:50:56.458 INFO:tasks.workunit.client.1.smithi100.stdout:1/111: unlink d8/de/f16 0 2024-03-18T14:50:56.459 INFO:tasks.workunit.client.1.smithi100.stdout:6/75: dread d6/d10/f13 [0,4194304] 0 2024-03-18T14:50:56.459 INFO:tasks.workunit.client.0.smithi012.stdout:8/606: getdents d0/d5a/d19/dad 0 2024-03-18T14:50:56.459 INFO:tasks.workunit.client.0.smithi012.stdout:8/607: stat d0/d15/d6e/d84/d2a/d3e/lb6 0 2024-03-18T14:50:56.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/393: mkdir dc/d3c/d4c/d50/d54/d7a 0 2024-03-18T14:50:56.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/394: read dc/d30/d3a/f4f [317656,73708] 0 2024-03-18T14:50:56.461 INFO:tasks.workunit.client.1.smithi100.stdout:8/99: write d4/f6 [2337089,121228] 0 2024-03-18T14:50:56.463 INFO:tasks.workunit.client.0.smithi012.stdout:2/465: link d0/d10/d18/d31/c39 d0/d10/d18/d4f/c8e 0 2024-03-18T14:50:56.463 INFO:tasks.workunit.client.0.smithi012.stdout:2/466: chown d0/f6 13 1 2024-03-18T14:50:56.465 INFO:tasks.workunit.client.1.smithi100.stdout:5/89: read f18 [2314743,67418] 0 2024-03-18T14:50:56.465 INFO:tasks.workunit.client.0.smithi012.stdout:6/454: link d17/d26/f79 d17/d19/d27/d58/f8a 0 2024-03-18T14:50:56.466 INFO:tasks.workunit.client.0.smithi012.stdout:6/455: chown d17/f44 43 1 2024-03-18T14:50:56.466 INFO:tasks.workunit.client.0.smithi012.stdout:4/441: dwrite d6/d27/f70 [0,4194304] 0 2024-03-18T14:50:56.466 INFO:tasks.workunit.client.0.smithi012.stdout:4/442: write d6/fe [4425264,9063] 0 2024-03-18T14:50:56.467 INFO:tasks.workunit.client.1.smithi100.stdout:2/80: dwrite f0 [4194304,4194304] 0 2024-03-18T14:50:56.467 INFO:tasks.workunit.client.1.smithi100.stdout:2/81: chown d3/da/l11 5292 1 2024-03-18T14:50:56.468 INFO:tasks.workunit.client.0.smithi012.stdout:3/395: rename dc/f24 to dc/d3c/d72/f7b 0 2024-03-18T14:50:56.469 INFO:tasks.workunit.client.0.smithi012.stdout:3/396: fsync dc/d30/f51 0 2024-03-18T14:50:56.474 INFO:tasks.workunit.client.0.smithi012.stdout:2/467: link d0/d10/d18/l2b d0/d10/d18/d57/l8f 0 2024-03-18T14:50:56.476 INFO:tasks.workunit.client.0.smithi012.stdout:3/397: dread dc/d30/d3a/f5e [0,4194304] 0 2024-03-18T14:50:56.477 INFO:tasks.workunit.client.0.smithi012.stdout:3/398: fdatasync dc/d30/d3a/f4f 0 2024-03-18T14:50:56.479 INFO:tasks.workunit.client.0.smithi012.stdout:3/399: dread dc/f43 [0,4194304] 0 2024-03-18T14:50:56.479 INFO:tasks.workunit.client.0.smithi012.stdout:9/484: sync 2024-03-18T14:50:56.480 INFO:tasks.workunit.client.0.smithi012.stdout:7/453: sync 2024-03-18T14:50:56.480 INFO:tasks.workunit.client.0.smithi012.stdout:9/485: write d3/f6 [275918,46955] 0 2024-03-18T14:50:56.484 INFO:tasks.workunit.client.1.smithi100.stdout:6/76: mkdir d6/d7/d18 0 2024-03-18T14:50:56.485 INFO:tasks.workunit.client.0.smithi012.stdout:3/400: creat dc/d3c/d4c/d57/f7c x:0 0 0 2024-03-18T14:50:56.485 INFO:tasks.workunit.client.0.smithi012.stdout:3/401: read dc/f48 [1961933,72838] 0 2024-03-18T14:50:56.489 INFO:tasks.workunit.client.0.smithi012.stdout:2/468: write d0/d3c/d40/f5e [2569204,96559] 0 2024-03-18T14:50:56.489 INFO:tasks.workunit.client.0.smithi012.stdout:2/469: fsync d0/d10/d18/d31/d4c/f51 0 2024-03-18T14:50:56.489 INFO:tasks.workunit.client.0.smithi012.stdout:2/470: fdatasync d0/d10/d18/d4f/f60 0 2024-03-18T14:50:56.492 INFO:tasks.workunit.client.1.smithi100.stdout:1/112: dwrite d8/de/f18 [0,4194304] 0 2024-03-18T14:50:56.492 INFO:tasks.workunit.client.1.smithi100.stdout:1/113: truncate d8/f1a 654506 0 2024-03-18T14:50:56.493 INFO:tasks.workunit.client.0.smithi012.stdout:4/443: rename d6/d19/d4f to d6/d1a/d92 0 2024-03-18T14:50:56.493 INFO:tasks.workunit.client.0.smithi012.stdout:4/444: chown d6/d1a 95202 1 2024-03-18T14:50:56.495 INFO:tasks.workunit.client.0.smithi012.stdout:8/608: dwrite d0/d47/f4f [0,4194304] 0 2024-03-18T14:50:56.495 INFO:tasks.workunit.client.0.smithi012.stdout:7/454: write dd/d13/d23/f27 [1026357,19011] 0 2024-03-18T14:50:56.496 INFO:tasks.workunit.client.0.smithi012.stdout:6/456: dwrite d17/d19/d27/f54 [0,4194304] 0 2024-03-18T14:50:56.499 INFO:tasks.workunit.client.1.smithi100.stdout:2/82: symlink d3/d6/l1f 0 2024-03-18T14:50:56.501 INFO:tasks.workunit.client.0.smithi012.stdout:2/471: mkdir d0/d10/d18/d33/d71/d90 0 2024-03-18T14:50:56.502 INFO:tasks.workunit.client.1.smithi100.stdout:0/139: dread f3 [0,4194304] 0 2024-03-18T14:50:56.505 INFO:tasks.workunit.client.1.smithi100.stdout:6/77: unlink d6/d7/f9 0 2024-03-18T14:50:56.506 INFO:tasks.workunit.client.0.smithi012.stdout:7/455: getdents dd/d13/d23/d33/d37 0 2024-03-18T14:50:56.507 INFO:tasks.workunit.client.0.smithi012.stdout:8/609: dread d0/d5a/d19/f93 [0,4194304] 0 2024-03-18T14:50:56.508 INFO:tasks.workunit.client.1.smithi100.stdout:5/90: link le l1a 0 2024-03-18T14:50:56.509 INFO:tasks.workunit.client.1.smithi100.stdout:5/91: truncate f2 911677 0 2024-03-18T14:50:56.512 INFO:tasks.workunit.client.0.smithi012.stdout:2/472: dread d0/d10/d18/f3a [0,4194304] 0 2024-03-18T14:50:56.513 INFO:tasks.workunit.client.0.smithi012.stdout:2/473: write d0/d10/d18/d4f/f60 [962784,26793] 0 2024-03-18T14:50:56.513 INFO:tasks.workunit.client.0.smithi012.stdout:9/486: dwrite d3/d9/d19/d66/f44 [0,4194304] 0 2024-03-18T14:50:56.513 INFO:tasks.workunit.client.0.smithi012.stdout:9/487: truncate d3/d9/f29 4707293 0 2024-03-18T14:50:56.513 INFO:tasks.workunit.client.0.smithi012.stdout:7/456: dread dd/d13/d23/d33/f98 [0,4194304] 0 2024-03-18T14:50:56.515 INFO:tasks.workunit.client.0.smithi012.stdout:6/457: rename d17/f44 to d17/d26/d36/d4d/d82/f8b 0 2024-03-18T14:50:56.515 INFO:tasks.workunit.client.0.smithi012.stdout:6/458: truncate fb 1653005 0 2024-03-18T14:50:56.515 INFO:tasks.workunit.client.1.smithi100.stdout:3/99: sync 2024-03-18T14:50:56.515 INFO:tasks.workunit.client.1.smithi100.stdout:4/103: sync 2024-03-18T14:50:56.516 INFO:tasks.workunit.client.1.smithi100.stdout:1/114: dwrite d8/dd/ff [0,4194304] 0 2024-03-18T14:50:56.516 INFO:tasks.workunit.client.0.smithi012.stdout:3/402: dwrite dc/f2b [4194304,4194304] 0 2024-03-18T14:50:56.517 INFO:tasks.workunit.client.1.smithi100.stdout:2/83: mkdir d3/da/de/d20 0 2024-03-18T14:50:56.518 INFO:tasks.workunit.client.0.smithi012.stdout:8/610: link d0/d5a/c98 d0/d5a/d19/d63/dbe/cc1 0 2024-03-18T14:50:56.519 INFO:tasks.workunit.client.1.smithi100.stdout:0/140: mknod dd/d31/c34 0 2024-03-18T14:50:56.522 INFO:tasks.workunit.client.1.smithi100.stdout:0/141: dread dd/f1d [0,4194304] 0 2024-03-18T14:50:56.524 INFO:tasks.workunit.client.1.smithi100.stdout:6/78: rename c0 to d6/c19 0 2024-03-18T14:50:56.524 INFO:tasks.workunit.client.1.smithi100.stdout:1/115: write d8/de/f18 [730390,38768] 0 2024-03-18T14:50:56.527 INFO:tasks.workunit.client.0.smithi012.stdout:7/457: write dd/d13/f75 [2974699,26368] 0 2024-03-18T14:50:56.529 INFO:tasks.workunit.client.0.smithi012.stdout:2/474: mknod d0/d3c/d40/c91 0 2024-03-18T14:50:56.530 INFO:tasks.workunit.client.0.smithi012.stdout:2/475: write d0/d3c/d40/f4b [5027436,102239] 0 2024-03-18T14:50:56.530 INFO:tasks.workunit.client.0.smithi012.stdout:2/476: fdatasync d0/d10/d18/d57/f64 0 2024-03-18T14:50:56.530 INFO:tasks.workunit.client.1.smithi100.stdout:7/99: sync 2024-03-18T14:50:56.530 INFO:tasks.workunit.client.1.smithi100.stdout:9/95: sync 2024-03-18T14:50:56.531 INFO:tasks.workunit.client.1.smithi100.stdout:7/100: dread - d5/d14/f1e zero size 2024-03-18T14:50:56.531 INFO:tasks.workunit.client.1.smithi100.stdout:7/101: read - d5/d14/f1e zero size 2024-03-18T14:50:56.531 INFO:tasks.workunit.client.1.smithi100.stdout:3/100: creat d3/d16/d13/d17/f1a x:0 0 0 2024-03-18T14:50:56.534 INFO:tasks.workunit.client.0.smithi012.stdout:9/488: unlink d3/d49/c97 0 2024-03-18T14:50:56.534 INFO:tasks.workunit.client.0.smithi012.stdout:9/489: chown d3/d9/d19/f6a 1 1 2024-03-18T14:50:56.534 INFO:tasks.workunit.client.0.smithi012.stdout:9/490: fdatasync d3/d9/d19/d77/d52/d65/f86 0 2024-03-18T14:50:56.535 INFO:tasks.workunit.client.1.smithi100.stdout:4/104: dwrite d9/d17/f1d [0,4194304] 0 2024-03-18T14:50:56.535 INFO:tasks.workunit.client.1.smithi100.stdout:4/105: fsync d9/f1a 0 2024-03-18T14:50:56.538 INFO:tasks.workunit.client.0.smithi012.stdout:3/403: symlink dc/d3c/d72/l7d 0 2024-03-18T14:50:56.539 INFO:tasks.workunit.client.1.smithi100.stdout:0/142: mknod dd/d31/c35 0 2024-03-18T14:50:56.539 INFO:tasks.workunit.client.1.smithi100.stdout:0/143: stat c0 0 2024-03-18T14:50:56.540 INFO:tasks.workunit.client.0.smithi012.stdout:6/459: dwrite f12 [0,4194304] 0 2024-03-18T14:50:56.541 INFO:tasks.workunit.client.0.smithi012.stdout:2/477: dread d0/d10/d18/d33/f62 [0,4194304] 0 2024-03-18T14:50:56.541 INFO:tasks.workunit.client.0.smithi012.stdout:2/478: chown d0/d3c/d40/d47 28151 1 2024-03-18T14:50:56.542 INFO:tasks.workunit.client.0.smithi012.stdout:3/404: read dc/f43 [791856,43809] 0 2024-03-18T14:50:56.542 INFO:tasks.workunit.client.0.smithi012.stdout:3/405: stat dc/f14 0 2024-03-18T14:50:56.542 INFO:tasks.workunit.client.1.smithi100.stdout:2/84: rename d3/da/de/f19 to d3/d1a/f21 0 2024-03-18T14:50:56.544 INFO:tasks.workunit.client.0.smithi012.stdout:8/611: creat d0/d47/d59/d86/d87/fc2 x:0 0 0 2024-03-18T14:50:56.544 INFO:tasks.workunit.client.0.smithi012.stdout:8/612: readlink d0/d15/d6e/d84/l5b 0 2024-03-18T14:50:56.545 INFO:tasks.workunit.client.1.smithi100.stdout:6/79: unlink d6/d7/l11 0 2024-03-18T14:50:56.547 INFO:tasks.workunit.client.0.smithi012.stdout:7/458: link dd/d13/d23/d2f/d53/f8e dd/d13/d23/d2f/f9f 0 2024-03-18T14:50:56.549 INFO:tasks.workunit.client.1.smithi100.stdout:1/116: mknod d8/dd/c20 0 2024-03-18T14:50:56.551 INFO:tasks.workunit.client.0.smithi012.stdout:3/406: dread dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:56.551 INFO:tasks.workunit.client.0.smithi012.stdout:3/407: chown dc/le 83369 1 2024-03-18T14:50:56.551 INFO:tasks.workunit.client.0.smithi012.stdout:3/408: write dc/d30/d35/f2c [5102814,109760] 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:2/85: write d3/da/de/ff [1818399,32102] 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:2/86: chown d3/d1a/f21 34 1 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:3/101: dwrite d3/d16/d13/d17/f1a [0,4194304] 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:3/102: write d3/ff [164199,5489] 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:3/103: fsync d3/d16/fc 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:2/87: creat d3/f22 x:0 0 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:5/92: rename f8 to f1b 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:5/93: truncate f10 931589 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:5/94: write f10 [1922411,36855] 0 2024-03-18T14:50:56.553 INFO:tasks.workunit.client.1.smithi100.stdout:5/95: write f12 [1231634,33547] 0 2024-03-18T14:50:56.554 INFO:tasks.workunit.client.0.smithi012.stdout:8/613: write d0/d15/d6e/d84/d2a/fa1 [1318373,6800] 0 2024-03-18T14:50:56.554 INFO:tasks.workunit.client.1.smithi100.stdout:5/96: dread fa [0,4194304] 0 2024-03-18T14:50:56.554 INFO:tasks.workunit.client.1.smithi100.stdout:9/96: truncate f6 507019 0 2024-03-18T14:50:56.556 INFO:tasks.workunit.client.0.smithi012.stdout:9/491: mkdir d3/d9/d19/d77/d52/d99 0 2024-03-18T14:50:56.558 INFO:tasks.workunit.client.1.smithi100.stdout:7/102: mknod d5/d14/c27 0 2024-03-18T14:50:56.558 INFO:tasks.workunit.client.1.smithi100.stdout:1/117: read f7 [7739802,62078] 0 2024-03-18T14:50:56.558 INFO:tasks.workunit.client.1.smithi100.stdout:5/97: write f10 [837493,127021] 0 2024-03-18T14:50:56.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/106: symlink d9/d17/l26 0 2024-03-18T14:50:56.569 INFO:tasks.workunit.client.1.smithi100.stdout:6/80: mknod d6/c1a 0 2024-03-18T14:50:56.575 INFO:tasks.workunit.client.0.smithi012.stdout:2/479: mknod d0/d10/d18/d57/d61/c92 0 2024-03-18T14:50:56.576 INFO:tasks.workunit.client.0.smithi012.stdout:2/480: fsync d0/f7b 0 2024-03-18T14:50:56.576 INFO:tasks.workunit.client.1.smithi100.stdout:0/144: write dd/f1c [177747,49001] 0 2024-03-18T14:50:56.576 INFO:tasks.workunit.client.1.smithi100.stdout:0/145: stat fa 0 2024-03-18T14:50:56.577 INFO:tasks.workunit.client.0.smithi012.stdout:7/459: symlink dd/d13/d23/d33/la0 0 2024-03-18T14:50:56.578 INFO:tasks.workunit.client.1.smithi100.stdout:3/104: mknod d3/dd/c1b 0 2024-03-18T14:50:56.578 INFO:tasks.workunit.client.1.smithi100.stdout:3/105: chown d3/d16/fc 1 1 2024-03-18T14:50:56.581 INFO:tasks.workunit.client.0.smithi012.stdout:3/409: creat dc/d3c/d4c/d50/f7e x:0 0 0 2024-03-18T14:50:56.584 INFO:tasks.workunit.client.1.smithi100.stdout:9/97: creat da/f1f x:0 0 0 2024-03-18T14:50:56.587 INFO:tasks.workunit.client.1.smithi100.stdout:7/103: symlink d5/l28 0 2024-03-18T14:50:56.588 INFO:tasks.workunit.client.1.smithi100.stdout:7/104: chown l2 2265292 1 2024-03-18T14:50:56.591 INFO:tasks.workunit.client.0.smithi012.stdout:2/481: symlink d0/d10/d18/d33/d71/d90/l93 0 2024-03-18T14:50:56.593 INFO:tasks.workunit.client.1.smithi100.stdout:3/106: dwrite d3/d16/d13/d17/f1a [0,4194304] 0 2024-03-18T14:50:56.593 INFO:tasks.workunit.client.1.smithi100.stdout:3/107: chown d3/d16/f14 224144143 1 2024-03-18T14:50:56.593 INFO:tasks.workunit.client.0.smithi012.stdout:7/460: creat dd/fa1 x:0 0 0 2024-03-18T14:50:56.596 INFO:tasks.workunit.client.0.smithi012.stdout:3/410: link dc/d30/d35/c39 dc/d30/d35/c7f 0 2024-03-18T14:50:56.599 INFO:tasks.workunit.client.0.smithi012.stdout:8/614: getdents d0/d15/d6e 0 2024-03-18T14:50:56.599 INFO:tasks.workunit.client.0.smithi012.stdout:8/615: stat d0/d15/d6e/d84/l5b 0 2024-03-18T14:50:56.599 INFO:tasks.workunit.client.0.smithi012.stdout:8/616: write d0/d5a/d19/f9d [624597,72018] 0 2024-03-18T14:50:56.602 INFO:tasks.workunit.client.0.smithi012.stdout:9/492: rename d3/d9/d19/d66/f44 to d3/d9/f9a 0 2024-03-18T14:50:56.604 INFO:tasks.workunit.client.0.smithi012.stdout:2/482: creat d0/d3c/d40/d47/d85/f94 x:0 0 0 2024-03-18T14:50:56.605 INFO:tasks.workunit.client.0.smithi012.stdout:8/617: mknod d0/d15/d6e/d84/d54/d62/cc3 0 2024-03-18T14:50:56.606 INFO:tasks.workunit.client.1.smithi100.stdout:4/107: mkdir d9/d27 0 2024-03-18T14:50:56.607 INFO:tasks.workunit.client.1.smithi100.stdout:9/98: dwrite da/fc [0,4194304] 0 2024-03-18T14:50:56.607 INFO:tasks.workunit.client.1.smithi100.stdout:9/99: stat da/db/l17 0 2024-03-18T14:50:56.607 INFO:tasks.workunit.client.1.smithi100.stdout:9/100: creat da/f20 x:0 0 0 2024-03-18T14:50:56.608 INFO:tasks.workunit.client.0.smithi012.stdout:7/461: dwrite fa [4194304,4194304] 0 2024-03-18T14:50:56.608 INFO:tasks.workunit.client.0.smithi012.stdout:3/411: rename dc/d30/d35/c5c to dc/d30/d3a/c80 0 2024-03-18T14:50:56.611 INFO:tasks.workunit.client.1.smithi100.stdout:6/81: mkdir d6/d7/d1b 0 2024-03-18T14:50:56.611 INFO:tasks.workunit.client.1.smithi100.stdout:6/82: dread - d6/fe zero size 2024-03-18T14:50:56.611 INFO:tasks.workunit.client.1.smithi100.stdout:6/83: chown c2 216677891 1 2024-03-18T14:50:56.612 INFO:tasks.workunit.client.0.smithi012.stdout:7/462: write dd/f10 [1320896,14381] 0 2024-03-18T14:50:56.613 INFO:tasks.workunit.client.0.smithi012.stdout:9/493: unlink d3/d9/d19/l1d 0 2024-03-18T14:50:56.616 INFO:tasks.workunit.client.0.smithi012.stdout:8/618: mkdir d0/d47/d59/d86/dc4 0 2024-03-18T14:50:56.616 INFO:tasks.workunit.client.0.smithi012.stdout:8/619: chown d0/d5a/d19/d63/c79 81529599 1 2024-03-18T14:50:56.616 INFO:tasks.workunit.client.0.smithi012.stdout:8/620: chown d0/d15/d6e 209 1 2024-03-18T14:50:56.617 INFO:tasks.workunit.client.0.smithi012.stdout:8/621: truncate d0/d5a/d19/dad/d33/f5c 3104382 0 2024-03-18T14:50:56.618 INFO:tasks.workunit.client.1.smithi100.stdout:0/146: symlink dd/d2b/l36 0 2024-03-18T14:50:56.618 INFO:tasks.workunit.client.1.smithi100.stdout:0/147: chown dd/d31 96 1 2024-03-18T14:50:56.620 INFO:tasks.workunit.client.1.smithi100.stdout:2/88: getdents d3/d6 0 2024-03-18T14:50:56.622 INFO:tasks.workunit.client.0.smithi012.stdout:8/622: dread d0/d5a/d19/f93 [0,4194304] 0 2024-03-18T14:50:56.622 INFO:tasks.workunit.client.0.smithi012.stdout:8/623: stat d0/lae 0 2024-03-18T14:50:56.624 INFO:tasks.workunit.client.0.smithi012.stdout:3/412: rmdir dc/d3c/d4c/d50/d54/d7a 0 2024-03-18T14:50:56.626 INFO:tasks.workunit.client.1.smithi100.stdout:7/105: mknod d5/c29 0 2024-03-18T14:50:56.628 INFO:tasks.workunit.client.1.smithi100.stdout:5/98: symlink l1c 0 2024-03-18T14:50:56.629 INFO:tasks.workunit.client.1.smithi100.stdout:7/106: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:56.630 INFO:tasks.workunit.client.0.smithi012.stdout:9/494: rename d3/d9/d19/d66/d69/d7f/f82 to d3/d9/d19/d66/d69/d7f/f9b 0 2024-03-18T14:50:56.631 INFO:tasks.workunit.client.0.smithi012.stdout:8/624: symlink d0/d47/d59/d86/dc4/lc5 0 2024-03-18T14:50:56.634 INFO:tasks.workunit.client.1.smithi100.stdout:4/108: mknod d9/c28 0 2024-03-18T14:50:56.640 INFO:tasks.workunit.client.1.smithi100.stdout:6/84: creat d6/d7/d18/f1c x:0 0 0 2024-03-18T14:50:56.640 INFO:tasks.workunit.client.1.smithi100.stdout:6/85: fdatasync d6/d10/f14 0 2024-03-18T14:50:56.643 INFO:tasks.workunit.client.1.smithi100.stdout:0/148: mknod dd/d20/c37 0 2024-03-18T14:50:56.644 INFO:tasks.workunit.client.0.smithi012.stdout:9/495: mkdir d3/d9/d19/d77/d52/d99/d9c 0 2024-03-18T14:50:56.646 INFO:tasks.workunit.client.0.smithi012.stdout:8/625: stat d0/d5a/d19/c90 0 2024-03-18T14:50:56.646 INFO:tasks.workunit.client.0.smithi012.stdout:8/626: truncate d0/d5a/f8b 1676264 0 2024-03-18T14:50:56.647 INFO:tasks.workunit.client.0.smithi012.stdout:2/483: dwrite d0/d10/d18/d33/d71/f87 [0,4194304] 0 2024-03-18T14:50:56.647 INFO:tasks.workunit.client.0.smithi012.stdout:2/484: fdatasync d0/d3c/d40/d47/f49 0 2024-03-18T14:50:56.647 INFO:tasks.workunit.client.0.smithi012.stdout:2/485: creat d0/f95 x:0 0 0 2024-03-18T14:50:56.647 INFO:tasks.workunit.client.0.smithi012.stdout:2/486: read d0/d3c/d40/f4b [1463126,10712] 0 2024-03-18T14:50:56.648 INFO:tasks.workunit.client.1.smithi100.stdout:3/108: symlink d3/l1c 0 2024-03-18T14:50:56.648 INFO:tasks.workunit.client.1.smithi100.stdout:3/109: chown d3/dd 7337 1 2024-03-18T14:50:56.649 INFO:tasks.workunit.client.0.smithi012.stdout:3/413: getdents dc/d3c/d4c/d50 0 2024-03-18T14:50:56.649 INFO:tasks.workunit.client.0.smithi012.stdout:3/414: fsync dc/f6a 0 2024-03-18T14:50:56.650 INFO:tasks.workunit.client.1.smithi100.stdout:9/101: dwrite da/dd/f10 [4194304,4194304] 0 2024-03-18T14:50:56.650 INFO:tasks.workunit.client.1.smithi100.stdout:9/102: fsync da/f1d 0 2024-03-18T14:50:56.650 INFO:tasks.workunit.client.1.smithi100.stdout:7/107: dwrite d5/f10 [0,4194304] 0 2024-03-18T14:50:56.650 INFO:tasks.workunit.client.1.smithi100.stdout:9/103: creat da/dd/f21 x:0 0 0 2024-03-18T14:50:56.651 INFO:tasks.workunit.client.0.smithi012.stdout:7/463: dwrite dd/d13/d23/d33/d48/f5c [0,4194304] 0 2024-03-18T14:50:56.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/149: unlink dd/d20/c2f 0 2024-03-18T14:50:56.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/150: fsync dd/d20/f2c 0 2024-03-18T14:50:56.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/151: truncate dd/f11 4565149 0 2024-03-18T14:50:56.652 INFO:tasks.workunit.client.0.smithi012.stdout:9/496: symlink d3/d9/d19/d77/d52/d65/l9d 0 2024-03-18T14:50:56.652 INFO:tasks.workunit.client.1.smithi100.stdout:8/100: sync 2024-03-18T14:50:56.654 INFO:tasks.workunit.client.1.smithi100.stdout:6/86: write d6/d7/f8 [1565814,29807] 0 2024-03-18T14:50:56.654 INFO:tasks.workunit.client.1.smithi100.stdout:5/99: link l13 l1d 0 2024-03-18T14:50:56.657 INFO:tasks.workunit.client.0.smithi012.stdout:8/627: rename d0/d5a/d19/dad/f20 to d0/d15/d6e/d84/fc6 0 2024-03-18T14:50:56.657 INFO:tasks.workunit.client.0.smithi012.stdout:8/628: getdents d0/d5a/d19/dad/d33/d50/d6a 0 2024-03-18T14:50:56.661 INFO:tasks.workunit.client.1.smithi100.stdout:7/108: dread d5/fa [0,4194304] 0 2024-03-18T14:50:56.661 INFO:tasks.workunit.client.1.smithi100.stdout:4/109: dwrite d9/f1a [0,4194304] 0 2024-03-18T14:50:56.661 INFO:tasks.workunit.client.1.smithi100.stdout:4/110: truncate d9/d17/f24 248842 0 2024-03-18T14:50:56.662 INFO:tasks.workunit.client.1.smithi100.stdout:9/104: dread da/db/f13 [0,4194304] 0 2024-03-18T14:50:56.663 INFO:tasks.workunit.client.1.smithi100.stdout:3/110: dwrite d3/f10 [4194304,4194304] 0 2024-03-18T14:50:56.665 INFO:tasks.workunit.client.1.smithi100.stdout:0/152: creat dd/d2b/f38 x:0 0 0 2024-03-18T14:50:56.666 INFO:tasks.workunit.client.0.smithi012.stdout:2/487: link d0/d10/d18/l26 d0/l96 0 2024-03-18T14:50:56.668 INFO:tasks.workunit.client.0.smithi012.stdout:2/488: dread d0/d3c/f46 [0,4194304] 0 2024-03-18T14:50:56.668 INFO:tasks.workunit.client.0.smithi012.stdout:2/489: chown d0/d10/f13 85566 1 2024-03-18T14:50:56.668 INFO:tasks.workunit.client.1.smithi100.stdout:8/101: mkdir d4/dc/dd/de/d21/d23 0 2024-03-18T14:50:56.669 INFO:tasks.workunit.client.0.smithi012.stdout:3/415: creat dc/d30/f81 x:0 0 0 2024-03-18T14:50:56.669 INFO:tasks.workunit.client.0.smithi012.stdout:3/416: write f6 [2932231,22497] 0 2024-03-18T14:50:56.672 INFO:tasks.workunit.client.1.smithi100.stdout:6/87: link d6/cd d6/d7/d18/c1d 0 2024-03-18T14:50:56.672 INFO:tasks.workunit.client.1.smithi100.stdout:6/88: write d6/f15 [275073,126664] 0 2024-03-18T14:50:56.673 INFO:tasks.workunit.client.0.smithi012.stdout:9/497: mknod d3/d9/d19/d77/d52/d78/c9e 0 2024-03-18T14:50:56.674 INFO:tasks.workunit.client.0.smithi012.stdout:9/498: fdatasync d3/d9/d19/d77/d52/d65/f86 0 2024-03-18T14:50:56.676 INFO:tasks.workunit.client.1.smithi100.stdout:8/102: dread d4/ff [0,4194304] 0 2024-03-18T14:50:56.676 INFO:tasks.workunit.client.1.smithi100.stdout:8/103: write d4/dc/dd/f11 [3363678,93233] 0 2024-03-18T14:50:56.676 INFO:tasks.workunit.client.1.smithi100.stdout:8/104: write d4/dc/dd/de/d19/f1e [643848,51326] 0 2024-03-18T14:50:56.680 INFO:tasks.workunit.client.0.smithi012.stdout:3/417: symlink dc/d3c/d4c/d5f/l82 0 2024-03-18T14:50:56.681 INFO:tasks.workunit.client.0.smithi012.stdout:3/418: readlink dc/d30/d3a/l3d 0 2024-03-18T14:50:56.683 INFO:tasks.workunit.client.1.smithi100.stdout:9/105: link l3 da/l22 0 2024-03-18T14:50:56.683 INFO:tasks.workunit.client.1.smithi100.stdout:9/106: dread - da/dd/f15 zero size 2024-03-18T14:50:56.684 INFO:tasks.workunit.client.0.smithi012.stdout:3/419: symlink dc/d3c/d4c/d50/d54/d69/l83 0 2024-03-18T14:50:56.686 INFO:tasks.workunit.client.0.smithi012.stdout:9/499: dread d3/f3e [0,4194304] 0 2024-03-18T14:50:56.687 INFO:tasks.workunit.client.1.smithi100.stdout:3/111: symlink d3/dd/l1d 0 2024-03-18T14:50:56.688 INFO:tasks.workunit.client.0.smithi012.stdout:3/420: creat dc/d30/d3a/f84 x:0 0 0 2024-03-18T14:50:56.688 INFO:tasks.workunit.client.0.smithi012.stdout:3/421: stat dc/d3c 0 2024-03-18T14:50:56.690 INFO:tasks.workunit.client.0.smithi012.stdout:9/500: rename d3/d20/f47 to d3/d9/d19/d66/d69/d7f/d8c/d8b/f9f 0 2024-03-18T14:50:56.692 INFO:tasks.workunit.client.1.smithi100.stdout:6/89: creat d6/d7/df/f1e x:0 0 0 2024-03-18T14:50:56.695 INFO:tasks.workunit.client.0.smithi012.stdout:2/490: dwrite d0/d10/f6a [0,4194304] 0 2024-03-18T14:50:56.695 INFO:tasks.workunit.client.1.smithi100.stdout:5/100: mkdir d1e 0 2024-03-18T14:50:56.695 INFO:tasks.workunit.client.1.smithi100.stdout:8/105: dwrite d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:56.696 INFO:tasks.workunit.client.1.smithi100.stdout:3/112: dread d3/f5 [0,4194304] 0 2024-03-18T14:50:56.696 INFO:tasks.workunit.client.1.smithi100.stdout:3/113: rename d3 to d3/dd/d1e 22 2024-03-18T14:50:56.697 INFO:tasks.workunit.client.1.smithi100.stdout:3/114: read d3/ff [4189,75472] 0 2024-03-18T14:50:56.697 INFO:tasks.workunit.client.0.smithi012.stdout:2/491: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:56.697 INFO:tasks.workunit.client.0.smithi012.stdout:2/492: creat d0/f97 x:0 0 0 2024-03-18T14:50:56.698 INFO:tasks.workunit.client.0.smithi012.stdout:3/422: rename dc/d3c/c49 to dc/d3c/d4c/d50/d54/d69/c85 0 2024-03-18T14:50:56.698 INFO:tasks.workunit.client.0.smithi012.stdout:3/423: chown dc/d3c/d4c/d50/d54/l71 0 1 2024-03-18T14:50:56.701 INFO:tasks.workunit.client.0.smithi012.stdout:9/501: rename d3/d9/d19/d77/d52/d99/d9c to d3/d9/d19/d77/d52/d99/da0 0 2024-03-18T14:50:56.701 INFO:tasks.workunit.client.0.smithi012.stdout:9/502: stat d3/d9/d19/d77/d52/f60 0 2024-03-18T14:50:56.708 INFO:tasks.workunit.client.0.smithi012.stdout:2/493: read d0/d3c/d40/f5a [2117057,96703] 0 2024-03-18T14:50:56.708 INFO:tasks.workunit.client.0.smithi012.stdout:2/494: dread - d0/d10/d18/d31/d4c/d59/f8c zero size 2024-03-18T14:50:56.708 INFO:tasks.workunit.client.1.smithi100.stdout:5/101: dread f18 [0,4194304] 0 2024-03-18T14:50:56.710 INFO:tasks.workunit.client.1.smithi100.stdout:5/102: dread fa [0,4194304] 0 2024-03-18T14:50:56.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/424: dread dc/d30/f51 [0,4194304] 0 2024-03-18T14:50:56.719 INFO:tasks.workunit.client.0.smithi012.stdout:2/495: creat d0/d10/d18/f98 x:0 0 0 2024-03-18T14:50:56.719 INFO:tasks.workunit.client.1.smithi100.stdout:4/111: rmdir d9/d17 39 2024-03-18T14:50:56.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/496: dread d0/d10/d18/d31/d4c/f51 [0,4194304] 0 2024-03-18T14:50:56.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/497: stat d0/d3c 0 2024-03-18T14:50:56.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/498: write d0/f79 [947051,93964] 0 2024-03-18T14:50:56.721 INFO:tasks.workunit.client.0.smithi012.stdout:3/425: rename dc/d3c/d4c/d50/d54/d69 to dc/d3c/d59/d86 0 2024-03-18T14:50:56.723 INFO:tasks.workunit.client.1.smithi100.stdout:0/153: mkdir dd/d39 0 2024-03-18T14:50:56.724 INFO:tasks.workunit.client.0.smithi012.stdout:2/499: mkdir d0/d10/d18/d33/d99 0 2024-03-18T14:50:56.725 INFO:tasks.workunit.client.1.smithi100.stdout:6/90: creat d6/d12/f1f x:0 0 0 2024-03-18T14:50:56.725 INFO:tasks.workunit.client.1.smithi100.stdout:6/91: chown d6/d10/f14 1098797 1 2024-03-18T14:50:56.725 INFO:tasks.workunit.client.1.smithi100.stdout:6/92: write f4 [4034397,67218] 0 2024-03-18T14:50:56.725 INFO:tasks.workunit.client.1.smithi100.stdout:6/93: stat f4 0 2024-03-18T14:50:56.725 INFO:tasks.workunit.client.1.smithi100.stdout:6/94: read d6/d10/f13 [3426761,100922] 0 2024-03-18T14:50:56.726 INFO:tasks.workunit.client.1.smithi100.stdout:6/95: fsync d6/fb 0 2024-03-18T14:50:56.726 INFO:tasks.workunit.client.0.smithi012.stdout:3/426: symlink dc/d3c/d4c/d50/l87 0 2024-03-18T14:50:56.726 INFO:tasks.workunit.client.1.smithi100.stdout:4/112: dread d9/f22 [0,4194304] 0 2024-03-18T14:50:56.726 INFO:tasks.workunit.client.1.smithi100.stdout:8/106: link d4/dc/dd/de/d19/l1d d4/l24 0 2024-03-18T14:50:56.727 INFO:tasks.workunit.client.0.smithi012.stdout:2/500: mknod d0/d10/d18/d33/d71/d90/c9a 0 2024-03-18T14:50:56.728 INFO:tasks.workunit.client.1.smithi100.stdout:3/115: mkdir d3/d16/d13/d1f 0 2024-03-18T14:50:56.729 INFO:tasks.workunit.client.1.smithi100.stdout:3/116: chown d3/fa 22009 1 2024-03-18T14:50:56.729 INFO:tasks.workunit.client.1.smithi100.stdout:3/117: stat d3/dd/l1d 0 2024-03-18T14:50:56.729 INFO:tasks.workunit.client.0.smithi012.stdout:3/427: readlink dc/l1e 0 2024-03-18T14:50:56.731 INFO:tasks.workunit.client.0.smithi012.stdout:2/501: link d0/d10/d18/d33/f62 d0/d3c/d40/d47/f9b 0 2024-03-18T14:50:56.732 INFO:tasks.workunit.client.1.smithi100.stdout:5/103: rmdir d1e 0 2024-03-18T14:50:56.733 INFO:tasks.workunit.client.1.smithi100.stdout:9/107: link da/dd/f21 da/db/f23 0 2024-03-18T14:50:56.738 INFO:tasks.workunit.client.0.smithi012.stdout:8/629: dwrite d0/d15/f28 [0,4194304] 0 2024-03-18T14:50:56.738 INFO:tasks.workunit.client.0.smithi012.stdout:8/630: chown d0/d5a/d19/f9d 13895519 1 2024-03-18T14:50:56.738 INFO:tasks.workunit.client.0.smithi012.stdout:8/631: fdatasync d0/f7 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.0.smithi012.stdout:9/503: dwrite d3/d9/d19/d66/d21/d35/d71/f96 [0,4194304] 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:8/107: mkdir d4/dc/dd/de/d21/d23/d25 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:3/118: mknod d3/d16/c20 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/108: mkdir da/db/d24 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/109: chown da 12414 1 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/110: chown da/db/d24 19044288 1 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:6/96: getdents d6/d10 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:6/97: readlink l5 0 2024-03-18T14:50:56.739 INFO:tasks.workunit.client.1.smithi100.stdout:3/119: mkdir d3/dd/d21 0 2024-03-18T14:50:56.740 INFO:tasks.workunit.client.1.smithi100.stdout:5/104: mknod c1f 0 2024-03-18T14:50:56.742 INFO:tasks.workunit.client.1.smithi100.stdout:9/111: mkdir da/d25 0 2024-03-18T14:50:56.742 INFO:tasks.workunit.client.1.smithi100.stdout:9/112: stat da/db/f23 0 2024-03-18T14:50:56.742 INFO:tasks.workunit.client.1.smithi100.stdout:9/113: chown l8 466897946 1 2024-03-18T14:50:56.744 INFO:tasks.workunit.client.0.smithi012.stdout:2/502: write d0/d10/f16 [929966,70822] 0 2024-03-18T14:50:56.744 INFO:tasks.workunit.client.0.smithi012.stdout:2/503: chown d0/d10/d18/d57/d61/l63 3051107 1 2024-03-18T14:50:56.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/98: mknod d6/d7/d18/c20 0 2024-03-18T14:50:56.745 INFO:tasks.workunit.client.1.smithi100.stdout:3/120: dread d3/f9 [0,4194304] 0 2024-03-18T14:50:56.747 INFO:tasks.workunit.client.1.smithi100.stdout:8/108: getdents d4/dc/dd 0 2024-03-18T14:50:56.748 INFO:tasks.workunit.client.1.smithi100.stdout:8/109: readlink d4/dc/dd/de/l16 0 2024-03-18T14:50:56.750 INFO:tasks.workunit.client.0.smithi012.stdout:8/632: symlink d0/d15/d6e/lc7 0 2024-03-18T14:50:56.755 INFO:tasks.workunit.client.1.smithi100.stdout:4/113: dwrite d9/f1a [4194304,4194304] 0 2024-03-18T14:50:56.755 INFO:tasks.workunit.client.1.smithi100.stdout:9/114: write da/f1d [2530126,78511] 0 2024-03-18T14:50:56.755 INFO:tasks.workunit.client.1.smithi100.stdout:9/115: write da/db/f1a [4813143,101737] 0 2024-03-18T14:50:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:9/504: write d3/d9/d19/d66/d21/f26 [219512,113285] 0 2024-03-18T14:50:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:9/505: chown d3/d9/d19/d66/d69/c89 0 1 2024-03-18T14:50:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:9/506: dread - d3/d9/d19/d66/d21/f90 zero size 2024-03-18T14:50:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:8/633: mkdir d0/d15/d6e/d84/dc8 0 2024-03-18T14:50:56.756 INFO:tasks.workunit.client.0.smithi012.stdout:8/634: truncate d0/d5a/f1d 5839628 0 2024-03-18T14:50:56.757 INFO:tasks.workunit.client.1.smithi100.stdout:3/121: truncate d3/d16/d13/d17/f1a 1525031 0 2024-03-18T14:50:56.758 INFO:tasks.workunit.client.1.smithi100.stdout:8/110: unlink c0 0 2024-03-18T14:50:56.760 INFO:tasks.workunit.client.1.smithi100.stdout:5/105: link f17 f20 0 2024-03-18T14:50:56.761 INFO:tasks.workunit.client.1.smithi100.stdout:0/154: dwrite dd/f16 [0,4194304] 0 2024-03-18T14:50:56.761 INFO:tasks.workunit.client.0.smithi012.stdout:9/507: rmdir d3/d9/d19/d77/d52/d99 39 2024-03-18T14:50:56.762 INFO:tasks.workunit.client.1.smithi100.stdout:5/106: write f12 [685411,23060] 0 2024-03-18T14:50:56.762 INFO:tasks.workunit.client.0.smithi012.stdout:2/504: dwrite d0/d10/d18/d31/d4c/d59/f8a [0,4194304] 0 2024-03-18T14:50:56.765 INFO:tasks.workunit.client.1.smithi100.stdout:1/118: sync 2024-03-18T14:50:56.765 INFO:tasks.workunit.client.1.smithi100.stdout:1/119: chown d8/d1e/d11/f1b 22062 1 2024-03-18T14:50:56.767 INFO:tasks.workunit.client.0.smithi012.stdout:9/508: mknod d3/d9/d19/d66/d21/d35/d48/ca1 0 2024-03-18T14:50:56.771 INFO:tasks.workunit.client.1.smithi100.stdout:2/89: sync 2024-03-18T14:50:56.775 INFO:tasks.workunit.client.1.smithi100.stdout:4/114: truncate d9/d17/f1d 2289733 0 2024-03-18T14:50:56.781 INFO:tasks.workunit.client.1.smithi100.stdout:9/116: rmdir da/db/d24 0 2024-03-18T14:50:56.781 INFO:tasks.workunit.client.1.smithi100.stdout:9/117: rename da/db to da/db/d26 22 2024-03-18T14:50:56.781 INFO:tasks.workunit.client.1.smithi100.stdout:9/118: fdatasync f2 0 2024-03-18T14:50:56.781 INFO:tasks.workunit.client.1.smithi100.stdout:9/119: truncate f9 8580086 0 2024-03-18T14:50:56.781 INFO:tasks.workunit.client.1.smithi100.stdout:6/99: rmdir d6/d7 39 2024-03-18T14:50:56.782 INFO:tasks.workunit.client.1.smithi100.stdout:6/100: fdatasync d6/d7/d18/f1c 0 2024-03-18T14:50:56.782 INFO:tasks.workunit.client.1.smithi100.stdout:3/122: creat d3/dd/d21/f22 x:0 0 0 2024-03-18T14:50:56.787 INFO:tasks.workunit.client.0.smithi012.stdout:8/635: rmdir d0/d47/d59/d86/d87 39 2024-03-18T14:50:56.787 INFO:tasks.workunit.client.0.smithi012.stdout:8/636: chown d0/d5a/d19/f3a 13058289 1 2024-03-18T14:50:56.789 INFO:tasks.workunit.client.0.smithi012.stdout:8/637: rename d0/d47/d59/d86 to d0/d47/d59/dc9 0 2024-03-18T14:50:56.789 INFO:tasks.workunit.client.0.smithi012.stdout:8/638: creat d0/d47/d59/fca x:0 0 0 2024-03-18T14:50:56.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/509: dwrite d3/d9/d19/f3a [0,4194304] 0 2024-03-18T14:50:56.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/510: write d3/f63 [437989,54575] 0 2024-03-18T14:50:56.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/639: getdents d0/d5a/d19/d63 0 2024-03-18T14:50:56.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/640: dread - d0/d15/d6e/d84/d54/d62/daf/d80/f92 zero size 2024-03-18T14:50:56.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/641: fsync d0/d5a/d19/d57/daa/fb9 0 2024-03-18T14:50:56.794 INFO:tasks.workunit.client.0.smithi012.stdout:8/642: stat d0/d5a/d19/dad/f2d 0 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/155: creat dd/f3a x:0 0 0 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/156: chown dd/f1c 16568 1 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/157: truncate dd/d20/f32 42115 0 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/158: readlink dd/d2b/l36 0 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/159: creat dd/d20/f3b x:0 0 0 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/160: chown cc 137968922 1 2024-03-18T14:50:56.796 INFO:tasks.workunit.client.1.smithi100.stdout:0/161: write dd/d20/f32 [93836,99396] 0 2024-03-18T14:50:56.797 INFO:tasks.workunit.client.1.smithi100.stdout:9/120: dread da/dd/f10 [4194304,4194304] 0 2024-03-18T14:50:56.797 INFO:tasks.workunit.client.1.smithi100.stdout:9/121: creat da/dd/f27 x:0 0 0 2024-03-18T14:50:56.797 INFO:tasks.workunit.client.1.smithi100.stdout:9/122: truncate da/dd/f27 619919 0 2024-03-18T14:50:56.799 INFO:tasks.workunit.client.1.smithi100.stdout:9/123: fsync da/fc 0 2024-03-18T14:50:56.799 INFO:tasks.workunit.client.1.smithi100.stdout:9/124: write da/f1d [989023,111552] 0 2024-03-18T14:50:56.801 INFO:tasks.workunit.client.0.smithi012.stdout:8/643: rmdir d0/d15/d6e/d84/d54/d62/daf 39 2024-03-18T14:50:56.801 INFO:tasks.workunit.client.0.smithi012.stdout:8/644: fdatasync d0/d5a/f9a 0 2024-03-18T14:50:56.801 INFO:tasks.workunit.client.0.smithi012.stdout:8/645: truncate d0/f6 2999403 0 2024-03-18T14:50:56.802 INFO:tasks.workunit.client.1.smithi100.stdout:1/120: truncate f7 4888711 0 2024-03-18T14:50:56.808 INFO:tasks.workunit.client.0.smithi012.stdout:9/511: rename d3/d9/f9a to d3/d9/fa2 0 2024-03-18T14:50:56.808 INFO:tasks.workunit.client.0.smithi012.stdout:9/512: dread - d3/d9/d19/d66/d21/f8e zero size 2024-03-18T14:50:56.808 INFO:tasks.workunit.client.1.smithi100.stdout:2/90: creat d3/da/de/d20/f23 x:0 0 0 2024-03-18T14:50:56.810 INFO:tasks.workunit.client.1.smithi100.stdout:4/115: creat d9/d27/f29 x:0 0 0 2024-03-18T14:50:56.810 INFO:tasks.workunit.client.0.smithi012.stdout:8/646: dread d0/d5a/f1d [0,4194304] 0 2024-03-18T14:50:56.813 INFO:tasks.workunit.client.1.smithi100.stdout:6/101: rename c2 to d6/d10/c21 0 2024-03-18T14:50:56.813 INFO:tasks.workunit.client.1.smithi100.stdout:6/102: fdatasync d6/f15 0 2024-03-18T14:50:56.814 INFO:tasks.workunit.client.0.smithi012.stdout:9/513: mkdir d3/d9/d19/d77/d52/d95/da3 0 2024-03-18T14:50:56.817 INFO:tasks.workunit.client.1.smithi100.stdout:9/125: dwrite da/dd/f15 [0,4194304] 0 2024-03-18T14:50:56.817 INFO:tasks.workunit.client.1.smithi100.stdout:9/126: getdents da/d25 0 2024-03-18T14:50:56.818 INFO:tasks.workunit.client.0.smithi012.stdout:8/647: link d0/d15/d6e/d84/d2a/f5d d0/d5a/fcb 0 2024-03-18T14:50:56.819 INFO:tasks.workunit.client.0.smithi012.stdout:8/648: chown d0/d5a/d19/dad/cb5 132340 1 2024-03-18T14:50:56.819 INFO:tasks.workunit.client.0.smithi012.stdout:8/649: chown d0/d47/d59/f83 33821734 1 2024-03-18T14:50:56.819 INFO:tasks.workunit.client.1.smithi100.stdout:8/111: rmdir d4/dc/d1f 39 2024-03-18T14:50:56.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/109: sync 2024-03-18T14:50:56.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/110: write d5/d14/f1e [451275,94444] 0 2024-03-18T14:50:56.821 INFO:tasks.workunit.client.0.smithi012.stdout:9/514: creat d3/d9/d19/d66/d21/d35/d48/fa4 x:0 0 0 2024-03-18T14:50:56.822 INFO:tasks.workunit.client.0.smithi012.stdout:9/515: chown d3/f41 923 1 2024-03-18T14:50:56.824 INFO:tasks.workunit.client.1.smithi100.stdout:0/162: truncate dd/f11 3411289 0 2024-03-18T14:50:56.826 INFO:tasks.workunit.client.1.smithi100.stdout:7/111: dread d5/d14/f1e [0,4194304] 0 2024-03-18T14:50:56.826 INFO:tasks.workunit.client.1.smithi100.stdout:7/112: chown d5/d14/d16/c19 15936 1 2024-03-18T14:50:56.827 INFO:tasks.workunit.client.0.smithi012.stdout:8/650: link d0/d5a/d55/lab d0/d5a/lcc 0 2024-03-18T14:50:56.829 INFO:tasks.workunit.client.0.smithi012.stdout:9/516: write d3/d9/d19/d66/f2b [3674831,118760] 0 2024-03-18T14:50:56.832 INFO:tasks.workunit.client.1.smithi100.stdout:5/107: symlink l21 0 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:0/439: sync 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:5/466: sync 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:1/420: sync 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:5/467: creat d3/d84/f9f x:0 0 0 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:0/440: write d1/d20/d23/d6f/f75 [1014925,58401] 0 2024-03-18T14:50:56.833 INFO:tasks.workunit.client.0.smithi012.stdout:0/441: readlink d1/d20/d23/l69 0 2024-03-18T14:50:56.835 INFO:tasks.workunit.client.0.smithi012.stdout:9/517: rename d3/d9/d19/d66/d21/d35/d48/l80 to d3/d9/d19/d66/d21/la5 0 2024-03-18T14:50:56.837 INFO:tasks.workunit.client.0.smithi012.stdout:8/651: write d0/d5a/d19/fa3 [673660,9525] 0 2024-03-18T14:50:56.837 INFO:tasks.workunit.client.1.smithi100.stdout:2/91: truncate d3/f10 1100437 0 2024-03-18T14:50:56.838 INFO:tasks.workunit.client.1.smithi100.stdout:2/92: creat d3/da/de/d20/f24 x:0 0 0 2024-03-18T14:50:56.838 INFO:tasks.workunit.client.1.smithi100.stdout:2/93: write f0 [9091166,93668] 0 2024-03-18T14:50:56.842 INFO:tasks.workunit.client.0.smithi012.stdout:5/468: creat d3/d29/d36/d9b/d59/d8b/fa0 x:0 0 0 2024-03-18T14:50:56.846 INFO:tasks.workunit.client.1.smithi100.stdout:3/123: rmdir d3/dd 39 2024-03-18T14:50:56.848 INFO:tasks.workunit.client.0.smithi012.stdout:0/442: write d1/d20/d7b/ff [1931142,18385] 0 2024-03-18T14:50:56.848 INFO:tasks.workunit.client.0.smithi012.stdout:0/443: write d1/d20/d23/d26/f40 [1390239,14393] 0 2024-03-18T14:50:56.848 INFO:tasks.workunit.client.0.smithi012.stdout:0/444: write d1/d20/d23/d3b/f2c [4748864,77798] 0 2024-03-18T14:50:56.848 INFO:tasks.workunit.client.0.smithi012.stdout:0/445: creat d1/d20/d4e/d54/d67/f9a x:0 0 0 2024-03-18T14:50:56.848 INFO:tasks.workunit.client.0.smithi012.stdout:0/446: chown d1/d16/d79/f8f 62319177 1 2024-03-18T14:50:56.849 INFO:tasks.workunit.client.0.smithi012.stdout:9/518: rename d3/d9/d19/d66/d69/c89 to d3/d9/d19/d66/d21/d35/d48/ca6 0 2024-03-18T14:50:56.853 INFO:tasks.workunit.client.0.smithi012.stdout:8/652: write d0/d5a/d19/dad/f4d [102368,29111] 0 2024-03-18T14:50:56.853 INFO:tasks.workunit.client.0.smithi012.stdout:8/653: write d0/f10 [3472432,12174] 0 2024-03-18T14:50:56.853 INFO:tasks.workunit.client.0.smithi012.stdout:5/469: creat d3/d29/d36/d3d/d65/fa1 x:0 0 0 2024-03-18T14:50:56.859 INFO:tasks.workunit.client.1.smithi100.stdout:9/127: rename da/dd/f15 to da/dd/f28 0 2024-03-18T14:50:56.863 INFO:tasks.workunit.client.1.smithi100.stdout:8/112: symlink d4/l26 0 2024-03-18T14:50:56.864 INFO:tasks.workunit.client.1.smithi100.stdout:8/113: stat d4/dc/dd/de/d21/d23 0 2024-03-18T14:50:56.866 INFO:tasks.workunit.client.0.smithi012.stdout:8/654: dread d0/d15/d6e/d84/f77 [0,4194304] 0 2024-03-18T14:50:56.866 INFO:tasks.workunit.client.0.smithi012.stdout:8/655: write d0/d15/d6e/d84/d54/f72 [2334364,62762] 0 2024-03-18T14:50:56.866 INFO:tasks.workunit.client.0.smithi012.stdout:1/421: dwrite d8/f5a [0,4194304] 0 2024-03-18T14:50:56.866 INFO:tasks.workunit.client.0.smithi012.stdout:8/656: write d0/d5a/d19/d57/f66 [655174,55804] 0 2024-03-18T14:50:56.868 INFO:tasks.workunit.client.1.smithi100.stdout:7/113: symlink d5/d14/l2a 0 2024-03-18T14:50:56.869 INFO:tasks.workunit.client.0.smithi012.stdout:5/470: getdents d3/d29/d35 0 2024-03-18T14:50:56.870 INFO:tasks.workunit.client.0.smithi012.stdout:9/519: dwrite d3/d9/d5c/f85 [0,4194304] 0 2024-03-18T14:50:56.870 INFO:tasks.workunit.client.0.smithi012.stdout:9/520: stat d3/d9/d19/d66/d69/d7f/f9b 0 2024-03-18T14:50:56.872 INFO:tasks.workunit.client.0.smithi012.stdout:0/447: link d1/d20/d4e/d54/d67/f9a d1/d16/d6e/f9b 0 2024-03-18T14:50:56.875 INFO:tasks.workunit.client.1.smithi100.stdout:2/94: creat d3/da/f25 x:0 0 0 2024-03-18T14:50:56.876 INFO:tasks.workunit.client.1.smithi100.stdout:2/95: write d3/da/f25 [707167,9789] 0 2024-03-18T14:50:56.877 INFO:tasks.workunit.client.1.smithi100.stdout:3/124: symlink d3/dd/l23 0 2024-03-18T14:50:56.878 INFO:tasks.workunit.client.0.smithi012.stdout:1/422: truncate d8/db/d10/d31/f4d 76392 0 2024-03-18T14:50:56.878 INFO:tasks.workunit.client.0.smithi012.stdout:1/423: chown d8/db/d10/d31/d45/l57 3 1 2024-03-18T14:50:56.878 INFO:tasks.workunit.client.0.smithi012.stdout:1/424: chown d8/f5a 47521721 1 2024-03-18T14:50:56.879 INFO:tasks.workunit.client.0.smithi012.stdout:1/425: fdatasync d8/d1e/d27/d44/f4f 0 2024-03-18T14:50:56.879 INFO:tasks.workunit.client.1.smithi100.stdout:9/128: mkdir da/dd/d29 0 2024-03-18T14:50:56.879 INFO:tasks.workunit.client.1.smithi100.stdout:9/129: readlink l3 0 2024-03-18T14:50:56.880 INFO:tasks.workunit.client.0.smithi012.stdout:8/657: mknod d0/d15/d6e/d84/d54/d62/daf/ccd 0 2024-03-18T14:50:56.880 INFO:tasks.workunit.client.0.smithi012.stdout:8/658: chown d0/d15/d6e/d84/d54/d62/cc3 89047 1 2024-03-18T14:50:56.880 INFO:tasks.workunit.client.0.smithi012.stdout:8/659: fsync d0/d5a/f7d 0 2024-03-18T14:50:56.880 INFO:tasks.workunit.client.0.smithi012.stdout:8/660: chown d0/d15/l88 1 1 2024-03-18T14:50:56.881 INFO:tasks.workunit.client.1.smithi100.stdout:8/114: link d4/dc/d1f/f20 d4/dc/dd/de/d21/d23/f27 0 2024-03-18T14:50:56.882 INFO:tasks.workunit.client.1.smithi100.stdout:0/163: mknod dd/c3c 0 2024-03-18T14:50:56.882 INFO:tasks.workunit.client.1.smithi100.stdout:0/164: creat dd/d2b/f3d x:0 0 0 2024-03-18T14:50:56.883 INFO:tasks.workunit.client.1.smithi100.stdout:0/165: creat dd/d20/f3e x:0 0 0 2024-03-18T14:50:56.883 INFO:tasks.workunit.client.1.smithi100.stdout:0/166: write dd/d20/f3b [474287,23043] 0 2024-03-18T14:50:56.883 INFO:tasks.workunit.client.1.smithi100.stdout:7/114: creat d5/d14/d16/f2b x:0 0 0 2024-03-18T14:50:56.885 INFO:tasks.workunit.client.1.smithi100.stdout:5/108: symlink l22 0 2024-03-18T14:50:56.886 INFO:tasks.workunit.client.1.smithi100.stdout:5/109: write fb [2172029,93844] 0 2024-03-18T14:50:56.886 INFO:tasks.workunit.client.1.smithi100.stdout:3/125: write d3/f10 [3513996,68733] 0 2024-03-18T14:50:56.887 INFO:tasks.workunit.client.1.smithi100.stdout:9/130: unlink da/dd/f21 0 2024-03-18T14:50:56.887 INFO:tasks.workunit.client.1.smithi100.stdout:9/131: fdatasync da/db/f13 0 2024-03-18T14:50:56.888 INFO:tasks.workunit.client.0.smithi012.stdout:1/426: mknod d8/db/d10/d31/d45/c80 0 2024-03-18T14:50:56.888 INFO:tasks.workunit.client.0.smithi012.stdout:1/427: fdatasync d8/db/f7d 0 2024-03-18T14:50:56.888 INFO:tasks.workunit.client.0.smithi012.stdout:1/428: write d8/f1a [4774953,93331] 0 2024-03-18T14:50:56.888 INFO:tasks.workunit.client.0.smithi012.stdout:1/429: write d8/f1a [5905608,106233] 0 2024-03-18T14:50:56.896 INFO:tasks.workunit.client.1.smithi100.stdout:7/115: dwrite d5/d14/d16/f2b [0,4194304] 0 2024-03-18T14:50:56.927 INFO:tasks.workunit.client.1.smithi100.stdout:8/115: creat d4/dc/dd/de/d21/d23/f28 x:0 0 0 2024-03-18T14:50:56.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/96: fsync d3/da/de/d20/f24 0 2024-03-18T14:50:56.927 INFO:tasks.workunit.client.1.smithi100.stdout:2/97: stat d3 0 2024-03-18T14:50:56.928 INFO:tasks.workunit.client.0.smithi012.stdout:8/661: unlink d0/d5a/d19/dad/lbf 0 2024-03-18T14:50:56.929 INFO:tasks.workunit.client.0.smithi012.stdout:5/471: truncate d3/d29/d35/f48 1335914 0 2024-03-18T14:50:56.935 INFO:tasks.workunit.client.0.smithi012.stdout:1/430: dwrite d8/d1e/d27/d44/f59 [0,4194304] 0 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/662: write d0/d5a/d19/f3a [1686737,112522] 0 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/663: chown d0/d5a/d55/c91 293347 1 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/664: chown d0/d15/d6e/d84/d2a/fa1 4029 1 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/665: write d0/d5a/f6d [950565,96707] 0 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/666: truncate d0/d5a/d19/dad/d33/f44 146884 0 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/667: creat d0/d5a/d19/fce x:0 0 0 2024-03-18T14:50:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:8/668: chown d0/lae 5 1 2024-03-18T14:50:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:1/431: dread d8/d1e/d27/f38 [0,4194304] 0 2024-03-18T14:50:57.042 INFO:tasks.workunit.client.1.smithi100.stdout:0/167: rename dd/ff to dd/d39/f3f 0 2024-03-18T14:50:57.042 INFO:tasks.workunit.client.1.smithi100.stdout:0/168: write dd/d2b/f38 [683067,2258] 0 2024-03-18T14:50:57.042 INFO:tasks.workunit.client.1.smithi100.stdout:0/169: creat dd/d2b/f40 x:0 0 0 2024-03-18T14:50:57.045 INFO:tasks.workunit.client.1.smithi100.stdout:5/110: creat f23 x:0 0 0 2024-03-18T14:50:57.048 INFO:tasks.workunit.client.1.smithi100.stdout:3/126: creat d3/dd/f24 x:0 0 0 2024-03-18T14:50:57.048 INFO:tasks.workunit.client.1.smithi100.stdout:3/127: write d3/d16/f14 [3917129,81704] 0 2024-03-18T14:50:57.050 INFO:tasks.workunit.client.1.smithi100.stdout:9/132: mknod da/dd/c2a 0 2024-03-18T14:50:57.051 INFO:tasks.workunit.client.1.smithi100.stdout:9/133: dread da/dd/f27 [0,4194304] 0 2024-03-18T14:50:57.051 INFO:tasks.workunit.client.1.smithi100.stdout:9/134: readlink da/db/l14 0 2024-03-18T14:50:57.051 INFO:tasks.workunit.client.1.smithi100.stdout:9/135: fdatasync da/f16 0 2024-03-18T14:50:57.051 INFO:tasks.workunit.client.1.smithi100.stdout:7/116: mkdir d5/d14/d16/d2c 0 2024-03-18T14:50:57.052 INFO:tasks.workunit.client.1.smithi100.stdout:3/128: write d3/d16/d13/d17/f1a [1417563,130738] 0 2024-03-18T14:50:57.053 INFO:tasks.workunit.client.0.smithi012.stdout:9/521: rename d3/d9/d5c/f62 to d3/fa7 0 2024-03-18T14:50:57.053 INFO:tasks.workunit.client.0.smithi012.stdout:9/522: readlink d3/d49/l5d 0 2024-03-18T14:50:57.055 INFO:tasks.workunit.client.1.smithi100.stdout:8/116: link d4/dc/dd/l1a d4/dc/dd/de/l29 0 2024-03-18T14:50:57.056 INFO:tasks.workunit.client.0.smithi012.stdout:0/448: getdents d1/d20/d23/d3b/d33 0 2024-03-18T14:50:57.057 INFO:tasks.workunit.client.1.smithi100.stdout:2/98: creat d3/d6/f26 x:0 0 0 2024-03-18T14:50:57.057 INFO:tasks.workunit.client.1.smithi100.stdout:2/99: creat d3/da/de/d20/f27 x:0 0 0 2024-03-18T14:50:57.057 INFO:tasks.workunit.client.1.smithi100.stdout:2/100: creat d3/da/de/f28 x:0 0 0 2024-03-18T14:50:57.058 INFO:tasks.workunit.client.1.smithi100.stdout:2/101: chown d3/f4 4 1 2024-03-18T14:50:57.060 INFO:tasks.workunit.client.1.smithi100.stdout:5/111: dwrite fa [0,4194304] 0 2024-03-18T14:50:57.062 INFO:tasks.workunit.client.0.smithi012.stdout:5/472: rename d3/d29/d36/d9b/c8a to d3/d29/d36/d9b/ca2 0 2024-03-18T14:50:57.068 INFO:tasks.workunit.client.0.smithi012.stdout:8/669: symlink d0/d5a/d19/d63/lcf 0 2024-03-18T14:50:57.070 INFO:tasks.workunit.client.0.smithi012.stdout:1/432: mkdir d8/db/d10/d31/d81 0 2024-03-18T14:50:57.070 INFO:tasks.workunit.client.0.smithi012.stdout:1/433: write d8/db/d10/d16/f66 [3373272,15408] 0 2024-03-18T14:50:57.072 INFO:tasks.workunit.client.1.smithi100.stdout:2/102: dwrite d3/da/de/d20/f27 [0,4194304] 0 2024-03-18T14:50:57.077 INFO:tasks.workunit.client.1.smithi100.stdout:7/117: creat d5/d22/d24/f2d x:0 0 0 2024-03-18T14:50:57.077 INFO:tasks.workunit.client.0.smithi012.stdout:9/523: getdents d3/d9/d19/d77/d52 0 2024-03-18T14:50:57.082 INFO:tasks.workunit.client.1.smithi100.stdout:3/129: symlink d3/d16/d13/d1f/l25 0 2024-03-18T14:50:57.083 INFO:tasks.workunit.client.0.smithi012.stdout:6/460: sync 2024-03-18T14:50:57.084 INFO:tasks.workunit.client.0.smithi012.stdout:4/445: sync 2024-03-18T14:50:57.084 INFO:tasks.workunit.client.0.smithi012.stdout:5/473: symlink d3/d29/d36/d9b/d59/la3 0 2024-03-18T14:50:57.086 INFO:tasks.workunit.client.1.smithi100.stdout:8/117: mknod d4/dc/c2a 0 2024-03-18T14:50:57.086 INFO:tasks.workunit.client.1.smithi100.stdout:8/118: chown d4/dc/dd/de/l16 16 1 2024-03-18T14:50:57.090 INFO:tasks.workunit.client.1.smithi100.stdout:1/121: sync 2024-03-18T14:50:57.090 INFO:tasks.workunit.client.1.smithi100.stdout:6/103: sync 2024-03-18T14:50:57.090 INFO:tasks.workunit.client.1.smithi100.stdout:4/116: sync 2024-03-18T14:50:57.090 INFO:tasks.workunit.client.1.smithi100.stdout:4/117: readlink d9/d17/l18 0 2024-03-18T14:50:57.090 INFO:tasks.workunit.client.1.smithi100.stdout:1/122: chown d8/d1e/l13 628227 1 2024-03-18T14:50:57.091 INFO:tasks.workunit.client.0.smithi012.stdout:8/670: rename d0/d15/d6e/d84/d54/d62/daf/d80 to d0/d15/d6e/d84/d54/d62/dd0 0 2024-03-18T14:50:57.091 INFO:tasks.workunit.client.0.smithi012.stdout:8/671: write d0/d15/d6e/d84/f30 [3575797,36131] 0 2024-03-18T14:50:57.091 INFO:tasks.workunit.client.0.smithi012.stdout:8/672: fdatasync d0/d5a/f8b 0 2024-03-18T14:50:57.093 INFO:tasks.workunit.client.1.smithi100.stdout:4/118: dread d9/f10 [0,4194304] 0 2024-03-18T14:50:57.093 INFO:tasks.workunit.client.1.smithi100.stdout:4/119: fdatasync d9/fe 0 2024-03-18T14:50:57.093 INFO:tasks.workunit.client.1.smithi100.stdout:4/120: chown d9/d17/f23 753997 1 2024-03-18T14:50:57.094 INFO:tasks.workunit.client.0.smithi012.stdout:1/434: link d8/db/d10/d16/d1c/f4c d8/db/d10/d16/d54/f82 0 2024-03-18T14:50:57.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/170: fsync dd/d2b/f40 0 2024-03-18T14:50:57.098 INFO:tasks.workunit.client.1.smithi100.stdout:0/171: chown dd/d2b/f3d 46592057 1 2024-03-18T14:50:57.098 INFO:tasks.workunit.client.1.smithi100.stdout:6/104: write f4 [4628220,63442] 0 2024-03-18T14:50:57.100 INFO:tasks.workunit.client.0.smithi012.stdout:6/461: unlink d17/d26/d36/l83 0 2024-03-18T14:50:57.101 INFO:tasks.workunit.client.1.smithi100.stdout:2/103: mkdir d3/d6/d29 0 2024-03-18T14:50:57.102 INFO:tasks.workunit.client.0.smithi012.stdout:4/446: symlink d6/d1a/d30/l93 0 2024-03-18T14:50:57.104 INFO:tasks.workunit.client.0.smithi012.stdout:5/474: creat d3/d1e/fa4 x:0 0 0 2024-03-18T14:50:57.104 INFO:tasks.workunit.client.0.smithi012.stdout:5/475: stat d3/d29 0 2024-03-18T14:50:57.104 INFO:tasks.workunit.client.1.smithi100.stdout:9/136: rmdir da/dd 39 2024-03-18T14:50:57.105 INFO:tasks.workunit.client.1.smithi100.stdout:9/137: chown da/dd/c2a 0 1 2024-03-18T14:50:57.105 INFO:tasks.workunit.client.1.smithi100.stdout:9/138: write da/dd/f10 [9231983,27669] 0 2024-03-18T14:50:57.107 INFO:tasks.workunit.client.1.smithi100.stdout:3/130: rename d3/dd/l23 to d3/dd/l26 0 2024-03-18T14:50:57.107 INFO:tasks.workunit.client.1.smithi100.stdout:7/118: dwrite d5/d14/f18 [0,4194304] 0 2024-03-18T14:50:57.108 INFO:tasks.workunit.client.1.smithi100.stdout:1/123: read f5 [3827274,70723] 0 2024-03-18T14:50:57.109 INFO:tasks.workunit.client.0.smithi012.stdout:0/449: dwrite d1/d16/d19/f47 [4194304,4194304] 0 2024-03-18T14:50:57.109 INFO:tasks.workunit.client.1.smithi100.stdout:7/119: dread d5/d14/f1e [0,4194304] 0 2024-03-18T14:50:57.110 INFO:tasks.workunit.client.0.smithi012.stdout:8/673: rename d0/d15/d6e/d84/d54/d62/daf/c8a to d0/d5a/d55/cd1 0 2024-03-18T14:50:57.110 INFO:tasks.workunit.client.0.smithi012.stdout:8/674: read d0/d5a/f60 [1005074,55805] 0 2024-03-18T14:50:57.112 INFO:tasks.workunit.client.0.smithi012.stdout:9/524: rmdir d3/d9/d19/d66/d21/d35/d48 39 2024-03-18T14:50:57.114 INFO:tasks.workunit.client.1.smithi100.stdout:4/121: mknod d9/c2a 0 2024-03-18T14:50:57.116 INFO:tasks.workunit.client.0.smithi012.stdout:4/447: mkdir d6/da/d89/d94 0 2024-03-18T14:50:57.118 INFO:tasks.workunit.client.1.smithi100.stdout:5/112: rename fb to f24 0 2024-03-18T14:50:57.120 INFO:tasks.workunit.client.0.smithi012.stdout:5/476: mkdir d3/d29/d36/d3d/d65/da5 0 2024-03-18T14:50:57.123 INFO:tasks.workunit.client.0.smithi012.stdout:9/525: dread d3/f6 [0,4194304] 0 2024-03-18T14:50:57.124 INFO:tasks.workunit.client.1.smithi100.stdout:5/113: dread f18 [0,4194304] 0 2024-03-18T14:50:57.124 INFO:tasks.workunit.client.0.smithi012.stdout:7/464: sync 2024-03-18T14:50:57.125 INFO:tasks.workunit.client.1.smithi100.stdout:2/104: symlink d3/d6/l2a 0 2024-03-18T14:50:57.128 INFO:tasks.workunit.client.1.smithi100.stdout:5/114: dread f1b [0,4194304] 0 2024-03-18T14:50:57.129 INFO:tasks.workunit.client.0.smithi012.stdout:4/448: dread d6/d1a/d30/d46/d6f/d7c/f81 [0,4194304] 0 2024-03-18T14:50:57.132 INFO:tasks.workunit.client.0.smithi012.stdout:1/435: rename d8/db/d10/c50 to d8/db/d10/d16/d1c/d2e/c83 0 2024-03-18T14:50:57.132 INFO:tasks.workunit.client.0.smithi012.stdout:1/436: chown d8/db/d10/d31/f4d 1905 1 2024-03-18T14:50:57.134 INFO:tasks.workunit.client.1.smithi100.stdout:3/131: rename d3/f9 to d3/dd/f27 0 2024-03-18T14:50:57.134 INFO:tasks.workunit.client.1.smithi100.stdout:3/132: creat d3/dd/f28 x:0 0 0 2024-03-18T14:50:57.134 INFO:tasks.workunit.client.1.smithi100.stdout:3/133: readlink d3/dd/l1d 0 2024-03-18T14:50:57.135 INFO:tasks.workunit.client.1.smithi100.stdout:3/134: write d3/dd/f27 [9270286,24646] 0 2024-03-18T14:50:57.135 INFO:tasks.workunit.client.0.smithi012.stdout:8/675: link d0/d5a/d19/dad/d33/d50/la7 d0/d15/d6e/d84/d54/d62/daf/ld2 0 2024-03-18T14:50:57.136 INFO:tasks.workunit.client.1.smithi100.stdout:9/139: write da/db/f13 [2313813,29175] 0 2024-03-18T14:50:57.136 INFO:tasks.workunit.client.1.smithi100.stdout:0/172: dwrite dd/d2b/f38 [0,4194304] 0 2024-03-18T14:50:57.137 INFO:tasks.workunit.client.1.smithi100.stdout:6/105: dwrite d6/d12/f1f [0,4194304] 0 2024-03-18T14:50:57.137 INFO:tasks.workunit.client.0.smithi012.stdout:0/450: dwrite d1/f9 [0,4194304] 0 2024-03-18T14:50:57.137 INFO:tasks.workunit.client.0.smithi012.stdout:5/477: rmdir d3/d29/d35 39 2024-03-18T14:50:57.137 INFO:tasks.workunit.client.0.smithi012.stdout:5/478: creat d3/d15/d5f/fa6 x:0 0 0 2024-03-18T14:50:57.138 INFO:tasks.workunit.client.1.smithi100.stdout:1/124: creat d8/d1e/d11/f21 x:0 0 0 2024-03-18T14:50:57.139 INFO:tasks.workunit.client.0.smithi012.stdout:7/465: dread dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:57.139 INFO:tasks.workunit.client.0.smithi012.stdout:7/466: readlink dd/d13/d23/d76/l35 0 2024-03-18T14:50:57.139 INFO:tasks.workunit.client.0.smithi012.stdout:7/467: write dd/d13/d23/d2f/d7a/f8b [34931,68241] 0 2024-03-18T14:50:57.140 INFO:tasks.workunit.client.1.smithi100.stdout:7/120: mkdir d5/d14/d16/d2e 0 2024-03-18T14:50:57.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/462: rename d17/d19/d27/d58/f8a to d17/d26/f8c 0 2024-03-18T14:50:57.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/463: creat d17/d19/d27/d2f/f8d x:0 0 0 2024-03-18T14:50:57.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/464: fdatasync d17/f35 0 2024-03-18T14:50:57.141 INFO:tasks.workunit.client.0.smithi012.stdout:6/465: write d17/d19/d27/d2f/f62 [939878,28499] 0 2024-03-18T14:50:57.142 INFO:tasks.workunit.client.0.smithi012.stdout:6/466: read d17/d19/d27/d2f/f62 [142743,87370] 0 2024-03-18T14:50:57.142 INFO:tasks.workunit.client.0.smithi012.stdout:6/467: chown d17/d19/d27/d2f/f62 1437 1 2024-03-18T14:50:57.142 INFO:tasks.workunit.client.1.smithi100.stdout:1/125: dread d8/de/f18 [0,4194304] 0 2024-03-18T14:50:57.143 INFO:tasks.workunit.client.0.smithi012.stdout:4/449: dread d6/d19/d39/f40 [0,4194304] 0 2024-03-18T14:50:57.144 INFO:tasks.workunit.client.0.smithi012.stdout:1/437: mkdir d8/db/d10/d31/d43/d5e/d65/d84 0 2024-03-18T14:50:57.144 INFO:tasks.workunit.client.0.smithi012.stdout:1/438: write d8/f7a [833201,47227] 0 2024-03-18T14:50:57.149 INFO:tasks.workunit.client.0.smithi012.stdout:5/479: dread d3/d1e/f28 [4194304,4194304] 0 2024-03-18T14:50:57.149 INFO:tasks.workunit.client.0.smithi012.stdout:8/676: mknod d0/d15/d6e/d84/d2a/cd3 0 2024-03-18T14:50:57.149 INFO:tasks.workunit.client.0.smithi012.stdout:8/677: fdatasync d0/d5a/d19/d57/daa/fb9 0 2024-03-18T14:50:57.150 INFO:tasks.workunit.client.0.smithi012.stdout:8/678: chown d0/d15/d6e/d84/d2a/fa1 436 1 2024-03-18T14:50:57.153 INFO:tasks.workunit.client.1.smithi100.stdout:8/119: rmdir d4/dc/dd/de 39 2024-03-18T14:50:57.153 INFO:tasks.workunit.client.1.smithi100.stdout:8/120: stat d4/dc/dd/de/d19/f1e 0 2024-03-18T14:50:57.154 INFO:tasks.workunit.client.0.smithi012.stdout:0/451: getdents d1/d16/d19 0 2024-03-18T14:50:57.155 INFO:tasks.workunit.client.0.smithi012.stdout:0/452: fdatasync d1/d16/d31/f58 0 2024-03-18T14:50:57.155 INFO:tasks.workunit.client.1.smithi100.stdout:4/122: stat d9/d13/c20 0 2024-03-18T14:50:57.155 INFO:tasks.workunit.client.1.smithi100.stdout:4/123: creat d9/f2b x:0 0 0 2024-03-18T14:50:57.156 INFO:tasks.workunit.client.1.smithi100.stdout:8/121: dread d4/ff [0,4194304] 0 2024-03-18T14:50:57.156 INFO:tasks.workunit.client.0.smithi012.stdout:8/679: dread d0/d5a/d19/dad/f3c [0,4194304] 0 2024-03-18T14:50:57.157 INFO:tasks.workunit.client.0.smithi012.stdout:3/428: sync 2024-03-18T14:50:57.157 INFO:tasks.workunit.client.0.smithi012.stdout:3/429: fsync dc/d3c/d4c/d50/f52 0 2024-03-18T14:50:57.157 INFO:tasks.workunit.client.1.smithi100.stdout:5/115: creat f25 x:0 0 0 2024-03-18T14:50:57.157 INFO:tasks.workunit.client.1.smithi100.stdout:5/116: rmdir - no directory 2024-03-18T14:50:57.159 INFO:tasks.workunit.client.0.smithi012.stdout:7/468: symlink dd/d13/d23/d76/d6f/la2 0 2024-03-18T14:50:57.160 INFO:tasks.workunit.client.0.smithi012.stdout:6/468: dwrite fe [0,4194304] 0 2024-03-18T14:50:57.160 INFO:tasks.workunit.client.0.smithi012.stdout:6/469: write f16 [3572993,88630] 0 2024-03-18T14:50:57.160 INFO:tasks.workunit.client.0.smithi012.stdout:6/470: chown d17/d38/d3f 192069347 1 2024-03-18T14:50:57.161 INFO:tasks.workunit.client.1.smithi100.stdout:2/105: rename d3/d13/f1d to d3/d1a/f2b 0 2024-03-18T14:50:57.161 INFO:tasks.workunit.client.1.smithi100.stdout:2/106: creat d3/da/f2c x:0 0 0 2024-03-18T14:50:57.163 INFO:tasks.workunit.client.1.smithi100.stdout:4/124: write d9/d17/f1d [1849073,35179] 0 2024-03-18T14:50:57.168 INFO:tasks.workunit.client.1.smithi100.stdout:3/135: truncate d3/d16/d13/d17/f1a 1466959 0 2024-03-18T14:50:57.168 INFO:tasks.workunit.client.0.smithi012.stdout:1/439: mknod d8/db/d10/d31/d43/d5e/d65/c85 0 2024-03-18T14:50:57.169 INFO:tasks.workunit.client.1.smithi100.stdout:9/140: mknod da/dd/d29/c2b 0 2024-03-18T14:50:57.169 INFO:tasks.workunit.client.1.smithi100.stdout:9/141: write f9 [9508620,15496] 0 2024-03-18T14:50:57.170 INFO:tasks.workunit.client.0.smithi012.stdout:5/480: symlink d3/d84/la7 0 2024-03-18T14:50:57.171 INFO:tasks.workunit.client.0.smithi012.stdout:7/469: read dd/d13/d23/d33/f51 [160396,11467] 0 2024-03-18T14:50:57.171 INFO:tasks.workunit.client.0.smithi012.stdout:7/470: fsync dd/d13/f72 0 2024-03-18T14:50:57.174 INFO:tasks.workunit.client.0.smithi012.stdout:0/453: creat d1/f9c x:0 0 0 2024-03-18T14:50:57.174 INFO:tasks.workunit.client.0.smithi012.stdout:0/454: chown d1/d16/d19/f2a 421352 1 2024-03-18T14:50:57.174 INFO:tasks.workunit.client.0.smithi012.stdout:0/455: write d1/d20/d23/d3b/d33/d64/f74 [743888,34928] 0 2024-03-18T14:50:57.174 INFO:tasks.workunit.client.0.smithi012.stdout:0/456: fdatasync d1/d16/fe 0 2024-03-18T14:50:57.176 INFO:tasks.workunit.client.1.smithi100.stdout:9/142: fdatasync f7 0 2024-03-18T14:50:57.177 INFO:tasks.workunit.client.1.smithi100.stdout:5/117: dwrite fa [0,4194304] 0 2024-03-18T14:50:57.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/680: creat d0/d5a/d19/d63/db0/fd4 x:0 0 0 2024-03-18T14:50:57.177 INFO:tasks.workunit.client.0.smithi012.stdout:8/681: readlink d0/lae 0 2024-03-18T14:50:57.181 INFO:tasks.workunit.client.1.smithi100.stdout:6/106: creat d6/d10/f22 x:0 0 0 2024-03-18T14:50:57.181 INFO:tasks.workunit.client.1.smithi100.stdout:6/107: fdatasync d6/fb 0 2024-03-18T14:50:57.183 INFO:tasks.workunit.client.0.smithi012.stdout:7/471: dread dd/d13/d23/d2f/d53/f5d [0,4194304] 0 2024-03-18T14:50:57.183 INFO:tasks.workunit.client.0.smithi012.stdout:7/472: chown dd/d13/d23/d76/d6f/c79 105756927 1 2024-03-18T14:50:57.184 INFO:tasks.workunit.client.1.smithi100.stdout:7/121: symlink d5/l2f 0 2024-03-18T14:50:57.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/481: write d3/d15/d5f/f70 [304489,51380] 0 2024-03-18T14:50:57.186 INFO:tasks.workunit.client.0.smithi012.stdout:5/482: write d3/d29/d36/d3d/d86/f94 [908834,70176] 0 2024-03-18T14:50:57.188 INFO:tasks.workunit.client.1.smithi100.stdout:1/126: mknod d8/dd/c22 0 2024-03-18T14:50:57.194 INFO:tasks.workunit.client.1.smithi100.stdout:8/122: mkdir d4/dc/d1f/d2b 0 2024-03-18T14:50:57.194 INFO:tasks.workunit.client.1.smithi100.stdout:8/123: dread - d4/dc/d1f/f20 zero size 2024-03-18T14:50:57.194 INFO:tasks.workunit.client.1.smithi100.stdout:8/124: fdatasync d4/f15 0 2024-03-18T14:50:57.194 INFO:tasks.workunit.client.1.smithi100.stdout:8/125: write f3 [2498809,23931] 0 2024-03-18T14:50:57.198 INFO:tasks.workunit.client.1.smithi100.stdout:4/125: symlink d9/d13/d25/l2c 0 2024-03-18T14:50:57.198 INFO:tasks.workunit.client.1.smithi100.stdout:4/126: readlink l5 0 2024-03-18T14:50:57.199 INFO:tasks.workunit.client.1.smithi100.stdout:4/127: write d9/d13/f1e [1112684,44178] 0 2024-03-18T14:50:57.199 INFO:tasks.workunit.client.1.smithi100.stdout:4/128: fsync f8 0 2024-03-18T14:50:57.199 INFO:tasks.workunit.client.0.smithi012.stdout:1/440: mknod d8/db/d10/d16/d6b/d7e/c86 0 2024-03-18T14:50:57.203 INFO:tasks.workunit.client.0.smithi012.stdout:0/457: mknod d1/d20/d23/d82/c9d 0 2024-03-18T14:50:57.204 INFO:tasks.workunit.client.1.smithi100.stdout:4/129: dread d9/f1b [0,4194304] 0 2024-03-18T14:50:57.204 INFO:tasks.workunit.client.1.smithi100.stdout:4/130: write d9/d17/f1d [417374,11941] 0 2024-03-18T14:50:57.204 INFO:tasks.workunit.client.1.smithi100.stdout:3/136: truncate d3/d16/fc 2932960 0 2024-03-18T14:50:57.205 INFO:tasks.workunit.client.0.smithi012.stdout:8/682: truncate d0/d15/d6e/d84/d2a/fa1 2919314 0 2024-03-18T14:50:57.205 INFO:tasks.workunit.client.0.smithi012.stdout:8/683: fdatasync d0/f10 0 2024-03-18T14:50:57.206 INFO:tasks.workunit.client.1.smithi100.stdout:0/173: link dd/d20/f3e dd/f41 0 2024-03-18T14:50:57.207 INFO:tasks.workunit.client.1.smithi100.stdout:0/174: read dd/f1c [1728352,72508] 0 2024-03-18T14:50:57.207 INFO:tasks.workunit.client.0.smithi012.stdout:3/430: rmdir dc/d30/d3a 39 2024-03-18T14:50:57.207 INFO:tasks.workunit.client.0.smithi012.stdout:3/431: fsync dc/d30/d3a/f84 0 2024-03-18T14:50:57.208 INFO:tasks.workunit.client.1.smithi100.stdout:5/118: dwrite f2 [0,4194304] 0 2024-03-18T14:50:57.208 INFO:tasks.workunit.client.1.smithi100.stdout:5/119: fsync f25 0 2024-03-18T14:50:57.209 INFO:tasks.workunit.client.1.smithi100.stdout:9/143: dwrite da/dd/f10 [4194304,4194304] 0 2024-03-18T14:50:57.209 INFO:tasks.workunit.client.1.smithi100.stdout:6/108: dwrite d6/d10/f13 [0,4194304] 0 2024-03-18T14:50:57.209 INFO:tasks.workunit.client.0.smithi012.stdout:7/473: mknod dd/d4f/ca3 0 2024-03-18T14:50:57.209 INFO:tasks.workunit.client.0.smithi012.stdout:7/474: chown dd/d13/d23/d2f/d7a/f8b 119 1 2024-03-18T14:50:57.209 INFO:tasks.workunit.client.0.smithi012.stdout:7/475: chown dd/d13/d23/d33/d37/l40 7836 1 2024-03-18T14:50:57.212 INFO:tasks.workunit.client.0.smithi012.stdout:5/483: unlink d3/d29/d35/c97 0 2024-03-18T14:50:57.217 INFO:tasks.workunit.client.0.smithi012.stdout:9/526: rename d3/d9/d19/d66/d21/d35/d71 to d3/d9/d19/d77/da8 0 2024-03-18T14:50:57.217 INFO:tasks.workunit.client.0.smithi012.stdout:9/527: getdents d3/d9/d19/d77/d52/d95/da3 0 2024-03-18T14:50:57.217 INFO:tasks.workunit.client.0.smithi012.stdout:0/458: dread d1/d20/d4e/d54/d67/f72 [0,4194304] 0 2024-03-18T14:50:57.220 INFO:tasks.workunit.client.0.smithi012.stdout:8/684: creat d0/d15/d6e/d84/d54/d62/dd0/fd5 x:0 0 0 2024-03-18T14:50:57.222 INFO:tasks.workunit.client.1.smithi100.stdout:6/109: write d6/d12/f1f [468643,99194] 0 2024-03-18T14:50:57.222 INFO:tasks.workunit.client.1.smithi100.stdout:6/110: truncate d6/d12/f1f 4914118 0 2024-03-18T14:50:57.223 INFO:tasks.workunit.client.1.smithi100.stdout:7/122: mkdir d5/d30 0 2024-03-18T14:50:57.225 INFO:tasks.workunit.client.1.smithi100.stdout:1/127: mknod d8/d1e/d11/c23 0 2024-03-18T14:50:57.226 INFO:tasks.workunit.client.0.smithi012.stdout:9/528: mkdir d3/d9/d19/d77/d52/d95/da9 0 2024-03-18T14:50:57.229 INFO:tasks.workunit.client.0.smithi012.stdout:6/471: rename d17/d19/d27/d2f/f8d to d17/d26/f8e 0 2024-03-18T14:50:57.229 INFO:tasks.workunit.client.0.smithi012.stdout:0/459: dread d1/d20/d23/f46 [0,4194304] 0 2024-03-18T14:50:57.231 INFO:tasks.workunit.client.1.smithi100.stdout:8/126: mkdir d4/dc/dd/de/d19/d2c 0 2024-03-18T14:50:57.232 INFO:tasks.workunit.client.1.smithi100.stdout:0/175: dwrite dd/d20/f3e [0,4194304] 0 2024-03-18T14:50:57.234 INFO:tasks.workunit.client.0.smithi012.stdout:9/529: creat d3/d9/d19/d66/faa x:0 0 0 2024-03-18T14:50:57.236 INFO:tasks.workunit.client.1.smithi100.stdout:4/131: symlink d9/l2d 0 2024-03-18T14:50:57.236 INFO:tasks.workunit.client.1.smithi100.stdout:4/132: read - d9/f2b zero size 2024-03-18T14:50:57.236 INFO:tasks.workunit.client.1.smithi100.stdout:4/133: fdatasync d9/f11 0 2024-03-18T14:50:57.236 INFO:tasks.workunit.client.1.smithi100.stdout:1/128: dread f7 [0,4194304] 0 2024-03-18T14:50:57.238 INFO:tasks.workunit.client.1.smithi100.stdout:3/137: creat d3/f29 x:0 0 0 2024-03-18T14:50:57.238 INFO:tasks.workunit.client.1.smithi100.stdout:3/138: dread - d3/dd/f28 zero size 2024-03-18T14:50:57.239 INFO:tasks.workunit.client.0.smithi012.stdout:6/472: mknod d17/c8f 0 2024-03-18T14:50:57.239 INFO:tasks.workunit.client.0.smithi012.stdout:5/484: dwrite d3/d29/d36/d3d/f5c [0,4194304] 0 2024-03-18T14:50:57.242 INFO:tasks.workunit.client.0.smithi012.stdout:9/530: dread d3/d9/d19/d66/d21/f46 [0,4194304] 0 2024-03-18T14:50:57.242 INFO:tasks.workunit.client.0.smithi012.stdout:9/531: readlink d3/d9/d19/d66/d21/d35/l3d 0 2024-03-18T14:50:57.242 INFO:tasks.workunit.client.0.smithi012.stdout:9/532: truncate d3/d9/fd 5406872 0 2024-03-18T14:50:57.242 INFO:tasks.workunit.client.0.smithi012.stdout:9/533: read - d3/d9/d19/d77/d52/f60 zero size 2024-03-18T14:50:57.244 INFO:tasks.workunit.client.0.smithi012.stdout:0/460: link d1/d20/d23/d3b/d33/l91 d1/d20/d23/l9e 0 2024-03-18T14:50:57.248 INFO:tasks.workunit.client.0.smithi012.stdout:3/432: rename dc/d30/d35/c2f to dc/c88 0 2024-03-18T14:50:57.249 INFO:tasks.workunit.client.0.smithi012.stdout:8/685: dwrite d0/d5a/f60 [0,4194304] 0 2024-03-18T14:50:57.250 INFO:tasks.workunit.client.1.smithi100.stdout:4/134: write f8 [1259485,90888] 0 2024-03-18T14:50:57.250 INFO:tasks.workunit.client.1.smithi100.stdout:4/135: readlink d9/lf 0 2024-03-18T14:50:57.251 INFO:tasks.workunit.client.1.smithi100.stdout:4/136: truncate d9/fb 258850 0 2024-03-18T14:50:57.251 INFO:tasks.workunit.client.1.smithi100.stdout:4/137: chown d9/d17 1 1 2024-03-18T14:50:57.254 INFO:tasks.workunit.client.0.smithi012.stdout:9/534: dread d3/d9/d19/d66/d69/d7f/f9b [0,4194304] 0 2024-03-18T14:50:57.254 INFO:tasks.workunit.client.0.smithi012.stdout:9/535: chown d3/d9/d19/d66/l4b 1395603587 1 2024-03-18T14:50:57.255 INFO:tasks.workunit.client.1.smithi100.stdout:5/120: creat f26 x:0 0 0 2024-03-18T14:50:57.255 INFO:tasks.workunit.client.1.smithi100.stdout:5/121: truncate f23 851409 0 2024-03-18T14:50:57.255 INFO:tasks.workunit.client.1.smithi100.stdout:0/176: dwrite dd/d20/f25 [0,4194304] 0 2024-03-18T14:50:57.258 INFO:tasks.workunit.client.0.smithi012.stdout:6/473: symlink d17/d38/d51/d63/l90 0 2024-03-18T14:50:57.258 INFO:tasks.workunit.client.0.smithi012.stdout:6/474: chown d17/d19/d27/d2f/l32 2755866 1 2024-03-18T14:50:57.261 INFO:tasks.workunit.client.0.smithi012.stdout:2/505: sync 2024-03-18T14:50:57.261 INFO:tasks.workunit.client.1.smithi100.stdout:3/139: dwrite d3/f6 [0,4194304] 0 2024-03-18T14:50:57.261 INFO:tasks.workunit.client.1.smithi100.stdout:3/140: creat d3/dd/f2a x:0 0 0 2024-03-18T14:50:57.261 INFO:tasks.workunit.client.1.smithi100.stdout:3/141: chown d3/dd/f24 25410 1 2024-03-18T14:50:57.261 INFO:tasks.workunit.client.1.smithi100.stdout:9/144: mkdir da/d2c 0 2024-03-18T14:50:57.266 INFO:tasks.workunit.client.1.smithi100.stdout:6/111: mknod d6/d7/d1b/c23 0 2024-03-18T14:50:57.269 INFO:tasks.workunit.client.0.smithi012.stdout:0/461: unlink d1/d20/d23/d3b/d33/l73 0 2024-03-18T14:50:57.269 INFO:tasks.workunit.client.0.smithi012.stdout:0/462: creat d1/d20/d4e/d54/d67/f9f x:0 0 0 2024-03-18T14:50:57.271 INFO:tasks.workunit.client.1.smithi100.stdout:7/123: symlink d5/d14/d16/d2e/l31 0 2024-03-18T14:50:57.271 INFO:tasks.workunit.client.1.smithi100.stdout:7/124: truncate d5/d14/d16/f1d 1534663 0 2024-03-18T14:50:57.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/476: rename dd/d13/d23/d76/d6b/l8f to dd/d13/d23/la4 0 2024-03-18T14:50:57.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/477: write dd/d13/f73 [4238500,115774] 0 2024-03-18T14:50:57.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/478: creat dd/d13/d23/d2f/d53/fa5 x:0 0 0 2024-03-18T14:50:57.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/479: write dd/d13/d23/d2f/d7a/f8b [764806,115801] 0 2024-03-18T14:50:57.275 INFO:tasks.workunit.client.1.smithi100.stdout:8/127: creat d4/dc/f2d x:0 0 0 2024-03-18T14:50:57.275 INFO:tasks.workunit.client.1.smithi100.stdout:8/128: write d4/f9 [2191176,21518] 0 2024-03-18T14:50:57.277 INFO:tasks.workunit.client.0.smithi012.stdout:6/475: mkdir d17/d19/d27/d58/d91 0 2024-03-18T14:50:57.283 INFO:tasks.workunit.client.0.smithi012.stdout:7/480: symlink dd/d13/d23/d33/d37/la6 0 2024-03-18T14:50:57.283 INFO:tasks.workunit.client.0.smithi012.stdout:7/481: chown dd/d13/d23/d2f/d53/c64 360301 1 2024-03-18T14:50:57.283 INFO:tasks.workunit.client.0.smithi012.stdout:5/485: dwrite d3/d29/f7a [0,4194304] 0 2024-03-18T14:50:57.283 INFO:tasks.workunit.client.0.smithi012.stdout:0/463: read d1/fa [7392614,98560] 0 2024-03-18T14:50:57.284 INFO:tasks.workunit.client.0.smithi012.stdout:6/476: truncate d17/d38/f5a 3211286 0 2024-03-18T14:50:57.286 INFO:tasks.workunit.client.1.smithi100.stdout:8/129: read d4/dc/dd/f11 [2304442,97469] 0 2024-03-18T14:50:57.287 INFO:tasks.workunit.client.0.smithi012.stdout:5/486: dread d3/d29/d36/d3d/f3e [0,4194304] 0 2024-03-18T14:50:57.288 INFO:tasks.workunit.client.1.smithi100.stdout:1/129: mknod d8/de/c24 0 2024-03-18T14:50:57.288 INFO:tasks.workunit.client.1.smithi100.stdout:1/130: creat d8/de/f25 x:0 0 0 2024-03-18T14:50:57.291 INFO:tasks.workunit.client.0.smithi012.stdout:6/477: rename d17/d38/c78 to d17/d19/d27/d6e/d84/c92 0 2024-03-18T14:50:57.291 INFO:tasks.workunit.client.0.smithi012.stdout:6/478: fsync d17/d38/d51/d72/f74 0 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.1.smithi100.stdout:4/138: getdents d9/d13/d25 0 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.1.smithi100.stdout:4/139: creat d9/d27/f2e x:0 0 0 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.1.smithi100.stdout:4/140: dread - d9/f2b zero size 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.0.smithi012.stdout:2/506: dwrite d0/d10/f16 [0,4194304] 0 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.0.smithi012.stdout:2/507: read d0/d3c/d40/f5a [537372,96059] 0 2024-03-18T14:50:57.292 INFO:tasks.workunit.client.0.smithi012.stdout:2/508: fdatasync d0/d3c/f41 0 2024-03-18T14:50:57.293 INFO:tasks.workunit.client.0.smithi012.stdout:3/433: dwrite dc/d3c/f6c [0,4194304] 0 2024-03-18T14:50:57.293 INFO:tasks.workunit.client.0.smithi012.stdout:8/686: dwrite d0/d5a/f8b [0,4194304] 0 2024-03-18T14:50:57.293 INFO:tasks.workunit.client.0.smithi012.stdout:9/536: dwrite d3/d9/d19/d66/f45 [0,4194304] 0 2024-03-18T14:50:57.294 INFO:tasks.workunit.client.0.smithi012.stdout:5/487: creat d3/d29/d36/d9b/d59/fa8 x:0 0 0 2024-03-18T14:50:57.294 INFO:tasks.workunit.client.0.smithi012.stdout:5/488: readlink d3/d29/d35/l81 0 2024-03-18T14:50:57.296 INFO:tasks.workunit.client.0.smithi012.stdout:0/464: getdents d1/d20/d7b 0 2024-03-18T14:50:57.303 INFO:tasks.workunit.client.0.smithi012.stdout:7/482: dwrite fa [4194304,4194304] 0 2024-03-18T14:50:57.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/112: rmdir d6/d7 39 2024-03-18T14:50:57.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/113: truncate d6/d7/d18/f1c 522691 0 2024-03-18T14:50:57.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/114: truncate d6/d10/f22 642811 0 2024-03-18T14:50:57.305 INFO:tasks.workunit.client.0.smithi012.stdout:9/537: dread d3/d9/d5c/f85 [0,4194304] 0 2024-03-18T14:50:57.309 INFO:tasks.workunit.client.1.smithi100.stdout:8/130: symlink d4/dc/dd/de/d21/d23/l2e 0 2024-03-18T14:50:57.309 INFO:tasks.workunit.client.1.smithi100.stdout:8/131: write d4/dc/dd/de/d21/d23/f27 [175159,10587] 0 2024-03-18T14:50:57.313 INFO:tasks.workunit.client.1.smithi100.stdout:1/131: truncate d8/d1e/d11/f1b 498026 0 2024-03-18T14:50:57.317 INFO:tasks.workunit.client.0.smithi012.stdout:7/483: read dd/d13/d23/d33/d48/f5c [651077,116197] 0 2024-03-18T14:50:57.319 INFO:tasks.workunit.client.0.smithi012.stdout:0/465: mknod d1/d20/d4e/d54/d67/ca0 0 2024-03-18T14:50:57.320 INFO:tasks.workunit.client.1.smithi100.stdout:4/141: rename d9/d13/f19 to d9/d13/f2f 0 2024-03-18T14:50:57.321 INFO:tasks.workunit.client.0.smithi012.stdout:2/509: dwrite d0/d10/d18/f27 [8388608,4194304] 0 2024-03-18T14:50:57.323 INFO:tasks.workunit.client.0.smithi012.stdout:3/434: unlink dc/l20 0 2024-03-18T14:50:57.326 INFO:tasks.workunit.client.0.smithi012.stdout:9/538: link d3/d9/d19/d66/d21/la5 d3/d9/d19/d77/d52/d99/lab 0 2024-03-18T14:50:57.327 INFO:tasks.workunit.client.0.smithi012.stdout:9/539: dread - d3/d9/d19/d66/d21/f8e zero size 2024-03-18T14:50:57.327 INFO:tasks.workunit.client.1.smithi100.stdout:2/107: sync 2024-03-18T14:50:57.329 INFO:tasks.workunit.client.0.smithi012.stdout:8/687: truncate d0/d5a/f60 1969005 0 2024-03-18T14:50:57.332 INFO:tasks.workunit.client.0.smithi012.stdout:7/484: symlink dd/d13/d23/d2f/d7a/la7 0 2024-03-18T14:50:57.333 INFO:tasks.workunit.client.0.smithi012.stdout:2/510: write d0/d3c/d40/d47/f9b [3940109,99295] 0 2024-03-18T14:50:57.333 INFO:tasks.workunit.client.0.smithi012.stdout:2/511: chown d0/l96 29 1 2024-03-18T14:50:57.335 INFO:tasks.workunit.client.0.smithi012.stdout:9/540: dread d3/d9/d19/d77/d52/f6d [0,4194304] 0 2024-03-18T14:50:57.336 INFO:tasks.workunit.client.0.smithi012.stdout:0/466: rename d1/d20/d23/d26/f53 to d1/d20/d7b/fa1 0 2024-03-18T14:50:57.336 INFO:tasks.workunit.client.0.smithi012.stdout:0/467: write d1/f9c [418389,57639] 0 2024-03-18T14:50:57.338 INFO:tasks.workunit.client.0.smithi012.stdout:6/479: dread d17/f2c [0,4194304] 0 2024-03-18T14:50:57.338 INFO:tasks.workunit.client.1.smithi100.stdout:5/122: symlink l27 0 2024-03-18T14:50:57.340 INFO:tasks.workunit.client.1.smithi100.stdout:0/177: rename dd/d20/f3b to dd/f42 0 2024-03-18T14:50:57.340 INFO:tasks.workunit.client.1.smithi100.stdout:0/178: fsync dd/f41 0 2024-03-18T14:50:57.341 INFO:tasks.workunit.client.0.smithi012.stdout:2/512: creat d0/d6b/f9c x:0 0 0 2024-03-18T14:50:57.341 INFO:tasks.workunit.client.0.smithi012.stdout:8/688: write d0/d5a/d19/dad/f2d [3107929,125471] 0 2024-03-18T14:50:57.343 INFO:tasks.workunit.client.1.smithi100.stdout:6/115: unlink d6/d10/f14 0 2024-03-18T14:50:57.343 INFO:tasks.workunit.client.1.smithi100.stdout:6/116: readlink l5 0 2024-03-18T14:50:57.344 INFO:tasks.workunit.client.0.smithi012.stdout:9/541: mkdir d3/d9/d19/d66/d69/d7f/d8c/dac 0 2024-03-18T14:50:57.344 INFO:tasks.workunit.client.0.smithi012.stdout:9/542: creat d3/d9/d5c/fad x:0 0 0 2024-03-18T14:50:57.345 INFO:tasks.workunit.client.0.smithi012.stdout:3/435: dwrite dc/d30/f81 [0,4194304] 0 2024-03-18T14:50:57.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/485: rename dd/d13/d26/f5e to dd/d13/d23/d33/fa8 0 2024-03-18T14:50:57.348 INFO:tasks.workunit.client.1.smithi100.stdout:7/125: rmdir d5/d14 39 2024-03-18T14:50:57.348 INFO:tasks.workunit.client.1.smithi100.stdout:7/126: creat d5/d22/d24/f32 x:0 0 0 2024-03-18T14:50:57.348 INFO:tasks.workunit.client.1.smithi100.stdout:7/127: readlink d5/l8 0 2024-03-18T14:50:57.352 INFO:tasks.workunit.client.0.smithi012.stdout:0/468: unlink d1/d20/d23/d3b/d33/f6d 0 2024-03-18T14:50:57.354 INFO:tasks.workunit.client.1.smithi100.stdout:8/132: link d4/f9 d4/dc/dd/de/d19/f2f 0 2024-03-18T14:50:57.358 INFO:tasks.workunit.client.0.smithi012.stdout:6/480: unlink d17/d26/f8e 0 2024-03-18T14:50:57.361 INFO:tasks.workunit.client.1.smithi100.stdout:6/117: dwrite d6/d10/f22 [0,4194304] 0 2024-03-18T14:50:57.361 INFO:tasks.workunit.client.1.smithi100.stdout:1/132: link l3 d8/de/l26 0 2024-03-18T14:50:57.362 INFO:tasks.workunit.client.0.smithi012.stdout:2/513: symlink d0/d10/d18/d33/d71/d90/l9d 0 2024-03-18T14:50:57.362 INFO:tasks.workunit.client.0.smithi012.stdout:2/514: truncate d0/fa 9218405 0 2024-03-18T14:50:57.363 INFO:tasks.workunit.client.1.smithi100.stdout:6/118: dread d6/f15 [0,4194304] 0 2024-03-18T14:50:57.363 INFO:tasks.workunit.client.1.smithi100.stdout:6/119: write f4 [5096241,110707] 0 2024-03-18T14:50:57.363 INFO:tasks.workunit.client.1.smithi100.stdout:6/120: write d6/fe [351079,120276] 0 2024-03-18T14:50:57.363 INFO:tasks.workunit.client.1.smithi100.stdout:6/121: truncate d6/d7/df/f1e 752886 0 2024-03-18T14:50:57.366 INFO:tasks.workunit.client.1.smithi100.stdout:2/108: link d3/da/de/d20/f24 d3/d1a/f2d 0 2024-03-18T14:50:57.368 INFO:tasks.workunit.client.0.smithi012.stdout:9/543: unlink d3/fa7 0 2024-03-18T14:50:57.372 INFO:tasks.workunit.client.1.smithi100.stdout:0/179: creat dd/f43 x:0 0 0 2024-03-18T14:50:57.372 INFO:tasks.workunit.client.1.smithi100.stdout:0/180: chown dd 429 1 2024-03-18T14:50:57.372 INFO:tasks.workunit.client.1.smithi100.stdout:0/181: fsync dd/d39/f3f 0 2024-03-18T14:50:57.372 INFO:tasks.workunit.client.0.smithi012.stdout:2/515: write d0/d10/f58 [2219205,76852] 0 2024-03-18T14:50:57.372 INFO:tasks.workunit.client.0.smithi012.stdout:2/516: stat d0/d10/d18/d31/d4c/d59/f8c 0 2024-03-18T14:50:57.376 INFO:tasks.workunit.client.0.smithi012.stdout:3/436: symlink dc/d30/d35/l89 0 2024-03-18T14:50:57.376 INFO:tasks.workunit.client.1.smithi100.stdout:1/133: write d8/dd/ff [332759,25793] 0 2024-03-18T14:50:57.377 INFO:tasks.workunit.client.0.smithi012.stdout:2/517: read d0/d10/d18/f3a [698226,779] 0 2024-03-18T14:50:57.377 INFO:tasks.workunit.client.0.smithi012.stdout:2/518: write d0/d3c/f46 [435143,26544] 0 2024-03-18T14:50:57.378 INFO:tasks.workunit.client.0.smithi012.stdout:2/519: fdatasync d0/d10/f13 0 2024-03-18T14:50:57.378 INFO:tasks.workunit.client.0.smithi012.stdout:2/520: chown d0/d10/d18/d31/l78 8 1 2024-03-18T14:50:57.378 INFO:tasks.workunit.client.1.smithi100.stdout:7/128: mknod d5/c33 0 2024-03-18T14:50:57.379 INFO:tasks.workunit.client.0.smithi012.stdout:7/486: creat dd/d4f/fa9 x:0 0 0 2024-03-18T14:50:57.379 INFO:tasks.workunit.client.0.smithi012.stdout:0/469: dwrite d1/d20/d4e/d54/d67/f9f [0,4194304] 0 2024-03-18T14:50:57.381 INFO:tasks.workunit.client.0.smithi012.stdout:6/481: symlink d17/d19/d27/d6e/d84/l93 0 2024-03-18T14:50:57.381 INFO:tasks.workunit.client.0.smithi012.stdout:6/482: chown d17/d38/d3f/f55 62096229 1 2024-03-18T14:50:57.383 INFO:tasks.workunit.client.0.smithi012.stdout:9/544: creat d3/d9/d19/d66/d69/d7f/d8c/dac/fae x:0 0 0 2024-03-18T14:50:57.384 INFO:tasks.workunit.client.1.smithi100.stdout:6/122: mknod d6/d7/d1b/c24 0 2024-03-18T14:50:57.385 INFO:tasks.workunit.client.0.smithi012.stdout:3/437: mknod dc/d3c/d4c/d57/c8a 0 2024-03-18T14:50:57.386 INFO:tasks.workunit.client.1.smithi100.stdout:4/142: getdents d9/d27 0 2024-03-18T14:50:57.387 INFO:tasks.workunit.client.1.smithi100.stdout:8/133: dwrite d4/dc/d1f/f20 [0,4194304] 0 2024-03-18T14:50:57.388 INFO:tasks.workunit.client.0.smithi012.stdout:2/521: symlink d0/d10/d18/d33/d77/l9e 0 2024-03-18T14:50:57.390 INFO:tasks.workunit.client.1.smithi100.stdout:2/109: unlink d3/d6/f26 0 2024-03-18T14:50:57.390 INFO:tasks.workunit.client.1.smithi100.stdout:2/110: dread d3/f1b [0,4194304] 0 2024-03-18T14:50:57.393 INFO:tasks.workunit.client.1.smithi100.stdout:5/123: rename l21 to l28 0 2024-03-18T14:50:57.393 INFO:tasks.workunit.client.1.smithi100.stdout:5/124: chown f10 124 1 2024-03-18T14:50:57.393 INFO:tasks.workunit.client.1.smithi100.stdout:5/125: rmdir - no directory 2024-03-18T14:50:57.394 INFO:tasks.workunit.client.1.smithi100.stdout:5/126: fsync f18 0 2024-03-18T14:50:57.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/689: dwrite d0/d5a/f60 [0,4194304] 0 2024-03-18T14:50:57.401 INFO:tasks.workunit.client.0.smithi012.stdout:0/470: link d1/d20/d23/l69 d1/d2d/la2 0 2024-03-18T14:50:57.402 INFO:tasks.workunit.client.1.smithi100.stdout:1/134: mkdir d8/d1e/d11/d27 0 2024-03-18T14:50:57.402 INFO:tasks.workunit.client.1.smithi100.stdout:1/135: chown d8/dd/c1c 1033898421 1 2024-03-18T14:50:57.403 INFO:tasks.workunit.client.1.smithi100.stdout:1/136: truncate d8/de/f25 388410 0 2024-03-18T14:50:57.403 INFO:tasks.workunit.client.1.smithi100.stdout:1/137: chown l3 214876 1 2024-03-18T14:50:57.405 INFO:tasks.workunit.client.0.smithi012.stdout:6/483: creat d17/d18/f94 x:0 0 0 2024-03-18T14:50:57.410 INFO:tasks.workunit.client.1.smithi100.stdout:7/129: mkdir d5/d34 0 2024-03-18T14:50:57.410 INFO:tasks.workunit.client.1.smithi100.stdout:7/130: write d5/d14/f1e [471876,93948] 0 2024-03-18T14:50:57.410 INFO:tasks.workunit.client.1.smithi100.stdout:7/131: readlink d5/d14/d16/l26 0 2024-03-18T14:50:57.410 INFO:tasks.workunit.client.1.smithi100.stdout:7/132: dread - d5/d22/d24/f2d zero size 2024-03-18T14:50:57.413 INFO:tasks.workunit.client.1.smithi100.stdout:4/143: rmdir d9/d17 39 2024-03-18T14:50:57.415 INFO:tasks.workunit.client.0.smithi012.stdout:9/545: mknod d3/d9/d19/d77/caf 0 2024-03-18T14:50:57.415 INFO:tasks.workunit.client.0.smithi012.stdout:9/546: fsync d3/d9/d19/d77/da8/f76 0 2024-03-18T14:50:57.415 INFO:tasks.workunit.client.0.smithi012.stdout:9/547: fdatasync d3/d9/d19/f4d 0 2024-03-18T14:50:57.416 INFO:tasks.workunit.client.0.smithi012.stdout:9/548: fdatasync d3/d9/d5c/f85 0 2024-03-18T14:50:57.416 INFO:tasks.workunit.client.0.smithi012.stdout:9/549: fdatasync d3/d9/d19/d66/faa 0 2024-03-18T14:50:57.425 INFO:tasks.workunit.client.1.smithi100.stdout:2/111: mknod d3/da/c2e 0 2024-03-18T14:50:57.425 INFO:tasks.workunit.client.1.smithi100.stdout:2/112: truncate d3/da/f2c 724687 0 2024-03-18T14:50:57.425 INFO:tasks.workunit.client.1.smithi100.stdout:2/113: chown d3/da/de/d20/f23 154326690 1 2024-03-18T14:50:57.427 INFO:tasks.workunit.client.1.smithi100.stdout:7/133: dread d5/d14/f18 [0,4194304] 0 2024-03-18T14:50:57.427 INFO:tasks.workunit.client.1.smithi100.stdout:7/134: truncate d5/d14/d16/f1d 1694998 0 2024-03-18T14:50:57.429 INFO:tasks.workunit.client.1.smithi100.stdout:1/138: creat d8/d1e/f28 x:0 0 0 2024-03-18T14:50:57.429 INFO:tasks.workunit.client.0.smithi012.stdout:3/438: write dc/d30/f81 [4448840,121255] 0 2024-03-18T14:50:57.430 INFO:tasks.workunit.client.1.smithi100.stdout:8/134: dread d4/dc/dd/de/f17 [0,4194304] 0 2024-03-18T14:50:57.431 INFO:tasks.workunit.client.0.smithi012.stdout:7/487: dwrite dd/d13/d23/d2f/f32 [0,4194304] 0 2024-03-18T14:50:57.432 INFO:tasks.workunit.client.1.smithi100.stdout:5/127: mkdir d29 0 2024-03-18T14:50:57.433 INFO:tasks.workunit.client.0.smithi012.stdout:0/471: dwrite d1/d16/d19/f68 [4194304,4194304] 0 2024-03-18T14:50:57.433 INFO:tasks.workunit.client.0.smithi012.stdout:0/472: dread - d1/d20/d23/d3b/d33/d64/f96 zero size 2024-03-18T14:50:57.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/522: dwrite d0/d10/d18/d33/d71/f87 [0,4194304] 0 2024-03-18T14:50:57.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/523: chown d0/d3c 253 1 2024-03-18T14:50:57.433 INFO:tasks.workunit.client.0.smithi012.stdout:2/524: stat d0/d10/d18/d4f/f6f 0 2024-03-18T14:50:57.436 INFO:tasks.workunit.client.1.smithi100.stdout:0/182: dwrite fa [0,4194304] 0 2024-03-18T14:50:57.439 INFO:tasks.workunit.client.0.smithi012.stdout:9/550: read d3/d9/d19/f3a [2300355,130370] 0 2024-03-18T14:50:57.439 INFO:tasks.workunit.client.1.smithi100.stdout:1/139: creat d8/f29 x:0 0 0 2024-03-18T14:50:57.439 INFO:tasks.workunit.client.1.smithi100.stdout:1/140: stat d8/d1e/f28 0 2024-03-18T14:50:57.443 INFO:tasks.workunit.client.1.smithi100.stdout:5/128: creat d29/f2a x:0 0 0 2024-03-18T14:50:57.446 INFO:tasks.workunit.client.1.smithi100.stdout:5/129: dread f23 [0,4194304] 0 2024-03-18T14:50:57.446 INFO:tasks.workunit.client.1.smithi100.stdout:5/130: truncate f10 2390678 0 2024-03-18T14:50:57.448 INFO:tasks.workunit.client.1.smithi100.stdout:6/123: dwrite d6/d12/f1f [0,4194304] 0 2024-03-18T14:50:57.449 INFO:tasks.workunit.client.0.smithi012.stdout:8/690: getdents d0/d5a/d19/dad/d33 0 2024-03-18T14:50:57.450 INFO:tasks.workunit.client.1.smithi100.stdout:2/114: dwrite d3/da/f25 [0,4194304] 0 2024-03-18T14:50:57.452 INFO:tasks.workunit.client.0.smithi012.stdout:6/484: mknod d17/d19/d27/d2f/c95 0 2024-03-18T14:50:57.455 INFO:tasks.workunit.client.0.smithi012.stdout:3/439: truncate dc/f32 48908 0 2024-03-18T14:50:57.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/485: dread d17/d38/f5a [0,4194304] 0 2024-03-18T14:50:57.460 INFO:tasks.workunit.client.0.smithi012.stdout:0/473: dwrite d1/d20/d23/d26/f90 [0,4194304] 0 2024-03-18T14:50:57.462 INFO:tasks.workunit.client.1.smithi100.stdout:8/135: dwrite d4/fa [0,4194304] 0 2024-03-18T14:50:57.462 INFO:tasks.workunit.client.1.smithi100.stdout:8/136: chown d4/dc 69487376 1 2024-03-18T14:50:57.464 INFO:tasks.workunit.client.1.smithi100.stdout:2/115: dread d3/d1a/f21 [0,4194304] 0 2024-03-18T14:50:57.466 INFO:tasks.workunit.client.0.smithi012.stdout:7/488: rename dd/d13/d23/d33/d37/c44 to dd/d13/d23/d33/d48/caa 0 2024-03-18T14:50:57.477 INFO:tasks.workunit.client.0.smithi012.stdout:2/525: rmdir d0/d6c 39 2024-03-18T14:50:57.479 INFO:tasks.workunit.client.0.smithi012.stdout:8/691: link d0/d15/d6e/d84/d54/d62/dd0/f92 d0/d15/d6e/d84/d54/fd6 0 2024-03-18T14:50:57.483 INFO:tasks.workunit.client.1.smithi100.stdout:0/183: mkdir dd/d20/d44 0 2024-03-18T14:50:57.483 INFO:tasks.workunit.client.1.smithi100.stdout:0/184: truncate dd/d20/f28 713896 0 2024-03-18T14:50:57.488 INFO:tasks.workunit.client.1.smithi100.stdout:8/137: dwrite d4/ff [4194304,4194304] 0 2024-03-18T14:50:57.488 INFO:tasks.workunit.client.1.smithi100.stdout:2/116: dwrite f0 [0,4194304] 0 2024-03-18T14:50:57.488 INFO:tasks.workunit.client.1.smithi100.stdout:6/124: mknod d6/d7/d1b/c25 0 2024-03-18T14:50:57.499 INFO:tasks.workunit.client.1.smithi100.stdout:0/185: creat dd/d39/f45 x:0 0 0 2024-03-18T14:50:57.501 INFO:tasks.workunit.client.1.smithi100.stdout:1/141: getdents d8/dd 0 2024-03-18T14:50:57.501 INFO:tasks.workunit.client.1.smithi100.stdout:1/142: write d8/f29 [568450,6699] 0 2024-03-18T14:50:57.501 INFO:tasks.workunit.client.1.smithi100.stdout:1/143: chown d8/de/f18 339458 1 2024-03-18T14:50:57.501 INFO:tasks.workunit.client.1.smithi100.stdout:1/144: dread d8/de/f25 [0,4194304] 0 2024-03-18T14:50:57.502 INFO:tasks.workunit.client.0.smithi012.stdout:8/692: dwrite d0/d5a/d19/d57/f68 [0,4194304] 0 2024-03-18T14:50:57.502 INFO:tasks.workunit.client.0.smithi012.stdout:8/693: read - d0/d5a/d5e/f82 zero size 2024-03-18T14:50:57.502 INFO:tasks.workunit.client.0.smithi012.stdout:7/489: mkdir dd/d13/d23/d2f/d7a/dab 0 2024-03-18T14:50:57.503 INFO:tasks.workunit.client.1.smithi100.stdout:9/145: sync 2024-03-18T14:50:57.503 INFO:tasks.workunit.client.1.smithi100.stdout:3/142: sync 2024-03-18T14:50:57.503 INFO:tasks.workunit.client.1.smithi100.stdout:3/143: dread - d3/f29 zero size 2024-03-18T14:50:57.503 INFO:tasks.workunit.client.1.smithi100.stdout:3/144: fsync d3/dd/f11 0 2024-03-18T14:50:57.505 INFO:tasks.workunit.client.0.smithi012.stdout:6/486: rename d17/d33/f4e to d17/d26/f96 0 2024-03-18T14:50:57.506 INFO:tasks.workunit.client.1.smithi100.stdout:8/138: rmdir d4/dc/dd/de/d19 39 2024-03-18T14:50:57.506 INFO:tasks.workunit.client.1.smithi100.stdout:8/139: readlink d4/dc/dd/l1a 0 2024-03-18T14:50:57.507 INFO:tasks.workunit.client.1.smithi100.stdout:6/125: dwrite d6/d7/df/f1e [0,4194304] 0 2024-03-18T14:50:57.509 INFO:tasks.workunit.client.0.smithi012.stdout:2/526: mkdir d0/d6b/d9f 0 2024-03-18T14:50:57.509 INFO:tasks.workunit.client.1.smithi100.stdout:3/145: read d3/d16/fc [2912146,13858] 0 2024-03-18T14:50:57.509 INFO:tasks.workunit.client.1.smithi100.stdout:3/146: creat d3/d16/d13/d17/f2b x:0 0 0 2024-03-18T14:50:57.509 INFO:tasks.workunit.client.1.smithi100.stdout:0/186: mknod dd/d31/c46 0 2024-03-18T14:50:57.510 INFO:tasks.workunit.client.1.smithi100.stdout:0/187: stat dd/d2b/f38 0 2024-03-18T14:50:57.511 INFO:tasks.workunit.client.0.smithi012.stdout:9/551: getdents d3/d9/d19/d66/d69/d7f/d8c/dac 0 2024-03-18T14:50:57.511 INFO:tasks.workunit.client.0.smithi012.stdout:9/552: write d3/d9/d19/f6a [1447408,67752] 0 2024-03-18T14:50:57.511 INFO:tasks.workunit.client.0.smithi012.stdout:9/553: stat d3/d9/d19/d66/d6e/l98 0 2024-03-18T14:50:57.514 INFO:tasks.workunit.client.1.smithi100.stdout:1/145: mkdir d8/d2a 0 2024-03-18T14:50:57.516 INFO:tasks.workunit.client.0.smithi012.stdout:3/440: getdents dc/d3c/d4c/d57 0 2024-03-18T14:50:57.519 INFO:tasks.workunit.client.1.smithi100.stdout:9/146: rmdir da/dd/d29 39 2024-03-18T14:50:57.519 INFO:tasks.workunit.client.0.smithi012.stdout:5/489: sync 2024-03-18T14:50:57.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/474: getdents d1/d20 0 2024-03-18T14:50:57.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/475: read d1/d16/d19/f2a [3478599,30004] 0 2024-03-18T14:50:57.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/476: dread - d1/d20/d4e/d54/d67/f9a zero size 2024-03-18T14:50:57.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/477: write d1/d2d/f62 [1000801,30532] 0 2024-03-18T14:50:57.522 INFO:tasks.workunit.client.1.smithi100.stdout:8/140: dread d4/fa [0,4194304] 0 2024-03-18T14:50:57.522 INFO:tasks.workunit.client.1.smithi100.stdout:8/141: dread - d4/f15 zero size 2024-03-18T14:50:57.522 INFO:tasks.workunit.client.0.smithi012.stdout:7/490: symlink dd/d4f/lac 0 2024-03-18T14:50:57.522 INFO:tasks.workunit.client.0.smithi012.stdout:7/491: chown dd/d13/d23/d2f/d7a 7 1 2024-03-18T14:50:57.523 INFO:tasks.workunit.client.0.smithi012.stdout:7/492: fsync dd/d13/d23/d2f/f32 0 2024-03-18T14:50:57.525 INFO:tasks.workunit.client.0.smithi012.stdout:6/487: dwrite d17/f1a [0,4194304] 0 2024-03-18T14:50:57.526 INFO:tasks.workunit.client.1.smithi100.stdout:6/126: unlink d6/d7/d18/c1d 0 2024-03-18T14:50:57.528 INFO:tasks.workunit.client.0.smithi012.stdout:2/527: mkdir d0/d10/d18/d33/d77/da0 0 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:1/441: sync 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:4/450: sync 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:1/442: fdatasync d8/d1e/d27/d44/f4f 0 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:1/443: write d8/d1e/f7c [734836,11930] 0 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:1/444: fsync d8/db/d10/f34 0 2024-03-18T14:50:57.530 INFO:tasks.workunit.client.0.smithi012.stdout:1/445: chown d8/d1e/f63 201 1 2024-03-18T14:50:57.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/554: dwrite d3/d9/d5c/f85 [0,4194304] 0 2024-03-18T14:50:57.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/555: stat d3/d20/f31 0 2024-03-18T14:50:57.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/556: chown d3/d9/d19/d66/d21/d35/d48/fa4 805 1 2024-03-18T14:50:57.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/557: chown d3/d9/d19/d66/d21/d35/d48/fa4 674116837 1 2024-03-18T14:50:57.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/558: dread - d3/d9/d19/d66/d69/d7f/f92 zero size 2024-03-18T14:50:57.533 INFO:tasks.workunit.client.1.smithi100.stdout:3/147: mknod d3/d16/c2c 0 2024-03-18T14:50:57.533 INFO:tasks.workunit.client.1.smithi100.stdout:3/148: truncate d3/dd/f27 9387291 0 2024-03-18T14:50:57.539 INFO:tasks.workunit.client.1.smithi100.stdout:8/142: dread d4/f6 [0,4194304] 0 2024-03-18T14:50:57.540 INFO:tasks.workunit.client.0.smithi012.stdout:5/490: rename d3/d15/c79 to d3/d15/ca9 0 2024-03-18T14:50:57.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/491: readlink d3/d29/d36/l8e 0 2024-03-18T14:50:57.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/492: chown d3/d29/d36/d3d/d73/d6b/d8f 613 1 2024-03-18T14:50:57.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/493: write d3/f9e [770088,97863] 0 2024-03-18T14:50:57.542 INFO:tasks.workunit.client.1.smithi100.stdout:9/147: mknod da/d2c/c2d 0 2024-03-18T14:50:57.542 INFO:tasks.workunit.client.1.smithi100.stdout:9/148: chown da/d2c/c2d 50 1 2024-03-18T14:50:57.542 INFO:tasks.workunit.client.0.smithi012.stdout:7/493: dwrite fa [4194304,4194304] 0 2024-03-18T14:50:57.544 INFO:tasks.workunit.client.1.smithi100.stdout:6/127: getdents d6/d7 0 2024-03-18T14:50:57.546 INFO:tasks.workunit.client.0.smithi012.stdout:8/694: rename d0/d15/d6e to d0/d5a/d5e/dd7 0 2024-03-18T14:50:57.546 INFO:tasks.workunit.client.0.smithi012.stdout:8/695: chown d0/d5a/d5e/dd7/d84/l49 5165 1 2024-03-18T14:50:57.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/188: getdents dd 0 2024-03-18T14:50:57.552 INFO:tasks.workunit.client.0.smithi012.stdout:6/488: mkdir d17/d26/d36/d4d/d82/d97 0 2024-03-18T14:50:57.552 INFO:tasks.workunit.client.0.smithi012.stdout:6/489: chown d17/d26/f8c 48957412 1 2024-03-18T14:50:57.552 INFO:tasks.workunit.client.0.smithi012.stdout:6/490: write d17/d19/d27/d2f/f62 [876020,100634] 0 2024-03-18T14:50:57.552 INFO:tasks.workunit.client.0.smithi012.stdout:6/491: stat d17/d38 0 2024-03-18T14:50:57.552 INFO:tasks.workunit.client.0.smithi012.stdout:6/492: write d17/d19/f3c [106246,510] 0 2024-03-18T14:50:57.553 INFO:tasks.workunit.client.1.smithi100.stdout:8/143: mknod d4/dc/dd/de/d19/c30 0 2024-03-18T14:50:57.553 INFO:tasks.workunit.client.1.smithi100.stdout:8/144: fsync d4/dc/dd/de/d21/d23/f27 0 2024-03-18T14:50:57.553 INFO:tasks.workunit.client.1.smithi100.stdout:6/128: dread d6/d7/df/f1e [0,4194304] 0 2024-03-18T14:50:57.555 INFO:tasks.workunit.client.0.smithi012.stdout:1/446: link d8/db/d10/d31/d45/l5d d8/db/d10/d56/l87 0 2024-03-18T14:50:57.555 INFO:tasks.workunit.client.0.smithi012.stdout:1/447: fdatasync d8/f1a 0 2024-03-18T14:50:57.556 INFO:tasks.workunit.client.1.smithi100.stdout:9/149: rmdir da/db 39 2024-03-18T14:50:57.558 INFO:tasks.workunit.client.0.smithi012.stdout:6/493: dread fb [0,4194304] 0 2024-03-18T14:50:57.559 INFO:tasks.workunit.client.0.smithi012.stdout:7/494: dread dd/d13/d23/d60/f6c [0,4194304] 0 2024-03-18T14:50:57.559 INFO:tasks.workunit.client.0.smithi012.stdout:4/451: dwrite d6/f5c [0,4194304] 0 2024-03-18T14:50:57.561 INFO:tasks.workunit.client.0.smithi012.stdout:9/559: unlink d3/d9/d19/f3a 0 2024-03-18T14:50:57.561 INFO:tasks.workunit.client.0.smithi012.stdout:9/560: chown d3/d9/d19/d66/d69/d7f 765489 1 2024-03-18T14:50:57.561 INFO:tasks.workunit.client.0.smithi012.stdout:9/561: readlink d3/l14 0 2024-03-18T14:50:57.563 INFO:tasks.workunit.client.1.smithi100.stdout:3/149: dwrite d3/fa [0,4194304] 0 2024-03-18T14:50:57.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/441: rename dc/d3c/d4c/d5f/c60 to dc/d3c/d4c/d57/c8b 0 2024-03-18T14:50:57.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/442: chown dc/f11 0 1 2024-03-18T14:50:57.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/443: write dc/d30/d35/f70 [934870,44303] 0 2024-03-18T14:50:57.569 INFO:tasks.workunit.client.0.smithi012.stdout:8/696: link d0/l34 d0/d5a/d19/dad/ld8 0 2024-03-18T14:50:57.569 INFO:tasks.workunit.client.0.smithi012.stdout:8/697: write d0/d15/f94 [837579,9053] 0 2024-03-18T14:50:57.569 INFO:tasks.workunit.client.0.smithi012.stdout:8/698: stat d0/d5a/d5e/dd7/d84/d2a/d3e/f5f 0 2024-03-18T14:50:57.570 INFO:tasks.workunit.client.0.smithi012.stdout:1/448: symlink d8/d1e/l88 0 2024-03-18T14:50:57.570 INFO:tasks.workunit.client.0.smithi012.stdout:1/449: write d8/db/d10/f34 [597555,118071] 0 2024-03-18T14:50:57.572 INFO:tasks.workunit.client.0.smithi012.stdout:6/494: mkdir d17/d19/d98 0 2024-03-18T14:50:57.572 INFO:tasks.workunit.client.0.smithi012.stdout:6/495: stat d17/d19/f6d 0 2024-03-18T14:50:57.573 INFO:tasks.workunit.client.1.smithi100.stdout:9/150: creat da/dd/d29/f2e x:0 0 0 2024-03-18T14:50:57.574 INFO:tasks.workunit.client.0.smithi012.stdout:9/562: mknod d3/d9/d19/d77/cb0 0 2024-03-18T14:50:57.576 INFO:tasks.workunit.client.0.smithi012.stdout:5/494: link d3/d29/d35/f48 d3/d1e/d33/faa 0 2024-03-18T14:50:57.577 INFO:tasks.workunit.client.0.smithi012.stdout:0/478: rename d1/d20/d23/d3b/d33/l78 to d1/d20/d23/d3b/d33/d51/la3 0 2024-03-18T14:50:57.578 INFO:tasks.workunit.client.1.smithi100.stdout:6/129: dwrite d6/fb [0,4194304] 0 2024-03-18T14:50:57.579 INFO:tasks.workunit.client.0.smithi012.stdout:3/444: truncate dc/d30/d35/f65 4647504 0 2024-03-18T14:50:57.580 INFO:tasks.workunit.client.0.smithi012.stdout:1/450: creat d8/db/d10/d16/d54/f89 x:0 0 0 2024-03-18T14:50:57.581 INFO:tasks.workunit.client.0.smithi012.stdout:4/452: dwrite d6/d27/f67 [0,4194304] 0 2024-03-18T14:50:57.581 INFO:tasks.workunit.client.1.smithi100.stdout:9/151: mkdir da/d2c/d2f 0 2024-03-18T14:50:57.583 INFO:tasks.workunit.client.0.smithi012.stdout:6/496: mknod d17/d19/d27/d6e/c99 0 2024-03-18T14:50:57.583 INFO:tasks.workunit.client.0.smithi012.stdout:6/497: chown d17/d33/f88 4324166 1 2024-03-18T14:50:57.584 INFO:tasks.workunit.client.0.smithi012.stdout:6/498: read d17/f35 [891885,106376] 0 2024-03-18T14:50:57.585 INFO:tasks.workunit.client.1.smithi100.stdout:3/150: symlink d3/l2d 0 2024-03-18T14:50:57.586 INFO:tasks.workunit.client.1.smithi100.stdout:0/189: dwrite f3 [0,4194304] 0 2024-03-18T14:50:57.587 INFO:tasks.workunit.client.0.smithi012.stdout:9/563: mknod d3/d9/d19/d77/d52/d65/cb1 0 2024-03-18T14:50:57.587 INFO:tasks.workunit.client.0.smithi012.stdout:9/564: truncate d3/d9/d19/d66/d69/d7f/d8c/d8b/f9f 1433954 0 2024-03-18T14:50:57.590 INFO:tasks.workunit.client.0.smithi012.stdout:0/479: write d1/f5f [1750316,66238] 0 2024-03-18T14:50:57.590 INFO:tasks.workunit.client.0.smithi012.stdout:0/480: creat d1/d20/d23/d26/fa4 x:0 0 0 2024-03-18T14:50:57.590 INFO:tasks.workunit.client.0.smithi012.stdout:0/481: creat d1/d20/d23/d6f/fa5 x:0 0 0 2024-03-18T14:50:57.591 INFO:tasks.workunit.client.0.smithi012.stdout:2/528: rename d0/d10/d18/d31/d4c/d59/f8a to d0/d10/d18/d33/d71/fa1 0 2024-03-18T14:50:57.591 INFO:tasks.workunit.client.0.smithi012.stdout:1/451: dread d8/f5a [0,4194304] 0 2024-03-18T14:50:57.592 INFO:tasks.workunit.client.1.smithi100.stdout:6/130: write d6/d12/f1f [1469948,30345] 0 2024-03-18T14:50:57.592 INFO:tasks.workunit.client.1.smithi100.stdout:8/145: rmdir d4 39 2024-03-18T14:50:57.593 INFO:tasks.workunit.client.1.smithi100.stdout:9/152: creat da/d2c/d2f/f30 x:0 0 0 2024-03-18T14:50:57.593 INFO:tasks.workunit.client.1.smithi100.stdout:9/153: stat f2 0 2024-03-18T14:50:57.595 INFO:tasks.workunit.client.1.smithi100.stdout:8/146: dread d4/ff [0,4194304] 0 2024-03-18T14:50:57.595 INFO:tasks.workunit.client.1.smithi100.stdout:3/151: symlink d3/d16/d13/d1f/l2e 0 2024-03-18T14:50:57.596 INFO:tasks.workunit.client.0.smithi012.stdout:9/565: dread d3/d9/d19/d66/f45 [0,4194304] 0 2024-03-18T14:50:57.596 INFO:tasks.workunit.client.0.smithi012.stdout:5/495: dwrite d3/d29/d36/d9b/d59/fa8 [0,4194304] 0 2024-03-18T14:50:57.597 INFO:tasks.workunit.client.0.smithi012.stdout:3/445: symlink dc/d30/d35/l8c 0 2024-03-18T14:50:57.601 INFO:tasks.workunit.client.1.smithi100.stdout:0/190: symlink dd/d31/l47 0 2024-03-18T14:50:57.602 INFO:tasks.workunit.client.0.smithi012.stdout:5/496: read d3/fc [592309,37573] 0 2024-03-18T14:50:57.604 INFO:tasks.workunit.client.1.smithi100.stdout:9/154: dread da/db/f13 [0,4194304] 0 2024-03-18T14:50:57.604 INFO:tasks.workunit.client.1.smithi100.stdout:9/155: getdents da/d25 0 2024-03-18T14:50:57.608 INFO:tasks.workunit.client.0.smithi012.stdout:9/566: write d3/d9/fd [2583687,120465] 0 2024-03-18T14:50:57.608 INFO:tasks.workunit.client.1.smithi100.stdout:6/131: rename d6/d7/d18 to d6/d7/df/d26 0 2024-03-18T14:50:57.609 INFO:tasks.workunit.client.0.smithi012.stdout:4/453: mkdir d6/da/d89/d95 0 2024-03-18T14:50:57.609 INFO:tasks.workunit.client.0.smithi012.stdout:4/454: fdatasync d6/f80 0 2024-03-18T14:50:57.609 INFO:tasks.workunit.client.0.smithi012.stdout:4/455: chown d6/f5c 165282730 1 2024-03-18T14:50:57.611 INFO:tasks.workunit.client.1.smithi100.stdout:8/147: mkdir d4/dc/d31 0 2024-03-18T14:50:57.613 INFO:tasks.workunit.client.1.smithi100.stdout:8/148: write d4/dc/dd/de/d19/f1e [161025,48746] 0 2024-03-18T14:50:57.613 INFO:tasks.workunit.client.0.smithi012.stdout:5/497: dread d3/d29/f7a [0,4194304] 0 2024-03-18T14:50:57.613 INFO:tasks.workunit.client.0.smithi012.stdout:6/499: mknod d17/d38/d51/d72/c9a 0 2024-03-18T14:50:57.614 INFO:tasks.workunit.client.0.smithi012.stdout:4/456: dread d6/f21 [0,4194304] 0 2024-03-18T14:50:57.615 INFO:tasks.workunit.client.1.smithi100.stdout:3/152: mknod d3/d16/c2f 0 2024-03-18T14:50:57.615 INFO:tasks.workunit.client.1.smithi100.stdout:4/144: sync 2024-03-18T14:50:57.615 INFO:tasks.workunit.client.1.smithi100.stdout:4/145: dread - d9/d13/f2f zero size 2024-03-18T14:50:57.619 INFO:tasks.workunit.client.0.smithi012.stdout:3/446: dwrite dc/d3c/d4c/d50/f5d [0,4194304] 0 2024-03-18T14:50:57.619 INFO:tasks.workunit.client.0.smithi012.stdout:3/447: truncate dc/d3c/d59/f5a 1375739 0 2024-03-18T14:50:57.620 INFO:tasks.workunit.client.1.smithi100.stdout:0/191: symlink dd/d39/l48 0 2024-03-18T14:50:57.625 INFO:tasks.workunit.client.0.smithi012.stdout:6/500: dread d17/d19/d27/d2f/f62 [0,4194304] 0 2024-03-18T14:50:57.633 INFO:tasks.workunit.client.1.smithi100.stdout:6/132: dwrite d6/d7/f8 [0,4194304] 0 2024-03-18T14:50:57.633 INFO:tasks.workunit.client.1.smithi100.stdout:8/149: dwrite d4/dc/dd/de/d21/d23/f27 [4194304,4194304] 0 2024-03-18T14:50:57.633 INFO:tasks.workunit.client.1.smithi100.stdout:4/146: symlink d9/d13/l30 0 2024-03-18T14:50:57.633 INFO:tasks.workunit.client.1.smithi100.stdout:4/147: readlink d9/lf 0 2024-03-18T14:50:57.635 INFO:tasks.workunit.client.1.smithi100.stdout:0/192: creat dd/f49 x:0 0 0 2024-03-18T14:50:57.635 INFO:tasks.workunit.client.1.smithi100.stdout:0/193: write dd/f26 [1024412,37165] 0 2024-03-18T14:50:57.639 INFO:tasks.workunit.client.1.smithi100.stdout:4/148: dread d9/fe [0,4194304] 0 2024-03-18T14:50:57.639 INFO:tasks.workunit.client.1.smithi100.stdout:3/153: link d3/d16/d13/d17/f2b d3/f30 0 2024-03-18T14:50:57.640 INFO:tasks.workunit.client.1.smithi100.stdout:3/154: write d3/dd/f2a [977108,4852] 0 2024-03-18T14:50:57.640 INFO:tasks.workunit.client.1.smithi100.stdout:3/155: fdatasync d3/d16/fc 0 2024-03-18T14:50:57.640 INFO:tasks.workunit.client.1.smithi100.stdout:3/156: write d3/d16/d13/d17/f2b [142250,66496] 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.1.smithi100.stdout:9/156: dwrite da/db/f1a [0,4194304] 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.1.smithi100.stdout:9/157: write da/db/f13 [1950304,79040] 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.1.smithi100.stdout:8/150: write d4/dc/d1f/f20 [358580,93185] 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.1.smithi100.stdout:9/158: creat da/d2c/d2f/f31 x:0 0 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.1.smithi100.stdout:4/149: write d9/d13/f1e [175557,130528] 0 2024-03-18T14:50:57.641 INFO:tasks.workunit.client.0.smithi012.stdout:9/567: creat d3/d9/d19/d66/d69/fb2 x:0 0 0 2024-03-18T14:50:57.644 INFO:tasks.workunit.client.0.smithi012.stdout:5/498: mknod d3/d29/cab 0 2024-03-18T14:50:57.644 INFO:tasks.workunit.client.0.smithi012.stdout:5/499: chown d3/d29/d36/d3d/d73/f7d 3 1 2024-03-18T14:50:57.650 INFO:tasks.workunit.client.1.smithi100.stdout:4/150: dread d9/f1a [0,4194304] 0 2024-03-18T14:50:57.651 INFO:tasks.workunit.client.1.smithi100.stdout:6/133: rename d6/d7/d1b to d6/d10/d27 0 2024-03-18T14:50:57.651 INFO:tasks.workunit.client.1.smithi100.stdout:6/134: chown d6/d10/d27/c24 336 1 2024-03-18T14:50:57.651 INFO:tasks.workunit.client.1.smithi100.stdout:6/135: fsync d6/fb 0 2024-03-18T14:50:57.654 INFO:tasks.workunit.client.0.smithi012.stdout:3/448: unlink dc/c4e 0 2024-03-18T14:50:57.658 INFO:tasks.workunit.client.0.smithi012.stdout:6/501: link d17/d26/c31 d17/c9b 0 2024-03-18T14:50:57.658 INFO:tasks.workunit.client.0.smithi012.stdout:4/457: getdents d6/da/d89 0 2024-03-18T14:50:57.658 INFO:tasks.workunit.client.0.smithi012.stdout:4/458: chown d6/c74 36857229 1 2024-03-18T14:50:57.660 INFO:tasks.workunit.client.1.smithi100.stdout:0/194: symlink dd/l4a 0 2024-03-18T14:50:57.662 INFO:tasks.workunit.client.1.smithi100.stdout:3/157: write d3/dd/f11 [3259234,74740] 0 2024-03-18T14:50:57.663 INFO:tasks.workunit.client.0.smithi012.stdout:0/482: truncate d1/d20/d7b/f30 97286 0 2024-03-18T14:50:57.665 INFO:tasks.workunit.client.0.smithi012.stdout:2/529: rename d0/d10/d18/d31/d4c to d0/da2 0 2024-03-18T14:50:57.666 INFO:tasks.workunit.client.0.smithi012.stdout:2/530: readlink d0/d6c/l76 0 2024-03-18T14:50:57.667 INFO:tasks.workunit.client.0.smithi012.stdout:9/568: mknod d3/d9/d19/d77/cb3 0 2024-03-18T14:50:57.667 INFO:tasks.workunit.client.0.smithi012.stdout:9/569: chown d3/d9/d5c/f8f 4660770 1 2024-03-18T14:50:57.668 INFO:tasks.workunit.client.1.smithi100.stdout:4/151: link d9/d13/l30 d9/l31 0 2024-03-18T14:50:57.668 INFO:tasks.workunit.client.1.smithi100.stdout:4/152: readlink d9/la 0 2024-03-18T14:50:57.668 INFO:tasks.workunit.client.1.smithi100.stdout:4/153: chown d9/f11 244 1 2024-03-18T14:50:57.669 INFO:tasks.workunit.client.0.smithi012.stdout:5/500: mknod d3/d1e/cac 0 2024-03-18T14:50:57.670 INFO:tasks.workunit.client.1.smithi100.stdout:6/136: rmdir d6/d7/df 39 2024-03-18T14:50:57.671 INFO:tasks.workunit.client.0.smithi012.stdout:6/502: symlink d17/d26/l9c 0 2024-03-18T14:50:57.672 INFO:tasks.workunit.client.1.smithi100.stdout:0/195: symlink dd/d20/d44/l4b 0 2024-03-18T14:50:57.673 INFO:tasks.workunit.client.1.smithi100.stdout:0/196: write dd/d39/f3f [2400399,128865] 0 2024-03-18T14:50:57.673 INFO:tasks.workunit.client.0.smithi012.stdout:4/459: mkdir d6/d19/d6c/d96 0 2024-03-18T14:50:57.676 INFO:tasks.workunit.client.0.smithi012.stdout:0/483: rmdir d1/d20/d4e 39 2024-03-18T14:50:57.678 INFO:tasks.workunit.client.1.smithi100.stdout:3/158: mkdir d3/d16/d13/d1f/d31 0 2024-03-18T14:50:57.679 INFO:tasks.workunit.client.1.smithi100.stdout:6/137: write d6/d7/f8 [536965,47890] 0 2024-03-18T14:50:57.679 INFO:tasks.workunit.client.0.smithi012.stdout:6/503: read fe [4380326,27167] 0 2024-03-18T14:50:57.680 INFO:tasks.workunit.client.0.smithi012.stdout:6/504: chown d17/d26/c29 305735 1 2024-03-18T14:50:57.681 INFO:tasks.workunit.client.0.smithi012.stdout:7/495: rename dd/d13/d23/d2f/l43 to dd/d4f/lad 0 2024-03-18T14:50:57.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/699: rename d0 to d0/d5a/d19/d57/d9e/dd9 22 2024-03-18T14:50:57.681 INFO:tasks.workunit.client.0.smithi012.stdout:7/496: fsync dd/d13/d23/d76/f68 0 2024-03-18T14:50:57.683 INFO:tasks.workunit.client.0.smithi012.stdout:2/531: mknod d0/d6b/ca3 0 2024-03-18T14:50:57.685 INFO:tasks.workunit.client.1.smithi100.stdout:3/159: creat d3/d16/d13/d1f/d31/f32 x:0 0 0 2024-03-18T14:50:57.685 INFO:tasks.workunit.client.1.smithi100.stdout:3/160: creat d3/d16/d13/d17/f33 x:0 0 0 2024-03-18T14:50:57.687 INFO:tasks.workunit.client.0.smithi012.stdout:9/570: creat d3/d9/d19/d77/da8/fb4 x:0 0 0 2024-03-18T14:50:57.687 INFO:tasks.workunit.client.0.smithi012.stdout:9/571: dread - d3/d9/d19/d77/f56 zero size 2024-03-18T14:50:57.688 INFO:tasks.workunit.client.0.smithi012.stdout:5/501: mkdir d3/dad 0 2024-03-18T14:50:57.688 INFO:tasks.workunit.client.0.smithi012.stdout:5/502: chown d3/d29/d36/l72 6015667 1 2024-03-18T14:50:57.688 INFO:tasks.workunit.client.0.smithi012.stdout:5/503: write d3/d29/f61 [695321,24235] 0 2024-03-18T14:50:57.689 INFO:tasks.workunit.client.0.smithi012.stdout:4/460: mkdir d6/da/d89/d97 0 2024-03-18T14:50:57.692 INFO:tasks.workunit.client.0.smithi012.stdout:0/484: symlink d1/la6 0 2024-03-18T14:50:57.694 INFO:tasks.workunit.client.1.smithi100.stdout:3/161: symlink d3/l34 0 2024-03-18T14:50:57.694 INFO:tasks.workunit.client.0.smithi012.stdout:8/700: write d0/d5a/d5e/dd7/d84/f6b [375553,70126] 0 2024-03-18T14:50:57.694 INFO:tasks.workunit.client.0.smithi012.stdout:8/701: chown d0/d15/f43 0 1 2024-03-18T14:50:57.695 INFO:tasks.workunit.client.0.smithi012.stdout:1/452: rename d8/db/d10/d16/d1c/f2b to d8/db/d10/d56/f8a 0 2024-03-18T14:50:57.696 INFO:tasks.workunit.client.0.smithi012.stdout:7/497: mknod dd/d13/d23/cae 0 2024-03-18T14:50:57.697 INFO:tasks.workunit.client.1.smithi100.stdout:8/151: dwrite d4/f6 [0,4194304] 0 2024-03-18T14:50:57.698 INFO:tasks.workunit.client.0.smithi012.stdout:9/572: rmdir d3/d9/d19/d77 39 2024-03-18T14:50:57.698 INFO:tasks.workunit.client.1.smithi100.stdout:3/162: creat d3/dd/d21/f35 x:0 0 0 2024-03-18T14:50:57.698 INFO:tasks.workunit.client.1.smithi100.stdout:3/163: write d3/ff [770266,9586] 0 2024-03-18T14:50:57.699 INFO:tasks.workunit.client.0.smithi012.stdout:5/504: truncate d3/d1e/f28 4204951 0 2024-03-18T14:50:57.700 INFO:tasks.workunit.client.0.smithi012.stdout:0/485: mkdir d1/d20/d23/d7f/da7 0 2024-03-18T14:50:57.700 INFO:tasks.workunit.client.0.smithi012.stdout:0/486: write d1/d16/d19/f56 [659001,18826] 0 2024-03-18T14:50:57.702 INFO:tasks.workunit.client.0.smithi012.stdout:6/505: getdents d17/d26/d36/d4d 0 2024-03-18T14:50:57.705 INFO:tasks.workunit.client.0.smithi012.stdout:1/453: dread d8/db/d10/f73 [0,4194304] 0 2024-03-18T14:50:57.706 INFO:tasks.workunit.client.0.smithi012.stdout:3/449: rename dc/d30/d35/c38 to dc/d3c/d4c/d50/d64/c8d 0 2024-03-18T14:50:57.708 INFO:tasks.workunit.client.0.smithi012.stdout:0/487: link d1/d16/d6e/f9b d1/d20/d23/d3b/d33/d51/fa8 0 2024-03-18T14:50:57.709 INFO:tasks.workunit.client.1.smithi100.stdout:0/197: fdatasync dd/d39/f3f 0 2024-03-18T14:50:57.709 INFO:tasks.workunit.client.1.smithi100.stdout:9/159: dwrite da/f1f [0,4194304] 0 2024-03-18T14:50:57.710 INFO:tasks.workunit.client.0.smithi012.stdout:6/506: symlink d17/d19/d27/d6e/d84/l9d 0 2024-03-18T14:50:57.710 INFO:tasks.workunit.client.0.smithi012.stdout:6/507: write d17/d26/d36/d4d/f81 [917525,24927] 0 2024-03-18T14:50:57.710 INFO:tasks.workunit.client.0.smithi012.stdout:4/461: read d6/d1a/d30/d46/d6f/d7c/f87 [1619161,1667] 0 2024-03-18T14:50:57.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/450: unlink dc/d3c/d4c/d57/f7c 0 2024-03-18T14:50:57.714 INFO:tasks.workunit.client.0.smithi012.stdout:2/532: rename d0/d3c/d40/f4b to d0/da2/d59/fa4 0 2024-03-18T14:50:57.716 INFO:tasks.workunit.client.0.smithi012.stdout:3/451: dread dc/d3c/d4c/d50/d54/f58 [0,4194304] 0 2024-03-18T14:50:57.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/198: rename dd/d20/f3e to dd/d2b/f4c 0 2024-03-18T14:50:57.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/199: stat dd/f1d 0 2024-03-18T14:50:57.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/200: creat dd/d20/f4d x:0 0 0 2024-03-18T14:50:57.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/201: truncate dd/f1c 4200750 0 2024-03-18T14:50:57.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/202: creat dd/d20/f4e x:0 0 0 2024-03-18T14:50:57.718 INFO:tasks.workunit.client.0.smithi012.stdout:0/488: truncate d1/d20/d7b/ff 3365163 0 2024-03-18T14:50:57.719 INFO:tasks.workunit.client.0.smithi012.stdout:5/505: dread d3/f1f [4194304,4194304] 0 2024-03-18T14:50:57.719 INFO:tasks.workunit.client.0.smithi012.stdout:5/506: chown d3/d29/d36/d9b/f27 99095 1 2024-03-18T14:50:57.719 INFO:tasks.workunit.client.0.smithi012.stdout:8/702: dread d0/f10 [0,4194304] 0 2024-03-18T14:50:57.720 INFO:tasks.workunit.client.0.smithi012.stdout:8/703: write d0/d5a/d19/dad/d33/d50/fa5 [545293,55383] 0 2024-03-18T14:50:57.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/533: mknod d0/d10/d18/d57/d61/ca5 0 2024-03-18T14:50:57.721 INFO:tasks.workunit.client.1.smithi100.stdout:4/154: dwrite d9/f1b [0,4194304] 0 2024-03-18T14:50:57.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/138: dwrite d6/d7/df/f1e [4194304,4194304] 0 2024-03-18T14:50:57.725 INFO:tasks.workunit.client.0.smithi012.stdout:3/452: mknod dc/d3c/d72/c8e 0 2024-03-18T14:50:57.727 INFO:tasks.workunit.client.1.smithi100.stdout:6/139: dread d6/d7/df/d26/f1c [0,4194304] 0 2024-03-18T14:50:57.728 INFO:tasks.workunit.client.1.smithi100.stdout:8/152: dwrite d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:57.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/155: write d9/f1a [888730,122028] 0 2024-03-18T14:50:57.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/156: creat d9/f32 x:0 0 0 2024-03-18T14:50:57.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/157: write f0 [5151373,111132] 0 2024-03-18T14:50:57.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/158: dread - d9/d27/f29 zero size 2024-03-18T14:50:57.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/159: stat d9/d13 0 2024-03-18T14:50:57.730 INFO:tasks.workunit.client.0.smithi012.stdout:7/498: dread dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:57.730 INFO:tasks.workunit.client.0.smithi012.stdout:7/499: chown dd/d13/f72 17930 1 2024-03-18T14:50:57.734 INFO:tasks.workunit.client.0.smithi012.stdout:0/489: mknod d1/d20/d23/ca9 0 2024-03-18T14:50:57.738 INFO:tasks.workunit.client.1.smithi100.stdout:3/164: rmdir d3/d16/d13/d1f/d31 39 2024-03-18T14:50:57.738 INFO:tasks.workunit.client.1.smithi100.stdout:8/153: write d4/ff [4215200,125038] 0 2024-03-18T14:50:57.740 INFO:tasks.workunit.client.0.smithi012.stdout:5/507: creat d3/d29/d36/d3d/d65/da5/fae x:0 0 0 2024-03-18T14:50:57.747 INFO:tasks.workunit.client.0.smithi012.stdout:0/490: dread d1/f4f [0,4194304] 0 2024-03-18T14:50:57.747 INFO:tasks.workunit.client.0.smithi012.stdout:0/491: creat d1/faa x:0 0 0 2024-03-18T14:50:57.750 INFO:tasks.workunit.client.0.smithi012.stdout:1/454: dwrite d8/db/f2d [0,4194304] 0 2024-03-18T14:50:57.751 INFO:tasks.workunit.client.0.smithi012.stdout:1/455: creat d8/db/d10/f8b x:0 0 0 2024-03-18T14:50:57.751 INFO:tasks.workunit.client.0.smithi012.stdout:1/456: write d8/db/d10/d31/d45/f6d [63999,49065] 0 2024-03-18T14:50:57.751 INFO:tasks.workunit.client.0.smithi012.stdout:1/457: write d8/db/d10/d31/f68 [662021,102767] 0 2024-03-18T14:50:57.751 INFO:tasks.workunit.client.1.smithi100.stdout:9/160: dwrite da/d2c/d2f/f31 [0,4194304] 0 2024-03-18T14:50:57.752 INFO:tasks.workunit.client.0.smithi012.stdout:4/462: dwrite d6/d1a/d30/d46/d6f/d7c/f87 [0,4194304] 0 2024-03-18T14:50:57.752 INFO:tasks.workunit.client.0.smithi012.stdout:8/704: link d0/d5a/f1d d0/d47/d59/d67/fda 0 2024-03-18T14:50:57.755 INFO:tasks.workunit.client.0.smithi012.stdout:2/534: rmdir d0/da2 39 2024-03-18T14:50:57.755 INFO:tasks.workunit.client.0.smithi012.stdout:2/535: chown d0/d10/d18/d57/d61/ca5 1 1 2024-03-18T14:50:57.755 INFO:tasks.workunit.client.0.smithi012.stdout:2/536: fdatasync d0/d10/d18/d33/f62 0 2024-03-18T14:50:57.755 INFO:tasks.workunit.client.0.smithi012.stdout:2/537: write d0/d10/d18/d4f/f6f [1071597,73523] 0 2024-03-18T14:50:57.757 INFO:tasks.workunit.client.1.smithi100.stdout:0/203: symlink dd/d2b/l4f 0 2024-03-18T14:50:57.757 INFO:tasks.workunit.client.1.smithi100.stdout:0/204: write dd/f49 [137535,34639] 0 2024-03-18T14:50:57.761 INFO:tasks.workunit.client.0.smithi012.stdout:1/458: dread f5 [0,4194304] 0 2024-03-18T14:50:57.762 INFO:tasks.workunit.client.0.smithi012.stdout:1/459: write d8/db/d10/f73 [4770616,80690] 0 2024-03-18T14:50:57.762 INFO:tasks.workunit.client.0.smithi012.stdout:1/460: write d8/d1e/f63 [224187,98807] 0 2024-03-18T14:50:57.765 INFO:tasks.workunit.client.1.smithi100.stdout:4/160: dread f0 [0,4194304] 0 2024-03-18T14:50:57.765 INFO:tasks.workunit.client.1.smithi100.stdout:4/161: creat d9/d13/f33 x:0 0 0 2024-03-18T14:50:57.766 INFO:tasks.workunit.client.0.smithi012.stdout:1/461: dread d8/db/d10/d31/f68 [0,4194304] 0 2024-03-18T14:50:57.775 INFO:tasks.workunit.client.0.smithi012.stdout:7/500: mknod dd/d13/d23/d60/caf 0 2024-03-18T14:50:57.775 INFO:tasks.workunit.client.0.smithi012.stdout:7/501: write dd/d13/f72 [280766,81123] 0 2024-03-18T14:50:57.775 INFO:tasks.workunit.client.0.smithi012.stdout:7/502: fsync dd/f45 0 2024-03-18T14:50:57.775 INFO:tasks.workunit.client.0.smithi012.stdout:7/503: write dd/f45 [324018,124836] 0 2024-03-18T14:50:57.775 INFO:tasks.workunit.client.0.smithi012.stdout:7/504: creat dd/d13/d23/d76/d6b/fb0 x:0 0 0 2024-03-18T14:50:57.782 INFO:tasks.workunit.client.1.smithi100.stdout:3/165: creat d3/d16/f36 x:0 0 0 2024-03-18T14:50:57.782 INFO:tasks.workunit.client.1.smithi100.stdout:3/166: fsync d3/dd/f27 0 2024-03-18T14:50:57.783 INFO:tasks.workunit.client.0.smithi012.stdout:0/492: creat d1/d16/d19/d8d/fab x:0 0 0 2024-03-18T14:50:57.788 INFO:tasks.workunit.client.1.smithi100.stdout:9/161: rmdir da/dd 39 2024-03-18T14:50:57.790 INFO:tasks.workunit.client.1.smithi100.stdout:7/135: sync 2024-03-18T14:50:57.791 INFO:tasks.workunit.client.0.smithi012.stdout:2/538: dwrite d0/d10/d18/f73 [0,4194304] 0 2024-03-18T14:50:57.792 INFO:tasks.workunit.client.0.smithi012.stdout:8/705: rmdir d0/d5a/d19/dad/d33/d50 39 2024-03-18T14:50:57.794 INFO:tasks.workunit.client.1.smithi100.stdout:0/205: symlink dd/d39/l50 0 2024-03-18T14:50:57.794 INFO:tasks.workunit.client.1.smithi100.stdout:4/162: dwrite d9/f1a [0,4194304] 0 2024-03-18T14:50:57.797 INFO:tasks.workunit.client.0.smithi012.stdout:1/462: getdents d8/db/d10/d31/d43/d5e/d65 0 2024-03-18T14:50:57.798 INFO:tasks.workunit.client.0.smithi012.stdout:0/493: write d1/d16/d19/f37 [1425682,9491] 0 2024-03-18T14:50:57.798 INFO:tasks.workunit.client.0.smithi012.stdout:0/494: write d1/d20/d23/d3b/d33/f8b [596830,57985] 0 2024-03-18T14:50:57.799 INFO:tasks.workunit.client.1.smithi100.stdout:2/117: sync 2024-03-18T14:50:57.799 INFO:tasks.workunit.client.1.smithi100.stdout:1/146: sync 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/131: sync 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:1/147: read - d8/d1e/f28 zero size 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/132: creat d29/f2b x:0 0 0 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/133: creat d29/f2c x:0 0 0 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/134: fdatasync f25 0 2024-03-18T14:50:57.800 INFO:tasks.workunit.client.1.smithi100.stdout:5/135: fdatasync f1b 0 2024-03-18T14:50:57.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/148: dread f7 [4194304,4194304] 0 2024-03-18T14:50:57.806 INFO:tasks.workunit.client.0.smithi012.stdout:7/505: symlink dd/d13/d23/d33/lb1 0 2024-03-18T14:50:57.806 INFO:tasks.workunit.client.1.smithi100.stdout:6/140: rmdir d6/d12 39 2024-03-18T14:50:57.808 INFO:tasks.workunit.client.0.smithi012.stdout:5/508: getdents d3/d84 0 2024-03-18T14:50:57.810 INFO:tasks.workunit.client.0.smithi012.stdout:4/463: mkdir d6/d1a/d98 0 2024-03-18T14:50:57.811 INFO:tasks.workunit.client.0.smithi012.stdout:4/464: chown d6/f8c 118468 1 2024-03-18T14:50:57.812 INFO:tasks.workunit.client.0.smithi012.stdout:4/465: read d6/d19/f51 [11856,11562] 0 2024-03-18T14:50:57.813 INFO:tasks.workunit.client.0.smithi012.stdout:3/453: write dc/f12 [3103696,31606] 0 2024-03-18T14:50:57.813 INFO:tasks.workunit.client.1.smithi100.stdout:7/136: rename d5/d22/d24/f32 to d5/f35 0 2024-03-18T14:50:57.817 INFO:tasks.workunit.client.1.smithi100.stdout:0/206: link dd/c2a dd/d20/c51 0 2024-03-18T14:50:57.820 INFO:tasks.workunit.client.0.smithi012.stdout:2/539: mknod d0/d10/d18/d31/ca6 0 2024-03-18T14:50:57.820 INFO:tasks.workunit.client.0.smithi012.stdout:2/540: fdatasync d0/d10/d18/d33/d71/f87 0 2024-03-18T14:50:57.820 INFO:tasks.workunit.client.0.smithi012.stdout:2/541: chown d0/d3c/f41 12047 1 2024-03-18T14:50:57.821 INFO:tasks.workunit.client.1.smithi100.stdout:3/167: dwrite d3/d16/fc [0,4194304] 0 2024-03-18T14:50:57.821 INFO:tasks.workunit.client.1.smithi100.stdout:2/118: dwrite d3/da/de/d20/f24 [0,4194304] 0 2024-03-18T14:50:57.821 INFO:tasks.workunit.client.1.smithi100.stdout:2/119: dread d3/fc [0,4194304] 0 2024-03-18T14:50:57.824 INFO:tasks.workunit.client.0.smithi012.stdout:8/706: symlink d0/d5a/d19/dad/d33/d50/ldb 0 2024-03-18T14:50:57.828 INFO:tasks.workunit.client.1.smithi100.stdout:4/163: dwrite d9/fe [0,4194304] 0 2024-03-18T14:50:57.828 INFO:tasks.workunit.client.0.smithi012.stdout:1/463: rename d8/f5a to d8/db/d10/d31/d81/f8c 0 2024-03-18T14:50:57.830 INFO:tasks.workunit.client.1.smithi100.stdout:5/136: mkdir d29/d2d 0 2024-03-18T14:50:57.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/149: mknod d8/d1e/c2b 0 2024-03-18T14:50:57.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/150: write d8/d1e/f28 [50696,78873] 0 2024-03-18T14:50:57.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/151: fdatasync d8/d1e/d11/f21 0 2024-03-18T14:50:57.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/152: stat d8/d1e/c2b 0 2024-03-18T14:50:57.834 INFO:tasks.workunit.client.0.smithi012.stdout:0/495: mknod d1/d16/d31/cac 0 2024-03-18T14:50:57.835 INFO:tasks.workunit.client.1.smithi100.stdout:1/153: dread d8/de/f25 [0,4194304] 0 2024-03-18T14:50:57.835 INFO:tasks.workunit.client.1.smithi100.stdout:6/141: truncate f4 7223477 0 2024-03-18T14:50:57.835 INFO:tasks.workunit.client.1.smithi100.stdout:6/142: fsync d6/f15 0 2024-03-18T14:50:57.836 INFO:tasks.workunit.client.0.smithi012.stdout:5/509: mknod d3/d15/d5f/caf 0 2024-03-18T14:50:57.837 INFO:tasks.workunit.client.1.smithi100.stdout:9/162: mknod da/db/c32 0 2024-03-18T14:50:57.838 INFO:tasks.workunit.client.0.smithi012.stdout:3/454: mkdir dc/d3c/d72/d8f 0 2024-03-18T14:50:57.839 INFO:tasks.workunit.client.1.smithi100.stdout:7/137: link d5/l8 d5/d22/l36 0 2024-03-18T14:50:57.841 INFO:tasks.workunit.client.0.smithi012.stdout:2/542: link d0/d10/l12 d0/d10/d18/d31/la7 0 2024-03-18T14:50:57.841 INFO:tasks.workunit.client.1.smithi100.stdout:0/207: truncate f2 2283543 0 2024-03-18T14:50:57.842 INFO:tasks.workunit.client.0.smithi012.stdout:7/506: dwrite dd/d13/d23/d33/fa8 [0,4194304] 0 2024-03-18T14:50:57.842 INFO:tasks.workunit.client.0.smithi012.stdout:7/507: readlink dd/d13/l18 0 2024-03-18T14:50:57.842 INFO:tasks.workunit.client.0.smithi012.stdout:7/508: write dd/d4f/f69 [730892,122748] 0 2024-03-18T14:50:57.843 INFO:tasks.workunit.client.0.smithi012.stdout:1/464: truncate d8/d1e/d27/f48 242962 0 2024-03-18T14:50:57.843 INFO:tasks.workunit.client.0.smithi012.stdout:1/465: fdatasync d8/db/d10/f7f 0 2024-03-18T14:50:57.844 INFO:tasks.workunit.client.1.smithi100.stdout:3/168: mkdir d3/d16/d13/d1f/d37 0 2024-03-18T14:50:57.844 INFO:tasks.workunit.client.0.smithi012.stdout:5/510: symlink d3/d29/d36/lb0 0 2024-03-18T14:50:57.846 INFO:tasks.workunit.client.0.smithi012.stdout:3/455: mknod dc/d3c/d4c/d5f/c90 0 2024-03-18T14:50:57.848 INFO:tasks.workunit.client.0.smithi012.stdout:2/543: symlink d0/d10/d18/d4f/la8 0 2024-03-18T14:50:57.850 INFO:tasks.workunit.client.0.smithi012.stdout:0/496: dread d1/d20/f2e [0,4194304] 0 2024-03-18T14:50:57.850 INFO:tasks.workunit.client.0.smithi012.stdout:0/497: creat d1/d16/d19/d8d/fad x:0 0 0 2024-03-18T14:50:57.850 INFO:tasks.workunit.client.0.smithi012.stdout:0/498: fdatasync d1/d20/d23/d7f/f97 0 2024-03-18T14:50:57.850 INFO:tasks.workunit.client.1.smithi100.stdout:2/120: truncate d3/fc 220994 0 2024-03-18T14:50:57.851 INFO:tasks.workunit.client.1.smithi100.stdout:6/143: dread d6/fb [0,4194304] 0 2024-03-18T14:50:57.851 INFO:tasks.workunit.client.0.smithi012.stdout:7/509: creat dd/d13/d23/d76/fb2 x:0 0 0 2024-03-18T14:50:57.853 INFO:tasks.workunit.client.0.smithi012.stdout:1/466: mkdir d8/db/d10/d31/d81/d8d 0 2024-03-18T14:50:57.854 INFO:tasks.workunit.client.1.smithi100.stdout:5/137: unlink f24 0 2024-03-18T14:50:57.854 INFO:tasks.workunit.client.1.smithi100.stdout:5/138: fsync f26 0 2024-03-18T14:50:57.855 INFO:tasks.workunit.client.1.smithi100.stdout:1/154: symlink d8/de/l2c 0 2024-03-18T14:50:57.856 INFO:tasks.workunit.client.0.smithi012.stdout:2/544: creat d0/d6c/fa9 x:0 0 0 2024-03-18T14:50:57.857 INFO:tasks.workunit.client.1.smithi100.stdout:9/163: getdents da/d2c 0 2024-03-18T14:50:57.859 INFO:tasks.workunit.client.0.smithi012.stdout:4/466: dwrite d6/d19/d39/f50 [0,4194304] 0 2024-03-18T14:50:57.859 INFO:tasks.workunit.client.0.smithi012.stdout:4/467: chown d6/d1a/d30/d46/f6b 3798 1 2024-03-18T14:50:57.861 INFO:tasks.workunit.client.0.smithi012.stdout:1/467: dread d8/db/d10/d31/d45/f77 [0,4194304] 0 2024-03-18T14:50:57.862 INFO:tasks.workunit.client.1.smithi100.stdout:0/208: creat dd/d31/f52 x:0 0 0 2024-03-18T14:50:57.863 INFO:tasks.workunit.client.1.smithi100.stdout:0/209: chown dd/c19 106 1 2024-03-18T14:50:57.863 INFO:tasks.workunit.client.1.smithi100.stdout:0/210: chown dd/d31/c46 0 1 2024-03-18T14:50:57.864 INFO:tasks.workunit.client.0.smithi012.stdout:0/499: creat d1/d20/d23/d3b/d33/d51/fae x:0 0 0 2024-03-18T14:50:57.864 INFO:tasks.workunit.client.0.smithi012.stdout:0/500: readlink d1/d20/d23/l9e 0 2024-03-18T14:50:57.864 INFO:tasks.workunit.client.0.smithi012.stdout:0/501: creat d1/d20/d23/d7f/faf x:0 0 0 2024-03-18T14:50:57.865 INFO:tasks.workunit.client.1.smithi100.stdout:7/138: dwrite d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:50:57.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/169: truncate d3/fa 2228851 0 2024-03-18T14:50:57.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/170: write d3/ff [1226791,17715] 0 2024-03-18T14:50:57.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/171: creat d3/dd/f38 x:0 0 0 2024-03-18T14:50:57.866 INFO:tasks.workunit.client.1.smithi100.stdout:4/164: dwrite d9/d13/f2f [0,4194304] 0 2024-03-18T14:50:57.867 INFO:tasks.workunit.client.0.smithi012.stdout:8/707: dwrite d0/d47/d59/d67/fda [4194304,4194304] 0 2024-03-18T14:50:57.867 INFO:tasks.workunit.client.0.smithi012.stdout:7/510: unlink dd/d13/d23/d2f/d7a/f8b 0 2024-03-18T14:50:57.869 INFO:tasks.workunit.client.1.smithi100.stdout:5/139: symlink d29/l2e 0 2024-03-18T14:50:57.871 INFO:tasks.workunit.client.0.smithi012.stdout:3/456: mkdir dc/d91 0 2024-03-18T14:50:57.874 INFO:tasks.workunit.client.1.smithi100.stdout:9/164: write da/db/f23 [706077,80312] 0 2024-03-18T14:50:57.874 INFO:tasks.workunit.client.1.smithi100.stdout:9/165: chown f2 21987145 1 2024-03-18T14:50:57.878 INFO:tasks.workunit.client.1.smithi100.stdout:0/211: symlink dd/d2b/l53 0 2024-03-18T14:50:57.878 INFO:tasks.workunit.client.1.smithi100.stdout:7/139: dread d5/fd [0,4194304] 0 2024-03-18T14:50:57.883 INFO:tasks.workunit.client.0.smithi012.stdout:4/468: symlink d6/d1a/d30/d46/d6f/l99 0 2024-03-18T14:50:57.887 INFO:tasks.workunit.client.0.smithi012.stdout:7/511: dwrite dd/d13/d23/d33/f51 [0,4194304] 0 2024-03-18T14:50:57.887 INFO:tasks.workunit.client.0.smithi012.stdout:1/468: rename d8/db/d10/d31/d43/d5e/d65/l7b to d8/db/d10/d16/d6b/d7e/l8e 0 2024-03-18T14:50:57.887 INFO:tasks.workunit.client.0.smithi012.stdout:1/469: readlink d8/db/d10/d16/d54/l58 0 2024-03-18T14:50:57.889 INFO:tasks.workunit.client.0.smithi012.stdout:1/470: dread d8/db/f30 [0,4194304] 0 2024-03-18T14:50:57.894 INFO:tasks.workunit.client.0.smithi012.stdout:0/502: unlink d1/d16/d6e/c95 0 2024-03-18T14:50:57.895 INFO:tasks.workunit.client.0.smithi012.stdout:0/503: creat d1/d20/d23/d6f/fb0 x:0 0 0 2024-03-18T14:50:57.895 INFO:tasks.workunit.client.0.smithi012.stdout:0/504: fsync d1/d16/d19/d8d/fab 0 2024-03-18T14:50:57.895 INFO:tasks.workunit.client.0.smithi012.stdout:0/505: read d1/f5f [319355,101138] 0 2024-03-18T14:50:57.897 INFO:tasks.workunit.client.1.smithi100.stdout:3/172: link d3/d16/f14 d3/d16/d13/f39 0 2024-03-18T14:50:57.911 INFO:tasks.workunit.client.1.smithi100.stdout:5/140: rename l1a to d29/d2d/l2f 0 2024-03-18T14:50:57.913 INFO:tasks.workunit.client.1.smithi100.stdout:1/155: getdents d8/d1e/d11 0 2024-03-18T14:50:57.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/166: mknod da/dd/c33 0 2024-03-18T14:50:57.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/167: creat da/dd/d29/f34 x:0 0 0 2024-03-18T14:50:57.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/168: stat da/db/f13 0 2024-03-18T14:50:57.917 INFO:tasks.workunit.client.1.smithi100.stdout:9/169: chown da/f1d 1613 1 2024-03-18T14:50:57.919 INFO:tasks.workunit.client.1.smithi100.stdout:0/212: mkdir dd/d2b/d54 0 2024-03-18T14:50:57.922 INFO:tasks.workunit.client.1.smithi100.stdout:7/140: mkdir d5/d22/d24/d37 0 2024-03-18T14:50:57.924 INFO:tasks.workunit.client.1.smithi100.stdout:2/121: dwrite d3/fc [0,4194304] 0 2024-03-18T14:50:57.927 INFO:tasks.workunit.client.1.smithi100.stdout:3/173: symlink d3/d16/d13/d17/l3a 0 2024-03-18T14:50:57.932 INFO:tasks.workunit.client.1.smithi100.stdout:4/165: getdents d9 0 2024-03-18T14:50:57.932 INFO:tasks.workunit.client.1.smithi100.stdout:4/166: write f0 [7342010,83879] 0 2024-03-18T14:50:57.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/170: dwrite da/fc [0,4194304] 0 2024-03-18T14:50:57.948 INFO:tasks.workunit.client.0.smithi012.stdout:8/708: rmdir d0/d47/d59/dc9 39 2024-03-18T14:50:57.949 INFO:tasks.workunit.client.0.smithi012.stdout:2/545: dwrite d0/da2/d59/f8c [0,4194304] 0 2024-03-18T14:50:57.954 INFO:tasks.workunit.client.0.smithi012.stdout:3/457: write dc/d30/d3a/f5e [83488,50654] 0 2024-03-18T14:50:57.954 INFO:tasks.workunit.client.0.smithi012.stdout:3/458: dread - dc/d30/d35/f66 zero size 2024-03-18T14:50:57.954 INFO:tasks.workunit.client.1.smithi100.stdout:2/122: link d3/da/de/d20/f23 d3/d6/f2f 0 2024-03-18T14:50:57.955 INFO:tasks.workunit.client.0.smithi012.stdout:7/512: mkdir dd/d13/d23/d76/d6f/db3 0 2024-03-18T14:50:57.955 INFO:tasks.workunit.client.0.smithi012.stdout:7/513: chown dd/d13/d26 2659113 1 2024-03-18T14:50:57.955 INFO:tasks.workunit.client.0.smithi012.stdout:7/514: fdatasync dd/f10 0 2024-03-18T14:50:57.962 INFO:tasks.workunit.client.1.smithi100.stdout:9/171: dwrite f2 [0,4194304] 0 2024-03-18T14:50:57.963 INFO:tasks.workunit.client.0.smithi012.stdout:0/506: rmdir d1/d20/d23/d3b/d33/d51/d99 0 2024-03-18T14:50:57.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/469: dwrite d6/d1a/d30/d46/d6f/d7c/f87 [0,4194304] 0 2024-03-18T14:50:57.964 INFO:tasks.workunit.client.0.smithi012.stdout:1/471: dwrite d8/db/f15 [0,4194304] 0 2024-03-18T14:50:57.964 INFO:tasks.workunit.client.1.smithi100.stdout:1/156: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:50:57.964 INFO:tasks.workunit.client.1.smithi100.stdout:1/157: fdatasync d8/de/f25 0 2024-03-18T14:50:57.964 INFO:tasks.workunit.client.1.smithi100.stdout:1/158: write d8/de/f18 [1420697,117221] 0 2024-03-18T14:50:57.964 INFO:tasks.workunit.client.1.smithi100.stdout:1/159: read d8/de/f18 [3959122,112499] 0 2024-03-18T14:50:57.965 INFO:tasks.workunit.client.1.smithi100.stdout:4/167: unlink d9/l2d 0 2024-03-18T14:50:57.968 INFO:tasks.workunit.client.0.smithi012.stdout:8/709: unlink d0/d5a/d19/dad/d33/d50/ldb 0 2024-03-18T14:50:57.968 INFO:tasks.workunit.client.0.smithi012.stdout:8/710: readlink d0/d15/l51 0 2024-03-18T14:50:57.968 INFO:tasks.workunit.client.1.smithi100.stdout:0/213: rmdir dd/d39 39 2024-03-18T14:50:57.976 INFO:tasks.workunit.client.1.smithi100.stdout:5/141: rename c1f to d29/d2d/c30 0 2024-03-18T14:50:57.980 INFO:tasks.workunit.client.0.smithi012.stdout:2/546: mkdir d0/d10/d18/d33/d99/daa 0 2024-03-18T14:50:57.983 INFO:tasks.workunit.client.1.smithi100.stdout:9/172: dwrite da/dd/f28 [0,4194304] 0 2024-03-18T14:50:57.985 INFO:tasks.workunit.client.0.smithi012.stdout:3/459: creat dc/f92 x:0 0 0 2024-03-18T14:50:57.987 INFO:tasks.workunit.client.0.smithi012.stdout:7/515: symlink dd/d13/d23/lb4 0 2024-03-18T14:50:57.987 INFO:tasks.workunit.client.0.smithi012.stdout:7/516: write dd/d13/d23/f99 [219367,25090] 0 2024-03-18T14:50:57.990 INFO:tasks.workunit.client.1.smithi100.stdout:0/214: dwrite dd/d20/f32 [0,4194304] 0 2024-03-18T14:50:57.991 INFO:tasks.workunit.client.0.smithi012.stdout:0/507: rename d1/d20/d23/d82/c8e to d1/d20/d4e/d54/cb1 0 2024-03-18T14:50:57.993 INFO:tasks.workunit.client.0.smithi012.stdout:4/470: creat d6/da/d89/d97/f9a x:0 0 0 2024-03-18T14:50:57.996 INFO:tasks.workunit.client.0.smithi012.stdout:1/472: creat d8/db/d10/d56/f8f x:0 0 0 2024-03-18T14:50:57.997 INFO:tasks.workunit.client.1.smithi100.stdout:4/168: link d9/f32 d9/d17/f34 0 2024-03-18T14:50:57.997 INFO:tasks.workunit.client.1.smithi100.stdout:4/169: stat d9/f2b 0 2024-03-18T14:50:57.999 INFO:tasks.workunit.client.0.smithi012.stdout:8/711: unlink d0/d5a/c98 0 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.1.smithi100.stdout:5/142: chown l1d 42 1 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/460: dwrite dc/d30/f37 [0,4194304] 0 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/461: truncate dc/d30/f6d 151513 0 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/462: write dc/d30/d35/f75 [525432,127064] 0 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/463: truncate dc/d30/d35/f66 55874 0 2024-03-18T14:50:58.002 INFO:tasks.workunit.client.0.smithi012.stdout:3/464: stat dc/f12 0 2024-03-18T14:50:58.003 INFO:tasks.workunit.client.0.smithi012.stdout:2/547: getdents d0/da2/d59 0 2024-03-18T14:50:58.003 INFO:tasks.workunit.client.0.smithi012.stdout:2/548: chown d0/l4a 137 1 2024-03-18T14:50:58.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/141: rename d5/c33 to d5/d34/c38 0 2024-03-18T14:50:58.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/142: truncate d5/fd 4432038 0 2024-03-18T14:50:58.008 INFO:tasks.workunit.client.0.smithi012.stdout:7/517: symlink dd/d13/d26/lb5 0 2024-03-18T14:50:58.012 INFO:tasks.workunit.client.0.smithi012.stdout:8/712: dread d0/d5a/fa2 [0,4194304] 0 2024-03-18T14:50:58.012 INFO:tasks.workunit.client.0.smithi012.stdout:0/508: truncate d1/d2d/f60 4687474 0 2024-03-18T14:50:58.012 INFO:tasks.workunit.client.0.smithi012.stdout:0/509: readlink d1/d16/d19/l34 0 2024-03-18T14:50:58.014 INFO:tasks.workunit.client.1.smithi100.stdout:7/143: dread d5/d14/f18 [0,4194304] 0 2024-03-18T14:50:58.014 INFO:tasks.workunit.client.1.smithi100.stdout:7/144: readlink d5/d14/d16/d2e/l31 0 2024-03-18T14:50:58.014 INFO:tasks.workunit.client.1.smithi100.stdout:7/145: chown d5/ce 0 1 2024-03-18T14:50:58.015 INFO:tasks.workunit.client.0.smithi012.stdout:1/473: mknod d8/db/d10/d16/d6b/d7e/c90 0 2024-03-18T14:50:58.015 INFO:tasks.workunit.client.0.smithi012.stdout:1/474: chown d8/db/d10/d16/d1c 19418429 1 2024-03-18T14:50:58.017 INFO:tasks.workunit.client.0.smithi012.stdout:3/465: mkdir dc/d3c/d4c/d50/d64/d93 0 2024-03-18T14:50:58.017 INFO:tasks.workunit.client.0.smithi012.stdout:3/466: chown dc/d91 0 1 2024-03-18T14:50:58.017 INFO:tasks.workunit.client.0.smithi012.stdout:3/467: readlink dc/d3c/d4c/l6e 0 2024-03-18T14:50:58.017 INFO:tasks.workunit.client.0.smithi012.stdout:4/471: dwrite d6/d19/f2f [0,4194304] 0 2024-03-18T14:50:58.018 INFO:tasks.workunit.client.0.smithi012.stdout:4/472: chown d6/da/d89/c90 2030307 1 2024-03-18T14:50:58.018 INFO:tasks.workunit.client.0.smithi012.stdout:4/473: write d6/d19/f85 [430112,94807] 0 2024-03-18T14:50:58.018 INFO:tasks.workunit.client.0.smithi012.stdout:4/474: fdatasync d6/d19/d6c/f7a 0 2024-03-18T14:50:58.018 INFO:tasks.workunit.client.0.smithi012.stdout:2/549: mknod d0/d10/d18/d33/d71/d90/cab 0 2024-03-18T14:50:58.022 INFO:tasks.workunit.client.0.smithi012.stdout:2/550: dread d0/f7b [0,4194304] 0 2024-03-18T14:50:58.022 INFO:tasks.workunit.client.1.smithi100.stdout:2/123: rmdir d3 39 2024-03-18T14:50:58.024 INFO:tasks.workunit.client.0.smithi012.stdout:8/713: mknod d0/d5a/d5e/dd7/d84/cdc 0 2024-03-18T14:50:58.024 INFO:tasks.workunit.client.0.smithi012.stdout:8/714: write d0/d47/d59/f83 [581943,31610] 0 2024-03-18T14:50:58.024 INFO:tasks.workunit.client.0.smithi012.stdout:8/715: write d0/d5a/d19/f75 [825340,20628] 0 2024-03-18T14:50:58.025 INFO:tasks.workunit.client.1.smithi100.stdout:9/173: creat da/d25/f35 x:0 0 0 2024-03-18T14:50:58.026 INFO:tasks.workunit.client.1.smithi100.stdout:9/174: write da/db/f23 [432049,3618] 0 2024-03-18T14:50:58.029 INFO:tasks.workunit.client.0.smithi012.stdout:1/475: rmdir d8/db/d10/d31/d81/d8d 0 2024-03-18T14:50:58.029 INFO:tasks.workunit.client.0.smithi012.stdout:1/476: fdatasync d8/db/d10/d16/f66 0 2024-03-18T14:50:58.030 INFO:tasks.workunit.client.1.smithi100.stdout:2/124: write d3/da/de/d20/f24 [1617675,54426] 0 2024-03-18T14:50:58.030 INFO:tasks.workunit.client.1.smithi100.stdout:2/125: read - d3/da/de/f28 zero size 2024-03-18T14:50:58.030 INFO:tasks.workunit.client.1.smithi100.stdout:2/126: stat d3/d6 0 2024-03-18T14:50:58.033 INFO:tasks.workunit.client.0.smithi012.stdout:3/468: creat dc/d3c/d4c/f94 x:0 0 0 2024-03-18T14:50:58.035 INFO:tasks.workunit.client.0.smithi012.stdout:1/477: dread d8/f40 [0,4194304] 0 2024-03-18T14:50:58.035 INFO:tasks.workunit.client.0.smithi012.stdout:1/478: chown d8/db/d10/d16/d6b/d7e/c86 2432 1 2024-03-18T14:50:58.035 INFO:tasks.workunit.client.0.smithi012.stdout:1/479: write d8/db/d10/f33 [295711,9000] 0 2024-03-18T14:50:58.036 INFO:tasks.workunit.client.1.smithi100.stdout:8/154: sync 2024-03-18T14:50:58.039 INFO:tasks.workunit.client.1.smithi100.stdout:4/170: symlink d9/d13/d25/l35 0 2024-03-18T14:50:58.042 INFO:tasks.workunit.client.1.smithi100.stdout:5/143: fdatasync d29/f2a 0 2024-03-18T14:50:58.042 INFO:tasks.workunit.client.1.smithi100.stdout:4/171: dread d9/f10 [0,4194304] 0 2024-03-18T14:50:58.044 INFO:tasks.workunit.client.0.smithi012.stdout:3/469: dread dc/f6a [0,4194304] 0 2024-03-18T14:50:58.044 INFO:tasks.workunit.client.0.smithi012.stdout:2/551: mkdir d0/d10/d18/d33/dac 0 2024-03-18T14:50:58.046 INFO:tasks.workunit.client.0.smithi012.stdout:8/716: rmdir d0/d5a/d5e/dd7 39 2024-03-18T14:50:58.050 INFO:tasks.workunit.client.0.smithi012.stdout:0/510: getdents d1/d20/d4e/d54/d67 0 2024-03-18T14:50:58.050 INFO:tasks.workunit.client.0.smithi012.stdout:0/511: stat d1/d20/d23/d6f/fb0 0 2024-03-18T14:50:58.050 INFO:tasks.workunit.client.0.smithi012.stdout:0/512: chown d1/d20/d4e 12797 1 2024-03-18T14:50:58.055 INFO:tasks.workunit.client.0.smithi012.stdout:3/470: write dc/d3c/d4c/d50/f5d [3998892,102531] 0 2024-03-18T14:50:58.061 INFO:tasks.workunit.client.0.smithi012.stdout:3/471: write dc/f11 [1394742,96398] 0 2024-03-18T14:50:58.061 INFO:tasks.workunit.client.0.smithi012.stdout:3/472: fsync f3 0 2024-03-18T14:50:58.062 INFO:tasks.workunit.client.0.smithi012.stdout:3/473: dread dc/d3c/d59/f5a [0,4194304] 0 2024-03-18T14:50:58.063 INFO:tasks.workunit.client.0.smithi012.stdout:7/518: rename dd/d13/d26 to dd/d13/d23/d2f/d7a/db6 0 2024-03-18T14:50:58.071 INFO:tasks.workunit.client.0.smithi012.stdout:4/475: rename d6/da/d58/c5f to d6/d1a/d30/d46/d6f/d8b/c9b 0 2024-03-18T14:50:58.073 INFO:tasks.workunit.client.1.smithi100.stdout:9/175: creat da/db/f36 x:0 0 0 2024-03-18T14:50:58.073 INFO:tasks.workunit.client.0.smithi012.stdout:7/519: mkdir dd/d13/db7 0 2024-03-18T14:50:58.073 INFO:tasks.workunit.client.0.smithi012.stdout:7/520: readlink dd/d13/d23/d2f/d7a/db6/l7c 0 2024-03-18T14:50:58.076 INFO:tasks.workunit.client.1.smithi100.stdout:2/127: dwrite d3/f17 [0,4194304] 0 2024-03-18T14:50:58.077 INFO:tasks.workunit.client.0.smithi012.stdout:3/474: rename dc/c88 to dc/d3c/d59/c95 0 2024-03-18T14:50:58.078 INFO:tasks.workunit.client.0.smithi012.stdout:3/475: write dc/d3c/f4d [508963,96620] 0 2024-03-18T14:50:58.079 INFO:tasks.workunit.client.0.smithi012.stdout:0/513: dwrite d1/d16/d19/f2a [4194304,4194304] 0 2024-03-18T14:50:58.080 INFO:tasks.workunit.client.0.smithi012.stdout:1/480: dwrite d8/db/d10/d31/d81/f8c [0,4194304] 0 2024-03-18T14:50:58.081 INFO:tasks.workunit.client.0.smithi012.stdout:8/717: dwrite d0/d5a/d19/fa3 [4194304,4194304] 0 2024-03-18T14:50:58.081 INFO:tasks.workunit.client.0.smithi012.stdout:1/481: chown d8/db/d10/l32 23 1 2024-03-18T14:50:58.081 INFO:tasks.workunit.client.0.smithi012.stdout:1/482: fsync d8/db/d10/f60 0 2024-03-18T14:50:58.085 INFO:tasks.workunit.client.1.smithi100.stdout:0/215: rmdir dd 39 2024-03-18T14:50:58.086 INFO:tasks.workunit.client.0.smithi012.stdout:7/521: rename dd/d13/l77 to dd/d13/d23/d33/d37/lb8 0 2024-03-18T14:50:58.086 INFO:tasks.workunit.client.0.smithi012.stdout:2/552: rmdir d0/d10/d18/d33 39 2024-03-18T14:50:58.092 INFO:tasks.workunit.client.0.smithi012.stdout:3/476: dread dc/f44 [0,4194304] 0 2024-03-18T14:50:58.092 INFO:tasks.workunit.client.0.smithi012.stdout:3/477: stat dc/d3c/d59/f5a 0 2024-03-18T14:50:58.093 INFO:tasks.workunit.client.1.smithi100.stdout:8/155: creat d4/dc/d1f/f32 x:0 0 0 2024-03-18T14:50:58.095 INFO:tasks.workunit.client.0.smithi012.stdout:0/514: symlink d1/d20/d7b/lb2 0 2024-03-18T14:50:58.095 INFO:tasks.workunit.client.0.smithi012.stdout:0/515: readlink d1/d20/d7b/l11 0 2024-03-18T14:50:58.097 INFO:tasks.workunit.client.1.smithi100.stdout:5/144: mknod d29/c31 0 2024-03-18T14:50:58.101 INFO:tasks.workunit.client.1.smithi100.stdout:4/172: creat d9/d27/f36 x:0 0 0 2024-03-18T14:50:58.102 INFO:tasks.workunit.client.1.smithi100.stdout:9/176: dwrite da/db/f36 [0,4194304] 0 2024-03-18T14:50:58.102 INFO:tasks.workunit.client.1.smithi100.stdout:9/177: fdatasync da/f1f 0 2024-03-18T14:50:58.105 INFO:tasks.workunit.client.1.smithi100.stdout:7/146: getdents d5 0 2024-03-18T14:50:58.108 INFO:tasks.workunit.client.1.smithi100.stdout:2/128: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-18T14:50:58.110 INFO:tasks.workunit.client.1.smithi100.stdout:8/156: unlink d4/cb 0 2024-03-18T14:50:58.112 INFO:tasks.workunit.client.0.smithi012.stdout:0/516: dwrite d1/d20/d4e/d54/f57 [0,4194304] 0 2024-03-18T14:50:58.112 INFO:tasks.workunit.client.0.smithi012.stdout:8/718: dwrite d0/d5a/d19/f75 [0,4194304] 0 2024-03-18T14:50:58.113 INFO:tasks.workunit.client.1.smithi100.stdout:5/145: write f7 [2440947,79972] 0 2024-03-18T14:50:58.117 INFO:tasks.workunit.client.1.smithi100.stdout:4/173: truncate f8 2014367 0 2024-03-18T14:50:58.117 INFO:tasks.workunit.client.1.smithi100.stdout:4/174: write d9/d27/f36 [346196,115029] 0 2024-03-18T14:50:58.117 INFO:tasks.workunit.client.1.smithi100.stdout:4/175: write d9/f1f [419262,7829] 0 2024-03-18T14:50:58.117 INFO:tasks.workunit.client.1.smithi100.stdout:4/176: readlink d9/la 0 2024-03-18T14:50:58.121 INFO:tasks.workunit.client.1.smithi100.stdout:9/178: rmdir da/d2c 39 2024-03-18T14:50:58.121 INFO:tasks.workunit.client.1.smithi100.stdout:9/179: chown da/dd/f28 11235 1 2024-03-18T14:50:58.123 INFO:tasks.workunit.client.1.smithi100.stdout:1/160: rename d8/de to d8/d1e/d11/d2d 0 2024-03-18T14:50:58.123 INFO:tasks.workunit.client.1.smithi100.stdout:1/161: chown d8/d1e/l13 19323293 1 2024-03-18T14:50:58.123 INFO:tasks.workunit.client.1.smithi100.stdout:1/162: fdatasync d8/d1e/d11/d2d/f14 0 2024-03-18T14:50:58.125 INFO:tasks.workunit.client.1.smithi100.stdout:7/147: truncate d5/d14/d16/f1a 69698 0 2024-03-18T14:50:58.125 INFO:tasks.workunit.client.1.smithi100.stdout:8/157: fsync d4/dc/d1f/f32 0 2024-03-18T14:50:58.125 INFO:tasks.workunit.client.1.smithi100.stdout:8/158: readlink d4/l24 0 2024-03-18T14:50:58.131 INFO:tasks.workunit.client.1.smithi100.stdout:8/159: dread f1 [0,4194304] 0 2024-03-18T14:50:58.131 INFO:tasks.workunit.client.1.smithi100.stdout:8/160: read d4/dc/d1f/f20 [1021815,68272] 0 2024-03-18T14:50:58.132 INFO:tasks.workunit.client.1.smithi100.stdout:2/129: dwrite d3/da/de/d20/f23 [0,4194304] 0 2024-03-18T14:50:58.132 INFO:tasks.workunit.client.1.smithi100.stdout:8/161: chown d4/dc/dd/de/l16 125617 1 2024-03-18T14:50:58.132 INFO:tasks.workunit.client.1.smithi100.stdout:8/162: fsync f3 0 2024-03-18T14:50:58.132 INFO:tasks.workunit.client.1.smithi100.stdout:5/146: fdatasync f12 0 2024-03-18T14:50:58.138 INFO:tasks.workunit.client.1.smithi100.stdout:9/180: dwrite da/dd/f10 [0,4194304] 0 2024-03-18T14:50:58.140 INFO:tasks.workunit.client.1.smithi100.stdout:0/216: rename dd/d20/f25 to dd/f55 0 2024-03-18T14:50:58.141 INFO:tasks.workunit.client.1.smithi100.stdout:7/148: rmdir d5/d14/d16/d2e 39 2024-03-18T14:50:58.142 INFO:tasks.workunit.client.0.smithi012.stdout:1/483: rename d8/db/d10/d16/d1c/d2e/c83 to d8/db/d10/d31/d43/d5e/c91 0 2024-03-18T14:50:58.143 INFO:tasks.workunit.client.1.smithi100.stdout:3/174: sync 2024-03-18T14:50:58.143 INFO:tasks.workunit.client.1.smithi100.stdout:6/144: sync 2024-03-18T14:50:58.144 INFO:tasks.workunit.client.1.smithi100.stdout:0/217: dread dd/d39/f3f [0,4194304] 0 2024-03-18T14:50:58.144 INFO:tasks.workunit.client.1.smithi100.stdout:0/218: truncate fa 4756751 0 2024-03-18T14:50:58.145 INFO:tasks.workunit.client.0.smithi012.stdout:3/478: creat dc/d3c/d59/d86/f96 x:0 0 0 2024-03-18T14:50:58.177 INFO:tasks.workunit.client.1.smithi100.stdout:2/130: creat d3/da/f30 x:0 0 0 2024-03-18T14:50:58.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/163: mkdir d4/dc/d1f/d33 0 2024-03-18T14:50:58.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/164: fsync d4/dc/dd/f11 0 2024-03-18T14:50:58.180 INFO:tasks.workunit.client.0.smithi012.stdout:0/517: link d1/f9c d1/d16/d19/fb3 0 2024-03-18T14:50:58.180 INFO:tasks.workunit.client.1.smithi100.stdout:5/147: stat f2 0 2024-03-18T14:50:58.183 INFO:tasks.workunit.client.1.smithi100.stdout:8/165: dread f1 [0,4194304] 0 2024-03-18T14:50:58.183 INFO:tasks.workunit.client.0.smithi012.stdout:8/719: creat d0/fdd x:0 0 0 2024-03-18T14:50:58.183 INFO:tasks.workunit.client.0.smithi012.stdout:8/720: readlink d0/d47/la0 0 2024-03-18T14:50:58.185 INFO:tasks.workunit.client.0.smithi012.stdout:6/508: sync 2024-03-18T14:50:58.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/511: sync 2024-03-18T14:50:58.185 INFO:tasks.workunit.client.0.smithi012.stdout:9/573: sync 2024-03-18T14:50:58.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/512: write d3/f9e [1197382,77528] 0 2024-03-18T14:50:58.185 INFO:tasks.workunit.client.0.smithi012.stdout:5/513: fdatasync d3/d29/d36/d3d/d65/f6c 0 2024-03-18T14:50:58.186 INFO:tasks.workunit.client.0.smithi012.stdout:0/518: dread d1/f3 [0,4194304] 0 2024-03-18T14:50:58.186 INFO:tasks.workunit.client.0.smithi012.stdout:7/522: getdents dd/d4f 0 2024-03-18T14:50:58.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/181: unlink da/db/f23 0 2024-03-18T14:50:58.187 INFO:tasks.workunit.client.1.smithi100.stdout:9/182: chown f6 95461924 1 2024-03-18T14:50:58.187 INFO:tasks.workunit.client.1.smithi100.stdout:9/183: creat da/d25/f37 x:0 0 0 2024-03-18T14:50:58.189 INFO:tasks.workunit.client.0.smithi012.stdout:1/484: creat d8/db/d10/d31/d43/d5e/d65/d84/f92 x:0 0 0 2024-03-18T14:50:58.189 INFO:tasks.workunit.client.0.smithi012.stdout:1/485: creat d8/db/d10/d16/d54/f93 x:0 0 0 2024-03-18T14:50:58.189 INFO:tasks.workunit.client.0.smithi012.stdout:1/486: fsync d8/d1e/d27/f48 0 2024-03-18T14:50:58.189 INFO:tasks.workunit.client.0.smithi012.stdout:1/487: chown d8/db/d10/d16/d54/l58 1871046006 1 2024-03-18T14:50:58.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/175: creat d3/d16/d13/f3b x:0 0 0 2024-03-18T14:50:58.194 INFO:tasks.workunit.client.1.smithi100.stdout:6/145: symlink d6/d7/df/d26/l28 0 2024-03-18T14:50:58.197 INFO:tasks.workunit.client.0.smithi012.stdout:9/574: dread d3/f83 [0,4194304] 0 2024-03-18T14:50:58.197 INFO:tasks.workunit.client.0.smithi012.stdout:9/575: truncate d3/f83 4488120 0 2024-03-18T14:50:58.198 INFO:tasks.workunit.client.0.smithi012.stdout:1/488: write f5 [5912910,26307] 0 2024-03-18T14:50:58.198 INFO:tasks.workunit.client.0.smithi012.stdout:1/489: dread - d8/db/d10/d31/d43/f69 zero size 2024-03-18T14:50:58.198 INFO:tasks.workunit.client.0.smithi012.stdout:1/490: creat d8/db/d10/f94 x:0 0 0 2024-03-18T14:50:58.198 INFO:tasks.workunit.client.0.smithi012.stdout:8/721: unlink d0/d5a/l7f 0 2024-03-18T14:50:58.199 INFO:tasks.workunit.client.0.smithi012.stdout:8/722: write d0/d47/f4f [3138398,83426] 0 2024-03-18T14:50:58.199 INFO:tasks.workunit.client.0.smithi012.stdout:3/479: fsync dc/d3c/d59/d86/f96 0 2024-03-18T14:50:58.200 INFO:tasks.workunit.client.1.smithi100.stdout:0/219: mkdir dd/d39/d56 0 2024-03-18T14:50:58.200 INFO:tasks.workunit.client.1.smithi100.stdout:0/220: read dd/f2e [356194,27531] 0 2024-03-18T14:50:58.200 INFO:tasks.workunit.client.0.smithi012.stdout:6/509: fdatasync f8 0 2024-03-18T14:50:58.203 INFO:tasks.workunit.client.1.smithi100.stdout:2/131: link d3/d13/c1e d3/da/de/c31 0 2024-03-18T14:50:58.203 INFO:tasks.workunit.client.1.smithi100.stdout:6/146: dread f4 [4194304,4194304] 0 2024-03-18T14:50:58.204 INFO:tasks.workunit.client.1.smithi100.stdout:8/166: dwrite d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:58.205 INFO:tasks.workunit.client.1.smithi100.stdout:8/167: chown d4/dc/dd 3716 1 2024-03-18T14:50:58.205 INFO:tasks.workunit.client.1.smithi100.stdout:8/168: read d4/dc/dd/de/d19/f1e [543376,54777] 0 2024-03-18T14:50:58.205 INFO:tasks.workunit.client.0.smithi012.stdout:5/514: symlink d3/d29/d35/d5b/lb1 0 2024-03-18T14:50:58.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/515: write d3/fc [975258,128676] 0 2024-03-18T14:50:58.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/516: truncate d3/d1e/d33/f4b 1744845 0 2024-03-18T14:50:58.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/517: stat d3/d29/d36/d3d/d86 0 2024-03-18T14:50:58.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/518: chown d3/d29/d36/d9b/ca2 188915152 1 2024-03-18T14:50:58.206 INFO:tasks.workunit.client.0.smithi012.stdout:5/519: write d3/d29/d36/d3d/f5c [4382988,72168] 0 2024-03-18T14:50:58.207 INFO:tasks.workunit.client.0.smithi012.stdout:0/519: rename d1/d20/c63 to d1/d16/d19/cb4 0 2024-03-18T14:50:58.208 INFO:tasks.workunit.client.0.smithi012.stdout:0/520: dread - d1/d20/d23/d3b/d33/d64/f96 zero size 2024-03-18T14:50:58.209 INFO:tasks.workunit.client.1.smithi100.stdout:7/149: getdents d5/d14/d16/d2e 0 2024-03-18T14:50:58.210 INFO:tasks.workunit.client.1.smithi100.stdout:9/184: dwrite da/dd/d29/f34 [0,4194304] 0 2024-03-18T14:50:58.215 INFO:tasks.workunit.client.1.smithi100.stdout:3/176: rename d3/f6 to d3/d16/d13/d17/f3c 0 2024-03-18T14:50:58.216 INFO:tasks.workunit.client.1.smithi100.stdout:3/177: write d3/d16/d13/d17/f2b [83306,82480] 0 2024-03-18T14:50:58.216 INFO:tasks.workunit.client.1.smithi100.stdout:3/178: fdatasync d3/f5 0 2024-03-18T14:50:58.216 INFO:tasks.workunit.client.0.smithi012.stdout:9/576: mknod d3/d49/cb5 0 2024-03-18T14:50:58.218 INFO:tasks.workunit.client.0.smithi012.stdout:6/510: dwrite d17/d38/d5b/f87 [0,4194304] 0 2024-03-18T14:50:58.218 INFO:tasks.workunit.client.0.smithi012.stdout:6/511: fdatasync d17/f1a 0 2024-03-18T14:50:58.218 INFO:tasks.workunit.client.0.smithi012.stdout:6/512: fdatasync d17/d19/d27/d58/f6f 0 2024-03-18T14:50:58.219 INFO:tasks.workunit.client.1.smithi100.stdout:0/221: unlink dd/d39/f45 0 2024-03-18T14:50:58.223 INFO:tasks.workunit.client.1.smithi100.stdout:2/132: unlink d3/da/de/f1c 0 2024-03-18T14:50:58.223 INFO:tasks.workunit.client.1.smithi100.stdout:2/133: fsync d3/da/de/d20/f27 0 2024-03-18T14:50:58.229 INFO:tasks.workunit.client.1.smithi100.stdout:6/147: mknod d6/d10/c29 0 2024-03-18T14:50:58.231 INFO:tasks.workunit.client.0.smithi012.stdout:1/491: mknod d8/db/c95 0 2024-03-18T14:50:58.232 INFO:tasks.workunit.client.1.smithi100.stdout:8/169: rmdir d4/dc/dd 39 2024-03-18T14:50:58.232 INFO:tasks.workunit.client.1.smithi100.stdout:8/170: chown f3 0 1 2024-03-18T14:50:58.232 INFO:tasks.workunit.client.1.smithi100.stdout:8/171: chown d4/dc/dd/de/d21/d23/f27 1036230 1 2024-03-18T14:50:58.233 INFO:tasks.workunit.client.0.smithi012.stdout:8/723: mknod d0/d5a/d5e/dd7/d84/dc8/cde 0 2024-03-18T14:50:58.235 INFO:tasks.workunit.client.1.smithi100.stdout:7/150: mkdir d5/d22/d24/d39 0 2024-03-18T14:50:58.240 INFO:tasks.workunit.client.0.smithi012.stdout:3/480: rename dc/d30/d35/f47 to dc/d3c/d4c/d50/d54/f97 0 2024-03-18T14:50:58.240 INFO:tasks.workunit.client.0.smithi012.stdout:3/481: fsync dc/d30/d3a/f4f 0 2024-03-18T14:50:58.240 INFO:tasks.workunit.client.0.smithi012.stdout:3/482: dread - dc/d3c/d4c/d50/f52 zero size 2024-03-18T14:50:58.240 INFO:tasks.workunit.client.0.smithi012.stdout:3/483: stat dc/f48 0 2024-03-18T14:50:58.243 INFO:tasks.workunit.client.0.smithi012.stdout:0/521: getdents d1/d20/d23/d88 0 2024-03-18T14:50:58.243 INFO:tasks.workunit.client.0.smithi012.stdout:0/522: write d1/f9c [1007301,73404] 0 2024-03-18T14:50:58.244 INFO:tasks.workunit.client.0.smithi012.stdout:3/484: dread dc/f43 [0,4194304] 0 2024-03-18T14:50:58.244 INFO:tasks.workunit.client.0.smithi012.stdout:3/485: creat dc/d30/f98 x:0 0 0 2024-03-18T14:50:58.244 INFO:tasks.workunit.client.0.smithi012.stdout:3/486: creat dc/d30/f99 x:0 0 0 2024-03-18T14:50:58.245 INFO:tasks.workunit.client.0.smithi012.stdout:3/487: read - dc/d3c/d4c/d50/f7e zero size 2024-03-18T14:50:58.246 INFO:tasks.workunit.client.0.smithi012.stdout:7/523: rmdir dd/d13/d23/d60 39 2024-03-18T14:50:58.246 INFO:tasks.workunit.client.0.smithi012.stdout:7/524: fsync dd/f5b 0 2024-03-18T14:50:58.247 INFO:tasks.workunit.client.1.smithi100.stdout:3/179: mknod d3/d16/d13/d1f/d31/c3d 0 2024-03-18T14:50:58.248 INFO:tasks.workunit.client.1.smithi100.stdout:3/180: truncate d3/dd/f38 741571 0 2024-03-18T14:50:58.249 INFO:tasks.workunit.client.0.smithi012.stdout:9/577: symlink d3/d9/d19/d66/d21/d35/d48/lb6 0 2024-03-18T14:50:58.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/222: creat dd/d31/f57 x:0 0 0 2024-03-18T14:50:58.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/223: stat dd 0 2024-03-18T14:50:58.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/224: chown fa 198014 1 2024-03-18T14:50:58.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/225: dread - dd/f1f zero size 2024-03-18T14:50:58.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/226: readlink dd/d20/d44/l4b 0 2024-03-18T14:50:58.253 INFO:tasks.workunit.client.1.smithi100.stdout:2/134: mkdir d3/da/d32 0 2024-03-18T14:50:58.255 INFO:tasks.workunit.client.1.smithi100.stdout:6/148: creat d6/d7/df/f2a x:0 0 0 2024-03-18T14:50:58.256 INFO:tasks.workunit.client.0.smithi012.stdout:1/492: symlink d8/l96 0 2024-03-18T14:50:58.258 INFO:tasks.workunit.client.1.smithi100.stdout:8/172: creat d4/dc/dd/f34 x:0 0 0 2024-03-18T14:50:58.258 INFO:tasks.workunit.client.1.smithi100.stdout:8/173: chown d4/dc/f2d 710 1 2024-03-18T14:50:58.259 INFO:tasks.workunit.client.0.smithi012.stdout:8/724: truncate d0/d47/d59/d67/fda 1098237 0 2024-03-18T14:50:58.259 INFO:tasks.workunit.client.1.smithi100.stdout:7/151: mknod d5/d14/d16/c3a 0 2024-03-18T14:50:58.261 INFO:tasks.workunit.client.0.smithi012.stdout:1/493: dread d8/db/d10/d31/d45/f6d [0,4194304] 0 2024-03-18T14:50:58.261 INFO:tasks.workunit.client.0.smithi012.stdout:5/520: rename d3/d29/d36/d9b/f27 to d3/d84/fb2 0 2024-03-18T14:50:58.265 INFO:tasks.workunit.client.0.smithi012.stdout:0/523: dwrite d1/f10 [0,4194304] 0 2024-03-18T14:50:58.268 INFO:tasks.workunit.client.1.smithi100.stdout:9/185: getdents da 0 2024-03-18T14:50:58.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/525: rmdir dd/d13/d23/d33 39 2024-03-18T14:50:58.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/526: getdents dd/d13/d23/d76/d6f/db3 0 2024-03-18T14:50:58.269 INFO:tasks.workunit.client.1.smithi100.stdout:7/152: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:58.269 INFO:tasks.workunit.client.1.smithi100.stdout:6/149: dread f4 [0,4194304] 0 2024-03-18T14:50:58.274 INFO:tasks.workunit.client.0.smithi012.stdout:9/578: symlink d3/d9/d19/d66/d69/d7f/d8c/d8b/lb7 0 2024-03-18T14:50:58.275 INFO:tasks.workunit.client.1.smithi100.stdout:0/227: symlink dd/d39/d56/l58 0 2024-03-18T14:50:58.275 INFO:tasks.workunit.client.1.smithi100.stdout:0/228: dread - dd/d20/f2c zero size 2024-03-18T14:50:58.275 INFO:tasks.workunit.client.1.smithi100.stdout:0/229: truncate dd/d2b/f3d 673882 0 2024-03-18T14:50:58.276 INFO:tasks.workunit.client.0.smithi012.stdout:1/494: unlink d8/db/d10/d16/f5b 0 2024-03-18T14:50:58.279 INFO:tasks.workunit.client.0.smithi012.stdout:1/495: dread d8/f40 [0,4194304] 0 2024-03-18T14:50:58.280 INFO:tasks.workunit.client.1.smithi100.stdout:3/181: rename d3/d16/d13/d1f/d31 to d3/dd/d21/d3e 0 2024-03-18T14:50:58.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/521: truncate d3/d29/d36/d3d/d73/f58 3283433 0 2024-03-18T14:50:58.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/522: dread - d3/d29/d36/d3d/d65/da5/fae zero size 2024-03-18T14:50:58.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/523: chown d3/d29/d36/d3d/f7f 1003162 1 2024-03-18T14:50:58.281 INFO:tasks.workunit.client.0.smithi012.stdout:5/524: dread - d3/d29/d36/d3d/d65/da5/fae zero size 2024-03-18T14:50:58.283 INFO:tasks.workunit.client.1.smithi100.stdout:2/135: rmdir d3/da/d32 0 2024-03-18T14:50:58.286 INFO:tasks.workunit.client.1.smithi100.stdout:8/174: symlink d4/dc/d1f/d2b/l35 0 2024-03-18T14:50:58.286 INFO:tasks.workunit.client.1.smithi100.stdout:8/175: readlink d4/dc/dd/de/d19/l1d 0 2024-03-18T14:50:58.289 INFO:tasks.workunit.client.0.smithi012.stdout:0/524: mknod d1/d20/d23/d6f/cb5 0 2024-03-18T14:50:58.290 INFO:tasks.workunit.client.1.smithi100.stdout:6/150: truncate d6/fb 4160149 0 2024-03-18T14:50:58.292 INFO:tasks.workunit.client.0.smithi012.stdout:6/513: rename d17/d26/d36/d4d/f5c to d17/d18/f9e 0 2024-03-18T14:50:58.293 INFO:tasks.workunit.client.1.smithi100.stdout:0/230: rmdir dd/d2b 39 2024-03-18T14:50:58.294 INFO:tasks.workunit.client.1.smithi100.stdout:3/182: mkdir d3/d16/d13/d1f/d3f 0 2024-03-18T14:50:58.294 INFO:tasks.workunit.client.1.smithi100.stdout:3/183: truncate d3/dd/f24 312289 0 2024-03-18T14:50:58.296 INFO:tasks.workunit.client.0.smithi012.stdout:5/525: write d3/d29/d36/d3d/d73/f7d [2404172,15621] 0 2024-03-18T14:50:58.296 INFO:tasks.workunit.client.0.smithi012.stdout:5/526: chown d3/d84/la7 8820058 1 2024-03-18T14:50:58.297 INFO:tasks.workunit.client.0.smithi012.stdout:7/527: dwrite dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:58.297 INFO:tasks.workunit.client.0.smithi012.stdout:3/488: rename dc/d3c/d4c/d50 to dc/d3c/d59/d86/d9a 0 2024-03-18T14:50:58.297 INFO:tasks.workunit.client.0.smithi012.stdout:3/489: truncate dc/f92 494190 0 2024-03-18T14:50:58.297 INFO:tasks.workunit.client.0.smithi012.stdout:3/490: fsync dc/f48 0 2024-03-18T14:50:58.299 INFO:tasks.workunit.client.0.smithi012.stdout:9/579: dwrite d3/d49/f57 [0,4194304] 0 2024-03-18T14:50:58.299 INFO:tasks.workunit.client.0.smithi012.stdout:9/580: write d3/d9/d19/f4d [281513,18273] 0 2024-03-18T14:50:58.299 INFO:tasks.workunit.client.0.smithi012.stdout:9/581: readlink d3/d9/d19/d77/d52/d99/lab 0 2024-03-18T14:50:58.301 INFO:tasks.workunit.client.1.smithi100.stdout:7/153: rename d5/d22/d24/d37 to d5/d22/d3b 0 2024-03-18T14:50:58.301 INFO:tasks.workunit.client.1.smithi100.stdout:7/154: chown d5/l8 453639309 1 2024-03-18T14:50:58.301 INFO:tasks.workunit.client.1.smithi100.stdout:7/155: dread - d5/d22/d24/f2d zero size 2024-03-18T14:50:58.302 INFO:tasks.workunit.client.0.smithi012.stdout:6/514: unlink d17/d38/d51/d63/f6a 0 2024-03-18T14:50:58.302 INFO:tasks.workunit.client.0.smithi012.stdout:6/515: write d17/d38/d5b/f80 [204596,131035] 0 2024-03-18T14:50:58.305 INFO:tasks.workunit.client.0.smithi012.stdout:6/516: dread d17/d26/d36/d4d/f81 [0,4194304] 0 2024-03-18T14:50:58.306 INFO:tasks.workunit.client.1.smithi100.stdout:7/156: dread d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:50:58.307 INFO:tasks.workunit.client.1.smithi100.stdout:2/136: unlink d3/da/de/d20/f27 0 2024-03-18T14:50:58.309 INFO:tasks.workunit.client.0.smithi012.stdout:9/582: dread d3/d9/d19/d66/d21/f26 [0,4194304] 0 2024-03-18T14:50:58.310 INFO:tasks.workunit.client.1.smithi100.stdout:8/176: creat d4/dc/d1f/d33/f36 x:0 0 0 2024-03-18T14:50:58.312 INFO:tasks.workunit.client.0.smithi012.stdout:5/527: creat d3/d29/d36/fb3 x:0 0 0 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:9/186: mknod da/db/c38 0 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:9/187: fdatasync f9 0 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:9/188: dread - da/dd/d29/f2e zero size 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:6/151: mkdir d6/d10/d2b 0 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:6/152: chown d6/c1a 907 1 2024-03-18T14:50:58.317 INFO:tasks.workunit.client.1.smithi100.stdout:6/153: chown d6/d10/d27 0 1 2024-03-18T14:50:58.318 INFO:tasks.workunit.client.1.smithi100.stdout:8/177: dread d4/dc/dd/de/d21/d23/f27 [0,4194304] 0 2024-03-18T14:50:58.318 INFO:tasks.workunit.client.1.smithi100.stdout:8/178: chown d4/dc/c10 8 1 2024-03-18T14:50:58.319 INFO:tasks.workunit.client.1.smithi100.stdout:8/179: chown d4/dc/dd/de/f17 107990340 1 2024-03-18T14:50:58.319 INFO:tasks.workunit.client.0.smithi012.stdout:7/528: unlink dd/d13/f96 0 2024-03-18T14:50:58.321 INFO:tasks.workunit.client.0.smithi012.stdout:0/525: dwrite d1/d2d/f60 [0,4194304] 0 2024-03-18T14:50:58.321 INFO:tasks.workunit.client.0.smithi012.stdout:0/526: dread - d1/d20/d23/d3b/d33/d51/fae zero size 2024-03-18T14:50:58.321 INFO:tasks.workunit.client.1.smithi100.stdout:0/231: mkdir dd/d39/d56/d59 0 2024-03-18T14:50:58.321 INFO:tasks.workunit.client.1.smithi100.stdout:0/232: chown dd/f1f 3041 1 2024-03-18T14:50:58.322 INFO:tasks.workunit.client.1.smithi100.stdout:0/233: dread - dd/fe zero size 2024-03-18T14:50:58.323 INFO:tasks.workunit.client.0.smithi012.stdout:3/491: truncate dc/d30/f51 3022803 0 2024-03-18T14:50:58.323 INFO:tasks.workunit.client.0.smithi012.stdout:3/492: truncate dc/d3c/d4c/f94 619590 0 2024-03-18T14:50:58.324 INFO:tasks.workunit.client.1.smithi100.stdout:1/163: sync 2024-03-18T14:50:58.324 INFO:tasks.workunit.client.1.smithi100.stdout:4/177: sync 2024-03-18T14:50:58.324 INFO:tasks.workunit.client.1.smithi100.stdout:1/164: write d8/d1e/d11/f1b [1181625,40186] 0 2024-03-18T14:50:58.324 INFO:tasks.workunit.client.1.smithi100.stdout:1/165: fsync f7 0 2024-03-18T14:50:58.326 INFO:tasks.workunit.client.1.smithi100.stdout:1/166: dread d8/f29 [0,4194304] 0 2024-03-18T14:50:58.326 INFO:tasks.workunit.client.1.smithi100.stdout:7/157: dwrite d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:50:58.327 INFO:tasks.workunit.client.0.smithi012.stdout:3/493: dread dc/d3c/d72/f7b [0,4194304] 0 2024-03-18T14:50:58.330 INFO:tasks.workunit.client.0.smithi012.stdout:7/529: write dd/f5a [4217166,8921] 0 2024-03-18T14:50:58.337 INFO:tasks.workunit.client.0.smithi012.stdout:6/517: creat d17/d19/d27/d58/f9f x:0 0 0 2024-03-18T14:50:58.340 INFO:tasks.workunit.client.1.smithi100.stdout:1/167: read d8/d1e/d11/d2d/f14 [2042537,81664] 0 2024-03-18T14:50:58.340 INFO:tasks.workunit.client.0.smithi012.stdout:5/528: symlink d3/d29/d36/d3d/d73/d6b/d8f/lb4 0 2024-03-18T14:50:58.342 INFO:tasks.workunit.client.1.smithi100.stdout:9/189: mknod da/dd/c39 0 2024-03-18T14:50:58.342 INFO:tasks.workunit.client.1.smithi100.stdout:9/190: creat da/d25/f3a x:0 0 0 2024-03-18T14:50:58.342 INFO:tasks.workunit.client.1.smithi100.stdout:9/191: chown da/db/f1a 4906 1 2024-03-18T14:50:58.343 INFO:tasks.workunit.client.0.smithi012.stdout:0/527: creat d1/d20/d23/d7f/da7/fb6 x:0 0 0 2024-03-18T14:50:58.345 INFO:tasks.workunit.client.1.smithi100.stdout:8/180: mkdir d4/dc/dd/de/d19/d37 0 2024-03-18T14:50:58.346 INFO:tasks.workunit.client.0.smithi012.stdout:8/725: rename d0/d5a/d19/dad/d33 to d0/d5a/d5e/ddf 0 2024-03-18T14:50:58.347 INFO:tasks.workunit.client.1.smithi100.stdout:6/154: dwrite d6/d7/df/f1e [0,4194304] 0 2024-03-18T14:50:58.348 INFO:tasks.workunit.client.1.smithi100.stdout:5/148: sync 2024-03-18T14:50:58.349 INFO:tasks.workunit.client.1.smithi100.stdout:0/234: symlink dd/d2b/d54/l5a 0 2024-03-18T14:50:58.353 INFO:tasks.workunit.client.0.smithi012.stdout:3/494: dwrite dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:50:58.354 INFO:tasks.workunit.client.0.smithi012.stdout:7/530: unlink dd/d13/d23/d76/c6a 0 2024-03-18T14:50:58.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/178: link d9/c2a d9/d13/c37 0 2024-03-18T14:50:58.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/179: write d9/d13/f33 [990183,61809] 0 2024-03-18T14:50:58.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/180: chown d9/lf 0 1 2024-03-18T14:50:58.358 INFO:tasks.workunit.client.0.smithi012.stdout:6/518: creat d17/d19/d27/d58/d91/fa0 x:0 0 0 2024-03-18T14:50:58.360 INFO:tasks.workunit.client.0.smithi012.stdout:6/519: dread d17/d26/d36/d4d/f81 [0,4194304] 0 2024-03-18T14:50:58.361 INFO:tasks.workunit.client.0.smithi012.stdout:5/529: getdents d3/d29/d36/d9b 0 2024-03-18T14:50:58.363 INFO:tasks.workunit.client.0.smithi012.stdout:5/530: read d3/d29/f61 [459050,100666] 0 2024-03-18T14:50:58.365 INFO:tasks.workunit.client.0.smithi012.stdout:0/528: unlink d1/c5d 0 2024-03-18T14:50:58.365 INFO:tasks.workunit.client.0.smithi012.stdout:0/529: write d1/d20/d7b/ff [1683521,125014] 0 2024-03-18T14:50:58.366 INFO:tasks.workunit.client.0.smithi012.stdout:7/531: dread dd/d13/d23/d60/f62 [0,4194304] 0 2024-03-18T14:50:58.366 INFO:tasks.workunit.client.0.smithi012.stdout:7/532: write dd/d13/d23/f99 [1040749,6318] 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/158: creat d5/d30/f3c x:0 0 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/159: fsync d5/fa 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/160: write d5/f35 [174563,13205] 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/161: fsync d5/d14/d16/f1d 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/162: getdents d5/d22/d3b 0 2024-03-18T14:50:58.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/163: chown d5/d14/d16/f1a 426 1 2024-03-18T14:50:58.369 INFO:tasks.workunit.client.0.smithi012.stdout:1/496: rename d8/db/d10/f8b to d8/d1e/d27/d44/f97 0 2024-03-18T14:50:58.369 INFO:tasks.workunit.client.0.smithi012.stdout:1/497: chown d8/db/d10/d16 1177367709 1 2024-03-18T14:50:58.370 INFO:tasks.workunit.client.0.smithi012.stdout:8/726: creat d0/d47/d59/d67/fe0 x:0 0 0 2024-03-18T14:50:58.373 INFO:tasks.workunit.client.0.smithi012.stdout:3/495: creat dc/d3c/d59/d86/d9a/f9b x:0 0 0 2024-03-18T14:50:58.374 INFO:tasks.workunit.client.0.smithi012.stdout:1/498: write d8/db/d10/d31/d45/f77 [2800355,104993] 0 2024-03-18T14:50:58.375 INFO:tasks.workunit.client.0.smithi012.stdout:1/499: chown d8/db/d10/d16/d54 436992 1 2024-03-18T14:50:58.375 INFO:tasks.workunit.client.0.smithi012.stdout:1/500: read d8/db/d10/d31/d45/f6d [82364,68967] 0 2024-03-18T14:50:58.375 INFO:tasks.workunit.client.1.smithi100.stdout:6/155: dwrite d6/d12/f1f [0,4194304] 0 2024-03-18T14:50:58.376 INFO:tasks.workunit.client.1.smithi100.stdout:9/192: mkdir da/dd/d3b 0 2024-03-18T14:50:58.377 INFO:tasks.workunit.client.1.smithi100.stdout:8/181: dwrite d4/dc/dd/f34 [0,4194304] 0 2024-03-18T14:50:58.377 INFO:tasks.workunit.client.1.smithi100.stdout:7/164: dread d5/fd [0,4194304] 0 2024-03-18T14:50:58.378 INFO:tasks.workunit.client.0.smithi012.stdout:6/520: stat d17/d18/c65 0 2024-03-18T14:50:58.380 INFO:tasks.workunit.client.1.smithi100.stdout:5/149: write f15 [2302969,28495] 0 2024-03-18T14:50:58.382 INFO:tasks.workunit.client.1.smithi100.stdout:0/235: creat dd/f5b x:0 0 0 2024-03-18T14:50:58.383 INFO:tasks.workunit.client.0.smithi012.stdout:0/530: stat d1/d16/c4c 0 2024-03-18T14:50:58.383 INFO:tasks.workunit.client.0.smithi012.stdout:0/531: write d1/d16/fe [194782,130604] 0 2024-03-18T14:50:58.383 INFO:tasks.workunit.client.0.smithi012.stdout:0/532: dread - d1/d20/d23/d3b/d33/d64/f96 zero size 2024-03-18T14:50:58.383 INFO:tasks.workunit.client.0.smithi012.stdout:0/533: write d1/d20/d7b/f30 [771429,13419] 0 2024-03-18T14:50:58.384 INFO:tasks.workunit.client.0.smithi012.stdout:0/534: chown d1/d20/d23/d26/f40 2580 1 2024-03-18T14:50:58.384 INFO:tasks.workunit.client.0.smithi012.stdout:0/535: chown d1/f10 0 1 2024-03-18T14:50:58.386 INFO:tasks.workunit.client.1.smithi100.stdout:7/165: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:58.386 INFO:tasks.workunit.client.1.smithi100.stdout:7/166: write d5/d30/f3c [903063,10508] 0 2024-03-18T14:50:58.390 INFO:tasks.workunit.client.0.smithi012.stdout:9/583: rename d3/f3e to d3/d9/d19/d66/d69/d7f/d8c/dac/fb8 0 2024-03-18T14:50:58.392 INFO:tasks.workunit.client.0.smithi012.stdout:5/531: dwrite d3/d1e/fa4 [0,4194304] 0 2024-03-18T14:50:58.392 INFO:tasks.workunit.client.0.smithi012.stdout:5/532: dread - d3/d29/d36/d9b/d59/d8b/fa0 zero size 2024-03-18T14:50:58.393 INFO:tasks.workunit.client.0.smithi012.stdout:7/533: dwrite dd/d13/d23/d2f/d53/f90 [0,4194304] 0 2024-03-18T14:50:58.394 INFO:tasks.workunit.client.0.smithi012.stdout:7/534: fdatasync dd/d13/d23/f99 0 2024-03-18T14:50:58.396 INFO:tasks.workunit.client.0.smithi012.stdout:8/727: link d0/d5a/d5e/dd7/d84/d54/fb2 d0/d5a/d19/d63/db0/fe1 0 2024-03-18T14:50:58.396 INFO:tasks.workunit.client.1.smithi100.stdout:0/236: dread dd/f1c [0,4194304] 0 2024-03-18T14:50:58.398 INFO:tasks.workunit.client.1.smithi100.stdout:6/156: mkdir d6/d12/d2c 0 2024-03-18T14:50:58.398 INFO:tasks.workunit.client.1.smithi100.stdout:2/137: getdents d3/da/de/d20 0 2024-03-18T14:50:58.398 INFO:tasks.workunit.client.1.smithi100.stdout:6/157: chown d6/d10/d2b 951936140 1 2024-03-18T14:50:58.398 INFO:tasks.workunit.client.1.smithi100.stdout:6/158: truncate d6/fb 4166384 0 2024-03-18T14:50:58.400 INFO:tasks.workunit.client.1.smithi100.stdout:9/193: chown da/db/l17 357475171 1 2024-03-18T14:50:58.400 INFO:tasks.workunit.client.1.smithi100.stdout:9/194: write da/dd/d29/f2e [644325,89718] 0 2024-03-18T14:50:58.402 INFO:tasks.workunit.client.0.smithi012.stdout:1/501: creat d8/db/d10/d16/f98 x:0 0 0 2024-03-18T14:50:58.407 INFO:tasks.workunit.client.0.smithi012.stdout:6/521: dwrite d17/d19/d27/f68 [0,4194304] 0 2024-03-18T14:50:58.407 INFO:tasks.workunit.client.0.smithi012.stdout:0/536: dwrite d1/f5f [0,4194304] 0 2024-03-18T14:50:58.408 INFO:tasks.workunit.client.1.smithi100.stdout:8/182: rename d4/c1c to d4/dc/dd/c38 0 2024-03-18T14:50:58.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/138: read f0 [5654163,26321] 0 2024-03-18T14:50:58.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/139: truncate d3/d6/f9 1149787 0 2024-03-18T14:50:58.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/140: write d3/d1a/f2d [55008,98531] 0 2024-03-18T14:50:58.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/141: creat d3/da/f33 x:0 0 0 2024-03-18T14:50:58.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/142: dread d3/f1b [0,4194304] 0 2024-03-18T14:50:58.411 INFO:tasks.workunit.client.1.smithi100.stdout:5/150: write f26 [1048388,7581] 0 2024-03-18T14:50:58.411 INFO:tasks.workunit.client.1.smithi100.stdout:5/151: read - d29/f2c zero size 2024-03-18T14:50:58.413 INFO:tasks.workunit.client.1.smithi100.stdout:7/167: symlink d5/d14/d16/d2e/l3d 0 2024-03-18T14:50:58.420 INFO:tasks.workunit.client.1.smithi100.stdout:6/159: creat d6/d10/d2b/f2d x:0 0 0 2024-03-18T14:50:58.423 INFO:tasks.workunit.client.1.smithi100.stdout:9/195: truncate da/f1f 2780988 0 2024-03-18T14:50:58.423 INFO:tasks.workunit.client.1.smithi100.stdout:9/196: truncate da/d2c/d2f/f30 402726 0 2024-03-18T14:50:58.426 INFO:tasks.workunit.client.1.smithi100.stdout:4/181: getdents d9/d13 0 2024-03-18T14:50:58.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/533: mknod d3/d29/d36/d9b/cb5 0 2024-03-18T14:50:58.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/534: fsync d3/f9e 0 2024-03-18T14:50:58.436 INFO:tasks.workunit.client.1.smithi100.stdout:8/183: link d4/dc/dd/de/l29 d4/dc/d1f/l39 0 2024-03-18T14:50:58.436 INFO:tasks.workunit.client.1.smithi100.stdout:8/184: readlink d4/dc/dd/de/d21/d23/l2e 0 2024-03-18T14:50:58.437 INFO:tasks.workunit.client.1.smithi100.stdout:8/185: write f1 [4348536,95194] 0 2024-03-18T14:50:58.439 INFO:tasks.workunit.client.1.smithi100.stdout:4/182: dwrite d9/f10 [0,4194304] 0 2024-03-18T14:50:58.441 INFO:tasks.workunit.client.1.smithi100.stdout:2/143: symlink d3/da/de/d20/l34 0 2024-03-18T14:50:58.441 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: pgmap v246: 65 pgs: 65 active+clean; 985 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 81 MiB/s rd, 109 MiB/s wr, 538 op/s 2024-03-18T14:50:58.441 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr fail", "who": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: osdmap e46: 6 total, 6 up, 6 in 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "mgr fail", "who": "smithi012.tmscgt"}]': finished 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: mgrmap e21: smithi100.rwnrcn(active, starting, since 0.0310445s) 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:50:58.442 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:50:58.443 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: Manager daemon smithi100.rwnrcn is now available 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/trash_purge_schedule"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/trash_purge_schedule"}]: dispatch 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:50:58.444 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:58 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:50:58.445 INFO:tasks.workunit.client.0.smithi012.stdout:7/535: creat dd/d13/d23/d2f/fb9 x:0 0 0 2024-03-18T14:50:58.447 INFO:tasks.workunit.client.1.smithi100.stdout:7/168: link d5/d14/c27 d5/d14/d16/d2c/c3e 0 2024-03-18T14:50:58.449 INFO:tasks.workunit.client.1.smithi100.stdout:6/160: symlink d6/d7/df/d26/l2e 0 2024-03-18T14:50:58.450 INFO:tasks.workunit.client.1.smithi100.stdout:6/161: write f4 [2581772,58639] 0 2024-03-18T14:50:58.450 INFO:tasks.workunit.client.1.smithi100.stdout:4/183: write d9/f10 [1884279,19186] 0 2024-03-18T14:50:58.450 INFO:tasks.workunit.client.1.smithi100.stdout:6/162: creat d6/f2f x:0 0 0 2024-03-18T14:50:58.452 INFO:tasks.workunit.client.1.smithi100.stdout:9/197: mkdir da/dd/d3c 0 2024-03-18T14:50:58.457 INFO:tasks.workunit.client.1.smithi100.stdout:8/186: truncate d4/dc/dd/de/d19/f2f 1212800 0 2024-03-18T14:50:58.466 INFO:tasks.workunit.client.1.smithi100.stdout:2/144: dwrite d3/f10 [0,4194304] 0 2024-03-18T14:50:58.466 INFO:tasks.workunit.client.1.smithi100.stdout:7/169: dwrite d5/d14/d16/f1d [0,4194304] 0 2024-03-18T14:50:58.471 INFO:tasks.workunit.client.0.smithi012.stdout:8/728: rename d0/d5a/d5e/dd7/d84/d54/f9b to d0/fe2 0 2024-03-18T14:50:58.474 INFO:tasks.workunit.client.1.smithi100.stdout:8/187: creat d4/f3a x:0 0 0 2024-03-18T14:50:58.478 INFO:tasks.workunit.client.1.smithi100.stdout:7/170: dread d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:50:58.478 INFO:tasks.workunit.client.1.smithi100.stdout:7/171: chown d5/d22/d24 54 1 2024-03-18T14:50:58.480 INFO:tasks.workunit.client.1.smithi100.stdout:5/152: dwrite f23 [0,4194304] 0 2024-03-18T14:50:58.480 INFO:tasks.workunit.client.1.smithi100.stdout:9/198: dwrite da/fc [0,4194304] 0 2024-03-18T14:50:58.488 INFO:tasks.workunit.client.1.smithi100.stdout:2/145: rename d3/d1a/f21 to d3/d6/d29/f35 0 2024-03-18T14:50:58.488 INFO:tasks.workunit.client.1.smithi100.stdout:2/146: chown d3/d13/c1e 21010 1 2024-03-18T14:50:58.488 INFO:tasks.workunit.client.1.smithi100.stdout:2/147: chown d3/da/de/d20 38790704 1 2024-03-18T14:50:58.490 INFO:tasks.workunit.client.0.smithi012.stdout:3/496: rmdir dc 39 2024-03-18T14:50:58.492 INFO:tasks.workunit.client.1.smithi100.stdout:8/188: dwrite d4/dc/dd/f34 [0,4194304] 0 2024-03-18T14:50:58.492 INFO:tasks.workunit.client.1.smithi100.stdout:8/189: write d4/dc/dd/de/d21/d23/f28 [169888,10729] 0 2024-03-18T14:50:58.492 INFO:tasks.workunit.client.1.smithi100.stdout:8/190: chown d4/f6 1231934556 1 2024-03-18T14:50:58.493 INFO:tasks.workunit.client.1.smithi100.stdout:8/191: chown d4/dc/d1f 99137 1 2024-03-18T14:50:58.497 INFO:tasks.workunit.client.0.smithi012.stdout:6/522: symlink d17/d38/d3f/la1 0 2024-03-18T14:50:58.499 INFO:tasks.workunit.client.1.smithi100.stdout:7/172: creat d5/f3f x:0 0 0 2024-03-18T14:50:58.500 INFO:tasks.workunit.client.0.smithi012.stdout:0/537: symlink d1/lb7 0 2024-03-18T14:50:58.500 INFO:tasks.workunit.client.0.smithi012.stdout:0/538: dread - d1/d20/d23/d7f/f97 zero size 2024-03-18T14:50:58.505 INFO:tasks.workunit.client.1.smithi100.stdout:5/153: unlink d29/d2d/c30 0 2024-03-18T14:50:58.505 INFO:tasks.workunit.client.1.smithi100.stdout:6/163: fsync d6/f2f 0 2024-03-18T14:50:58.505 INFO:tasks.workunit.client.1.smithi100.stdout:6/164: chown d6/f15 30658913 1 2024-03-18T14:50:58.506 INFO:tasks.workunit.client.0.smithi012.stdout:3/497: dread dc/d3c/d59/d86/d9a/d54/f97 [0,4194304] 0 2024-03-18T14:50:58.506 INFO:tasks.workunit.client.1.smithi100.stdout:7/173: dread d5/d14/f1e [0,4194304] 0 2024-03-18T14:50:58.506 INFO:tasks.workunit.client.1.smithi100.stdout:9/199: rmdir da/dd 39 2024-03-18T14:50:58.509 INFO:tasks.workunit.client.1.smithi100.stdout:2/148: dwrite d3/f1b [0,4194304] 0 2024-03-18T14:50:58.511 INFO:tasks.workunit.client.1.smithi100.stdout:5/154: truncate f2 682403 0 2024-03-18T14:50:58.513 INFO:tasks.workunit.client.1.smithi100.stdout:8/192: dwrite d4/f3a [0,4194304] 0 2024-03-18T14:50:58.515 INFO:tasks.workunit.client.1.smithi100.stdout:9/200: symlink da/d2c/d2f/l3d 0 2024-03-18T14:50:58.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/174: creat d5/d14/f40 x:0 0 0 2024-03-18T14:50:58.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/175: chown d5/d14/c27 16292 1 2024-03-18T14:50:58.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/176: chown d5/d30/f3c 611991 1 2024-03-18T14:50:58.520 INFO:tasks.workunit.client.0.smithi012.stdout:6/523: dwrite d17/f35 [0,4194304] 0 2024-03-18T14:50:58.521 INFO:tasks.workunit.client.1.smithi100.stdout:2/149: mkdir d3/d6/d29/d36 0 2024-03-18T14:50:58.522 INFO:tasks.workunit.client.1.smithi100.stdout:7/177: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:58.524 INFO:tasks.workunit.client.1.smithi100.stdout:5/155: dread - f17 zero size 2024-03-18T14:50:58.525 INFO:tasks.workunit.client.1.smithi100.stdout:8/193: read d4/fa [3341302,93712] 0 2024-03-18T14:50:58.526 INFO:tasks.workunit.client.0.smithi012.stdout:9/584: rmdir d3 39 2024-03-18T14:50:58.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/535: mkdir d3/db6 0 2024-03-18T14:50:58.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/536: fsync d3/d84/f85 0 2024-03-18T14:50:58.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/537: fsync d3/d1e/d33/f74 0 2024-03-18T14:50:58.531 INFO:tasks.workunit.client.0.smithi012.stdout:7/536: symlink dd/d13/d23/d2f/d7a/dab/lba 0 2024-03-18T14:50:58.532 INFO:tasks.workunit.client.0.smithi012.stdout:9/585: dread d3/d9/d19/d66/d21/d35/d48/f67 [0,4194304] 0 2024-03-18T14:50:58.535 INFO:tasks.workunit.client.0.smithi012.stdout:8/729: unlink d0/d5a/d5e/dd7/d84/d54/d62/daf/ld2 0 2024-03-18T14:50:58.537 INFO:tasks.workunit.client.0.smithi012.stdout:0/539: rename d1/d20/f5a to d1/d20/d23/d82/fb8 0 2024-03-18T14:50:58.540 INFO:tasks.workunit.client.0.smithi012.stdout:3/498: creat dc/d30/f9c x:0 0 0 2024-03-18T14:50:58.540 INFO:tasks.workunit.client.1.smithi100.stdout:7/178: dread d5/d14/d16/f2b [0,4194304] 0 2024-03-18T14:50:58.540 INFO:tasks.workunit.client.1.smithi100.stdout:7/179: dread - d5/d14/f40 zero size 2024-03-18T14:50:58.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: pgmap v246: 65 pgs: 65 active+clean; 985 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 81 MiB/s rd, 109 MiB/s wr, 538 op/s 2024-03-18T14:50:58.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:50:58.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr fail", "who": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: osdmap e46: 6 total, 6 up, 6 in 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.14220 172.21.15.12:0/3979957923' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "mgr fail", "who": "smithi012.tmscgt"}]': finished 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: mgrmap e21: smithi100.rwnrcn(active, starting, since 0.0310445s) 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:50:58.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: Manager daemon smithi100.rwnrcn is now available 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:50:58.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:50:58.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:50:58.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/trash_purge_schedule"}]: dispatch 2024-03-18T14:50:58.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi100.rwnrcn/trash_purge_schedule"}]: dispatch 2024-03-18T14:50:58.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:50:58.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:58 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:50:58.548 INFO:tasks.workunit.client.1.smithi100.stdout:9/201: dwrite da/d2c/d2f/f31 [0,4194304] 0 2024-03-18T14:50:58.549 INFO:tasks.workunit.client.0.smithi012.stdout:6/524: dwrite d17/d19/f3e [0,4194304] 0 2024-03-18T14:50:58.549 INFO:tasks.workunit.client.0.smithi012.stdout:3/499: write dc/d30/f51 [343480,85063] 0 2024-03-18T14:50:58.549 INFO:tasks.workunit.client.0.smithi012.stdout:3/500: chown dc/d30/f3f 501327178 1 2024-03-18T14:50:58.549 INFO:tasks.workunit.client.0.smithi012.stdout:5/538: link d3/d1e/f93 d3/db6/fb7 0 2024-03-18T14:50:58.549 INFO:tasks.workunit.client.0.smithi012.stdout:0/540: read d1/d20/d4e/f52 [3879353,68952] 0 2024-03-18T14:50:58.551 INFO:tasks.workunit.client.1.smithi100.stdout:6/165: rmdir d6/d7/df 39 2024-03-18T14:50:58.555 INFO:tasks.workunit.client.0.smithi012.stdout:7/537: dwrite dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:50:58.555 INFO:tasks.workunit.client.0.smithi012.stdout:9/586: creat d3/d9/d19/d66/d21/d35/fb9 x:0 0 0 2024-03-18T14:50:58.557 INFO:tasks.workunit.client.1.smithi100.stdout:2/150: creat d3/d13/f37 x:0 0 0 2024-03-18T14:50:58.557 INFO:tasks.workunit.client.1.smithi100.stdout:2/151: chown d3/da/c2e 2261 1 2024-03-18T14:50:58.559 INFO:tasks.workunit.client.0.smithi012.stdout:5/539: write d3/d29/d36/d3d/d73/f58 [620793,17490] 0 2024-03-18T14:50:58.559 INFO:tasks.workunit.client.0.smithi012.stdout:5/540: dread - d3/d15/f90 zero size 2024-03-18T14:50:58.559 INFO:tasks.workunit.client.0.smithi012.stdout:5/541: chown d3/d15/d5f 55989958 1 2024-03-18T14:50:58.560 INFO:tasks.workunit.client.0.smithi012.stdout:8/730: creat d0/d5a/d55/fe3 x:0 0 0 2024-03-18T14:50:58.561 INFO:tasks.workunit.client.1.smithi100.stdout:5/156: mkdir d29/d2d/d32 0 2024-03-18T14:50:58.564 INFO:tasks.workunit.client.1.smithi100.stdout:8/194: rename d4/dc/dd/l1a to d4/dc/dd/de/d21/d23/l3b 0 2024-03-18T14:50:58.565 INFO:tasks.workunit.client.1.smithi100.stdout:8/195: chown d4/dc/d31 1510123 1 2024-03-18T14:50:58.567 INFO:tasks.workunit.client.0.smithi012.stdout:6/525: creat d17/d38/fa2 x:0 0 0 2024-03-18T14:50:58.568 INFO:tasks.workunit.client.0.smithi012.stdout:7/538: dread dd/f5a [0,4194304] 0 2024-03-18T14:50:58.570 INFO:tasks.workunit.client.0.smithi012.stdout:3/501: mknod dc/d3c/c9d 0 2024-03-18T14:50:58.571 INFO:tasks.workunit.client.1.smithi100.stdout:3/184: sync 2024-03-18T14:50:58.571 INFO:tasks.workunit.client.1.smithi100.stdout:3/185: creat d3/d16/d13/d17/f40 x:0 0 0 2024-03-18T14:50:58.571 INFO:tasks.workunit.client.1.smithi100.stdout:9/202: dwrite da/d25/f35 [0,4194304] 0 2024-03-18T14:50:58.571 INFO:tasks.workunit.client.1.smithi100.stdout:6/166: creat d6/d12/d2c/f30 x:0 0 0 2024-03-18T14:50:58.573 INFO:tasks.workunit.client.0.smithi012.stdout:0/541: dwrite d1/d20/d23/d6f/fa5 [0,4194304] 0 2024-03-18T14:50:58.573 INFO:tasks.workunit.client.1.smithi100.stdout:2/152: unlink d3/d6/l2a 0 2024-03-18T14:50:58.573 INFO:tasks.workunit.client.0.smithi012.stdout:8/731: dread d0/d5a/f60 [0,4194304] 0 2024-03-18T14:50:58.574 INFO:tasks.workunit.client.0.smithi012.stdout:0/542: read d1/d16/d19/fb3 [37926,112587] 0 2024-03-18T14:50:58.574 INFO:tasks.workunit.client.0.smithi012.stdout:9/587: creat d3/d9/d19/d66/fba x:0 0 0 2024-03-18T14:50:58.574 INFO:tasks.workunit.client.0.smithi012.stdout:0/543: write d1/d20/d23/d26/fa4 [614668,125027] 0 2024-03-18T14:50:58.574 INFO:tasks.workunit.client.0.smithi012.stdout:0/544: write d1/d16/d79/f8f [1006550,91416] 0 2024-03-18T14:50:58.575 INFO:tasks.workunit.client.1.smithi100.stdout:5/157: stat le 0 2024-03-18T14:50:58.580 INFO:tasks.workunit.client.1.smithi100.stdout:7/180: rename d5/d14/f18 to d5/d14/d16/f41 0 2024-03-18T14:50:58.580 INFO:tasks.workunit.client.1.smithi100.stdout:7/181: getdents d5/d22/d3b 0 2024-03-18T14:50:58.583 INFO:tasks.workunit.client.0.smithi012.stdout:5/542: rename d3/d29/d36/l91 to d3/d15/lb8 0 2024-03-18T14:50:58.586 INFO:tasks.workunit.client.0.smithi012.stdout:6/526: mkdir d17/d26/d36/d4d/da3 0 2024-03-18T14:50:58.586 INFO:tasks.workunit.client.0.smithi012.stdout:6/527: write f16 [4773731,59816] 0 2024-03-18T14:50:58.586 INFO:tasks.workunit.client.0.smithi012.stdout:6/528: chown d17/d38/d51 5579737 1 2024-03-18T14:50:58.588 INFO:tasks.workunit.client.1.smithi100.stdout:3/186: symlink d3/d16/d13/l41 0 2024-03-18T14:50:58.589 INFO:tasks.workunit.client.0.smithi012.stdout:3/502: mknod dc/d30/d35/c9e 0 2024-03-18T14:50:58.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/196: dwrite d4/dc/dd/de/d19/f1e [0,4194304] 0 2024-03-18T14:50:58.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/197: read - d4/dc/d1f/d33/f36 zero size 2024-03-18T14:50:58.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/198: chown d4/dc/dd/de/d19/d2c 25806 1 2024-03-18T14:50:58.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/199: readlink d4/l5 0 2024-03-18T14:50:58.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/200: stat d4/dc/d1f/d33 0 2024-03-18T14:50:58.595 INFO:tasks.workunit.client.1.smithi100.stdout:1/168: sync 2024-03-18T14:50:58.595 INFO:tasks.workunit.client.1.smithi100.stdout:4/184: sync 2024-03-18T14:50:58.595 INFO:tasks.workunit.client.1.smithi100.stdout:4/185: stat l6 0 2024-03-18T14:50:58.595 INFO:tasks.workunit.client.1.smithi100.stdout:0/237: sync 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.1.smithi100.stdout:9/203: mkdir da/dd/d3e 0 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.1.smithi100.stdout:9/204: fdatasync da/f1d 0 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.1.smithi100.stdout:9/205: chown da/db 53320 1 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.1.smithi100.stdout:9/206: stat da/f20 0 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.1.smithi100.stdout:9/207: chown da/d25/f3a 865919 1 2024-03-18T14:50:58.597 INFO:tasks.workunit.client.0.smithi012.stdout:8/732: creat d0/d5a/d5e/ddf/d50/d6a/fe4 x:0 0 0 2024-03-18T14:50:58.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/167: link d6/d10/d27/c23 d6/d7/df/d26/c31 0 2024-03-18T14:50:58.603 INFO:tasks.workunit.client.0.smithi012.stdout:9/588: truncate d3/f3b 44454 0 2024-03-18T14:50:58.604 INFO:tasks.workunit.client.1.smithi100.stdout:2/153: truncate d3/da/de/f28 1965849 0 2024-03-18T14:50:58.605 INFO:tasks.workunit.client.0.smithi012.stdout:3/503: write dc/d30/d35/f2c [3016352,52806] 0 2024-03-18T14:50:58.605 INFO:tasks.workunit.client.1.smithi100.stdout:8/201: dread d4/dc/dd/f34 [0,4194304] 0 2024-03-18T14:50:58.606 INFO:tasks.workunit.client.0.smithi012.stdout:7/539: rename dd/d13/d23/d2f/d7a/c8c to dd/d13/d23/d33/d37/cbb 0 2024-03-18T14:50:58.607 INFO:tasks.workunit.client.1.smithi100.stdout:5/158: truncate f18 5125952 0 2024-03-18T14:50:58.608 INFO:tasks.workunit.client.0.smithi012.stdout:0/545: dwrite d1/d16/d19/fb3 [0,4194304] 0 2024-03-18T14:50:58.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/182: dwrite d5/d14/f1e [0,4194304] 0 2024-03-18T14:50:58.613 INFO:tasks.workunit.client.1.smithi100.stdout:1/169: getdents d8/d1e/d11/d2d 0 2024-03-18T14:50:58.614 INFO:tasks.workunit.client.0.smithi012.stdout:2/553: sync 2024-03-18T14:50:58.614 INFO:tasks.workunit.client.0.smithi012.stdout:4/476: sync 2024-03-18T14:50:58.616 INFO:tasks.workunit.client.0.smithi012.stdout:8/733: dwrite d0/d5a/d5e/dd7/d84/d2a/f5d [4194304,4194304] 0 2024-03-18T14:50:58.617 INFO:tasks.workunit.client.0.smithi012.stdout:8/734: dread - d0/d5a/d19/d63/db0/fe1 zero size 2024-03-18T14:50:58.617 INFO:tasks.workunit.client.1.smithi100.stdout:4/186: creat d9/d13/d25/f38 x:0 0 0 2024-03-18T14:50:58.619 INFO:tasks.workunit.client.1.smithi100.stdout:3/187: dwrite d3/d16/d13/d17/f2b [0,4194304] 0 2024-03-18T14:50:58.619 INFO:tasks.workunit.client.1.smithi100.stdout:3/188: stat d3/d16/l18 0 2024-03-18T14:50:58.620 INFO:tasks.workunit.client.1.smithi100.stdout:0/238: rename dd/f26 to dd/d20/f5c 0 2024-03-18T14:50:58.620 INFO:tasks.workunit.client.1.smithi100.stdout:0/239: readlink dd/d2b/l4f 0 2024-03-18T14:50:58.620 INFO:tasks.workunit.client.1.smithi100.stdout:0/240: readlink dd/d2b/l4f 0 2024-03-18T14:50:58.620 INFO:tasks.workunit.client.1.smithi100.stdout:0/241: creat dd/f5d x:0 0 0 2024-03-18T14:50:58.621 INFO:tasks.workunit.client.0.smithi012.stdout:9/589: symlink d3/d9/d19/d66/d69/d7f/lbb 0 2024-03-18T14:50:58.622 INFO:tasks.workunit.client.1.smithi100.stdout:6/168: getdents d6/d10/d27 0 2024-03-18T14:50:58.622 INFO:tasks.workunit.client.1.smithi100.stdout:6/169: stat d6/f15 0 2024-03-18T14:50:58.626 INFO:tasks.workunit.client.0.smithi012.stdout:3/504: unlink dc/d3c/d4c/d5f/l82 0 2024-03-18T14:50:58.627 INFO:tasks.workunit.client.0.smithi012.stdout:7/540: mkdir dd/d13/d23/d2f/d7a/dab/dbc 0 2024-03-18T14:50:58.627 INFO:tasks.workunit.client.1.smithi100.stdout:3/189: dread d3/d16/d13/d17/f3c [0,4194304] 0 2024-03-18T14:50:58.628 INFO:tasks.workunit.client.1.smithi100.stdout:5/159: dwrite f18 [0,4194304] 0 2024-03-18T14:50:58.630 INFO:tasks.workunit.client.1.smithi100.stdout:1/170: symlink d8/d1e/d11/d27/l2e 0 2024-03-18T14:50:58.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/154: fsync d3/da/de/f28 0 2024-03-18T14:50:58.632 INFO:tasks.workunit.client.1.smithi100.stdout:4/187: mkdir d9/d13/d39 0 2024-03-18T14:50:58.632 INFO:tasks.workunit.client.1.smithi100.stdout:4/188: chown d9/d13/d25/f38 217 1 2024-03-18T14:50:58.633 INFO:tasks.workunit.client.0.smithi012.stdout:0/546: unlink d1/c6 0 2024-03-18T14:50:58.634 INFO:tasks.workunit.client.1.smithi100.stdout:3/190: write d3/d16/f14 [3852296,110352] 0 2024-03-18T14:50:58.634 INFO:tasks.workunit.client.1.smithi100.stdout:3/191: truncate d3/ff 1274818 0 2024-03-18T14:50:58.636 INFO:tasks.workunit.client.0.smithi012.stdout:2/554: link d0/d3c/d40/d47/c65 d0/da2/cad 0 2024-03-18T14:50:58.636 INFO:tasks.workunit.client.0.smithi012.stdout:2/555: readlink d0/d10/d18/d33/d71/d90/l93 0 2024-03-18T14:50:58.638 INFO:tasks.workunit.client.0.smithi012.stdout:4/477: rename d6/da/f82 to d6/d1a/d64/f9c 0 2024-03-18T14:50:58.639 INFO:tasks.workunit.client.1.smithi100.stdout:9/208: rename da/f20 to da/d25/f3f 0 2024-03-18T14:50:58.640 INFO:tasks.workunit.client.0.smithi012.stdout:3/505: mknod dc/d3c/d59/d86/d9a/d54/c9f 0 2024-03-18T14:50:58.644 INFO:tasks.workunit.client.1.smithi100.stdout:0/242: creat dd/d20/d44/f5e x:0 0 0 2024-03-18T14:50:58.646 INFO:tasks.workunit.client.1.smithi100.stdout:6/170: mkdir d6/d7/df/d26/d32 0 2024-03-18T14:50:58.648 INFO:tasks.workunit.client.0.smithi012.stdout:0/547: creat d1/d20/d4e/d54/d8a/fb9 x:0 0 0 2024-03-18T14:50:58.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/556: creat d0/d10/d18/d33/d71/d90/fae x:0 0 0 2024-03-18T14:50:58.651 INFO:tasks.workunit.client.1.smithi100.stdout:5/160: mknod d29/d2d/c33 0 2024-03-18T14:50:58.651 INFO:tasks.workunit.client.1.smithi100.stdout:5/161: creat d29/f34 x:0 0 0 2024-03-18T14:50:58.653 INFO:tasks.workunit.client.0.smithi012.stdout:9/590: rename d3/d9/d19/d66/d21/d35/d48/f67 to d3/d9/d19/d66/d6e/fbc 0 2024-03-18T14:50:58.654 INFO:tasks.workunit.client.1.smithi100.stdout:1/171: mknod d8/d1e/d11/d27/c2f 0 2024-03-18T14:50:58.654 INFO:tasks.workunit.client.1.smithi100.stdout:1/172: write f5 [1251957,43319] 0 2024-03-18T14:50:58.654 INFO:tasks.workunit.client.1.smithi100.stdout:9/209: dwrite da/dd/f28 [4194304,4194304] 0 2024-03-18T14:50:58.655 INFO:tasks.workunit.client.1.smithi100.stdout:9/210: chown da/db/f36 859524000 1 2024-03-18T14:50:58.655 INFO:tasks.workunit.client.0.smithi012.stdout:4/478: rename d6/d1a/d30/c86 to d6/d1a/d30/c9d 0 2024-03-18T14:50:58.655 INFO:tasks.workunit.client.0.smithi012.stdout:4/479: chown d6/da/d58 925000 1 2024-03-18T14:50:58.657 INFO:tasks.workunit.client.1.smithi100.stdout:4/189: rmdir d9/d17 39 2024-03-18T14:50:58.658 INFO:tasks.workunit.client.1.smithi100.stdout:4/190: dread d9/d17/f24 [0,4194304] 0 2024-03-18T14:50:58.658 INFO:tasks.workunit.client.1.smithi100.stdout:4/191: readlink l6 0 2024-03-18T14:50:58.658 INFO:tasks.workunit.client.1.smithi100.stdout:4/192: read - d9/d13/d25/f38 zero size 2024-03-18T14:50:58.661 INFO:tasks.workunit.client.0.smithi012.stdout:2/557: dread d0/f6 [0,4194304] 0 2024-03-18T14:50:58.664 INFO:tasks.workunit.client.0.smithi012.stdout:3/506: mkdir dc/d30/d3a/da0 0 2024-03-18T14:50:58.666 INFO:tasks.workunit.client.1.smithi100.stdout:3/192: mknod d3/d16/d13/d17/c42 0 2024-03-18T14:50:58.666 INFO:tasks.workunit.client.1.smithi100.stdout:3/193: chown d3/dd/f28 5629737 1 2024-03-18T14:50:58.668 INFO:tasks.workunit.client.0.smithi012.stdout:7/541: rename dd/d13/d23/d33/l46 to dd/d13/d23/lbd 0 2024-03-18T14:50:58.670 INFO:tasks.workunit.client.1.smithi100.stdout:8/202: rename d4/dc/d1f/d2b to d4/d3c 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/558: creat d0/d10/d18/d57/faf x:0 0 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/559: truncate d0/d10/d18/d33/d71/d90/fae 794778 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/560: write d0/f97 [367325,67689] 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/561: write d0/d10/d18/d33/f62 [604710,110219] 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/562: readlink d0/d6b/l70 0 2024-03-18T14:50:58.671 INFO:tasks.workunit.client.0.smithi012.stdout:2/563: write d0/d10/d18/f1b [5214052,104068] 0 2024-03-18T14:50:58.672 INFO:tasks.workunit.client.1.smithi100.stdout:0/243: creat dd/d20/f5f x:0 0 0 2024-03-18T14:50:58.673 INFO:tasks.workunit.client.0.smithi012.stdout:3/507: creat dc/d3c/d4c/d57/fa1 x:0 0 0 2024-03-18T14:50:58.673 INFO:tasks.workunit.client.0.smithi012.stdout:3/508: write dc/d30/f99 [824585,82746] 0 2024-03-18T14:50:58.674 INFO:tasks.workunit.client.1.smithi100.stdout:6/171: truncate d6/d7/df/f1e 239113 0 2024-03-18T14:50:58.679 INFO:tasks.workunit.client.0.smithi012.stdout:9/591: dwrite d3/d9/d19/d66/f45 [0,4194304] 0 2024-03-18T14:50:58.679 INFO:tasks.workunit.client.0.smithi012.stdout:4/480: dwrite d6/d19/d39/f41 [0,4194304] 0 2024-03-18T14:50:58.679 INFO:tasks.workunit.client.0.smithi012.stdout:7/542: read dd/d13/d23/d33/f84 [3915327,6518] 0 2024-03-18T14:50:58.679 INFO:tasks.workunit.client.0.smithi012.stdout:9/592: readlink d3/d9/d19/d66/d21/d35/l68 0 2024-03-18T14:50:58.680 INFO:tasks.workunit.client.0.smithi012.stdout:2/564: symlink d0/d10/d18/d33/d71/lb0 0 2024-03-18T14:50:58.680 INFO:tasks.workunit.client.0.smithi012.stdout:2/565: write d0/da2/f67 [3433219,100954] 0 2024-03-18T14:50:58.680 INFO:tasks.workunit.client.0.smithi012.stdout:2/566: creat d0/d10/d18/d4f/fb1 x:0 0 0 2024-03-18T14:50:58.680 INFO:tasks.workunit.client.0.smithi012.stdout:2/567: creat d0/d6c/fb2 x:0 0 0 2024-03-18T14:50:58.681 INFO:tasks.workunit.client.1.smithi100.stdout:5/162: link f18 d29/d2d/d32/f35 0 2024-03-18T14:50:58.682 INFO:tasks.workunit.client.0.smithi012.stdout:2/568: dread d0/f79 [0,4194304] 0 2024-03-18T14:50:58.682 INFO:tasks.workunit.client.0.smithi012.stdout:3/509: symlink dc/d91/la2 0 2024-03-18T14:50:58.682 INFO:tasks.workunit.client.0.smithi012.stdout:3/510: stat dc/d30/d35/c53 0 2024-03-18T14:50:58.682 INFO:tasks.workunit.client.0.smithi012.stdout:2/569: fdatasync d0/f6 0 2024-03-18T14:50:58.683 INFO:tasks.workunit.client.0.smithi012.stdout:3/511: readlink dc/d3c/d59/d86/d9a/l78 0 2024-03-18T14:50:58.683 INFO:tasks.workunit.client.0.smithi012.stdout:2/570: write d0/d10/d18/d33/d71/d90/fae [1079924,31631] 0 2024-03-18T14:50:58.683 INFO:tasks.workunit.client.0.smithi012.stdout:2/571: write d0/d10/d18/d4f/fb1 [887846,65995] 0 2024-03-18T14:50:58.683 INFO:tasks.workunit.client.0.smithi012.stdout:3/512: dread dc/d30/f6d [0,4194304] 0 2024-03-18T14:50:58.686 INFO:tasks.workunit.client.1.smithi100.stdout:1/173: mkdir d8/d30 0 2024-03-18T14:50:58.686 INFO:tasks.workunit.client.1.smithi100.stdout:1/174: fdatasync f5 0 2024-03-18T14:50:58.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/481: dread d6/f33 [0,4194304] 0 2024-03-18T14:50:58.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/482: creat d6/da/d58/f9e x:0 0 0 2024-03-18T14:50:58.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/211: unlink da/db/l17 0 2024-03-18T14:50:58.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/212: chown da/l1b 1088 1 2024-03-18T14:50:58.692 INFO:tasks.workunit.client.0.smithi012.stdout:9/593: mkdir d3/d9/d5c/dbd 0 2024-03-18T14:50:58.693 INFO:tasks.workunit.client.0.smithi012.stdout:7/543: creat dd/d13/d23/d2f/d7a/dab/fbe x:0 0 0 2024-03-18T14:50:58.699 INFO:tasks.workunit.client.1.smithi100.stdout:9/213: read da/d2c/d2f/f31 [3368299,53901] 0 2024-03-18T14:50:58.702 INFO:tasks.workunit.client.0.smithi012.stdout:2/572: creat d0/d3c/fb3 x:0 0 0 2024-03-18T14:50:58.707 INFO:tasks.workunit.client.1.smithi100.stdout:7/183: rename d5/d34/c38 to d5/d22/d24/d39/c42 0 2024-03-18T14:50:58.710 INFO:tasks.workunit.client.1.smithi100.stdout:8/203: creat d4/dc/d31/f3d x:0 0 0 2024-03-18T14:50:58.710 INFO:tasks.workunit.client.1.smithi100.stdout:8/204: write d4/ff [9341828,59782] 0 2024-03-18T14:50:58.711 INFO:tasks.workunit.client.1.smithi100.stdout:8/205: read d4/dc/dd/de/f17 [159575,36943] 0 2024-03-18T14:50:58.711 INFO:tasks.workunit.client.1.smithi100.stdout:8/206: truncate d4/f15 79479 0 2024-03-18T14:50:58.712 INFO:tasks.workunit.client.0.smithi012.stdout:4/483: truncate d6/d27/f67 1599647 0 2024-03-18T14:50:58.712 INFO:tasks.workunit.client.0.smithi012.stdout:4/484: dread - d6/d19/d6c/f7a zero size 2024-03-18T14:50:58.714 INFO:tasks.workunit.client.1.smithi100.stdout:6/172: link d6/f2f d6/d7/df/d26/d32/f33 0 2024-03-18T14:50:58.714 INFO:tasks.workunit.client.1.smithi100.stdout:8/207: dread d4/dc/dd/de/d21/d23/f28 [0,4194304] 0 2024-03-18T14:50:58.716 INFO:tasks.workunit.client.0.smithi012.stdout:9/594: symlink d3/d9/d19/d66/d69/d7f/d8c/lbe 0 2024-03-18T14:50:58.718 INFO:tasks.workunit.client.1.smithi100.stdout:5/163: mknod d29/d2d/d32/c36 0 2024-03-18T14:50:58.724 INFO:tasks.workunit.client.1.smithi100.stdout:7/184: dwrite d5/f35 [0,4194304] 0 2024-03-18T14:50:58.724 INFO:tasks.workunit.client.1.smithi100.stdout:7/185: stat l2 0 2024-03-18T14:50:58.727 INFO:tasks.workunit.client.1.smithi100.stdout:7/186: write d5/d14/d16/f1a [2746511,109720] 0 2024-03-18T14:50:58.727 INFO:tasks.workunit.client.1.smithi100.stdout:7/187: fsync d5/d22/d24/f2d 0 2024-03-18T14:50:58.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/513: getdents dc/d30 0 2024-03-18T14:50:58.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/514: write f6 [3358957,15278] 0 2024-03-18T14:50:58.729 INFO:tasks.workunit.client.0.smithi012.stdout:3/515: truncate dc/d3c/d59/d86/f96 209364 0 2024-03-18T14:50:58.729 INFO:tasks.workunit.client.1.smithi100.stdout:3/194: rmdir d3 39 2024-03-18T14:50:58.729 INFO:tasks.workunit.client.0.smithi012.stdout:4/485: symlink d6/d27/l9f 0 2024-03-18T14:50:58.736 INFO:tasks.workunit.client.0.smithi012.stdout:3/516: write dc/d30/d35/f2e [2246284,56162] 0 2024-03-18T14:50:58.774 INFO:tasks.workunit.client.1.smithi100.stdout:9/214: mknod da/dd/c40 0 2024-03-18T14:50:58.777 INFO:tasks.workunit.client.1.smithi100.stdout:2/155: rename d3/da/f2c to d3/d13/f38 0 2024-03-18T14:50:58.778 INFO:tasks.workunit.client.0.smithi012.stdout:2/573: getdents d0/d3c/d40/d47/d85 0 2024-03-18T14:50:58.778 INFO:tasks.workunit.client.0.smithi012.stdout:2/574: write d0/d3c/fb3 [115758,59609] 0 2024-03-18T14:50:58.778 INFO:tasks.workunit.client.0.smithi012.stdout:2/575: dread - d0/d3c/d40/d47/d85/f94 zero size 2024-03-18T14:50:58.778 INFO:tasks.workunit.client.0.smithi012.stdout:2/576: stat d0/l30 0 2024-03-18T14:50:58.802 INFO:tasks.workunit.client.0.smithi012.stdout:2/577: dwrite d0/d10/d18/d33/d71/f87 [0,4194304] 0 2024-03-18T14:50:58.819 INFO:tasks.workunit.client.0.smithi012.stdout:2/578: dwrite d0/d3c/fb3 [0,4194304] 0 2024-03-18T14:50:58.843 INFO:tasks.workunit.client.0.smithi012.stdout:4/486: creat d6/d19/fa0 x:0 0 0 2024-03-18T14:50:58.843 INFO:tasks.workunit.client.0.smithi012.stdout:4/487: chown d6/d1a/d30/d46/f6b 164 1 2024-03-18T14:50:58.845 INFO:tasks.workunit.client.1.smithi100.stdout:8/208: symlink d4/dc/dd/de/d21/d23/d25/l3e 0 2024-03-18T14:50:58.846 INFO:tasks.workunit.client.1.smithi100.stdout:5/164: readlink l13 0 2024-03-18T14:50:58.846 INFO:tasks.workunit.client.1.smithi100.stdout:5/165: stat f7 0 2024-03-18T14:50:58.848 INFO:tasks.workunit.client.1.smithi100.stdout:1/175: getdents d8/d1e 0 2024-03-18T14:50:58.852 INFO:tasks.workunit.client.1.smithi100.stdout:8/209: dread d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:58.852 INFO:tasks.workunit.client.0.smithi012.stdout:2/579: mkdir d0/db4 0 2024-03-18T14:50:58.854 INFO:tasks.workunit.client.1.smithi100.stdout:7/188: getdents d5/d22/d24/d39 0 2024-03-18T14:50:58.855 INFO:tasks.workunit.client.1.smithi100.stdout:3/195: truncate d3/d16/f14 196934 0 2024-03-18T14:50:58.855 INFO:tasks.workunit.client.1.smithi100.stdout:3/196: write d3/dd/d21/f35 [411006,9062] 0 2024-03-18T14:50:58.857 INFO:tasks.workunit.client.0.smithi012.stdout:4/488: mkdir d6/da/d89/da1 0 2024-03-18T14:50:58.857 INFO:tasks.workunit.client.0.smithi012.stdout:4/489: write d6/d19/fa0 [322139,40616] 0 2024-03-18T14:50:58.860 INFO:tasks.workunit.client.0.smithi012.stdout:9/595: rename d3/d9/d19 to d3/d20/dbf 0 2024-03-18T14:50:58.861 INFO:tasks.workunit.client.0.smithi012.stdout:2/580: write d0/d10/d18/f27 [3199746,123309] 0 2024-03-18T14:50:58.862 INFO:tasks.workunit.client.1.smithi100.stdout:3/197: dread d3/dd/d21/f35 [0,4194304] 0 2024-03-18T14:50:58.862 INFO:tasks.workunit.client.1.smithi100.stdout:9/215: creat da/d2c/f41 x:0 0 0 2024-03-18T14:50:58.864 INFO:tasks.workunit.client.0.smithi012.stdout:4/490: mkdir d6/da/d89/d94/da2 0 2024-03-18T14:50:58.864 INFO:tasks.workunit.client.0.smithi012.stdout:4/491: write d6/f80 [114207,51691] 0 2024-03-18T14:50:58.865 INFO:tasks.workunit.client.1.smithi100.stdout:2/156: mknod d3/d13/c39 0 2024-03-18T14:50:58.865 INFO:tasks.workunit.client.1.smithi100.stdout:2/157: fsync d3/f22 0 2024-03-18T14:50:58.868 INFO:tasks.workunit.client.0.smithi012.stdout:4/492: fsync d6/d1a/f2d 0 2024-03-18T14:50:58.869 INFO:tasks.workunit.client.1.smithi100.stdout:6/173: rmdir d6/d12/d2c 39 2024-03-18T14:50:58.870 INFO:tasks.workunit.client.0.smithi012.stdout:3/517: rename dc/d30/d35/c7f to dc/ca3 0 2024-03-18T14:50:58.872 INFO:tasks.workunit.client.0.smithi012.stdout:4/493: link d6/l7 d6/da/d89/da1/la3 0 2024-03-18T14:50:58.872 INFO:tasks.workunit.client.0.smithi012.stdout:1/502: sync 2024-03-18T14:50:58.873 INFO:tasks.workunit.client.1.smithi100.stdout:6/174: dread d6/d10/f13 [0,4194304] 0 2024-03-18T14:50:58.873 INFO:tasks.workunit.client.1.smithi100.stdout:1/176: truncate d8/d1e/d11/d2d/f14 18909 0 2024-03-18T14:50:58.874 INFO:tasks.workunit.client.1.smithi100.stdout:4/193: sync 2024-03-18T14:50:58.874 INFO:tasks.workunit.client.1.smithi100.stdout:4/194: write d9/fb [62958,24594] 0 2024-03-18T14:50:58.875 INFO:tasks.workunit.client.1.smithi100.stdout:4/195: write d9/d27/f36 [13614,36846] 0 2024-03-18T14:50:58.875 INFO:tasks.workunit.client.0.smithi012.stdout:5/543: sync 2024-03-18T14:50:58.877 INFO:tasks.workunit.client.0.smithi012.stdout:2/581: rename d0/f8 to d0/d3c/fb5 0 2024-03-18T14:50:58.881 INFO:tasks.workunit.client.1.smithi100.stdout:8/210: unlink d4/dc/dd/de/l16 0 2024-03-18T14:50:58.884 INFO:tasks.workunit.client.0.smithi012.stdout:3/518: creat dc/fa4 x:0 0 0 2024-03-18T14:50:58.885 INFO:tasks.workunit.client.1.smithi100.stdout:7/189: link d5/d14/d16/f41 d5/d14/d16/d2c/f43 0 2024-03-18T14:50:58.887 INFO:tasks.workunit.client.1.smithi100.stdout:3/198: mknod d3/dd/d21/d3e/c43 0 2024-03-18T14:50:58.887 INFO:tasks.workunit.client.1.smithi100.stdout:0/244: sync 2024-03-18T14:50:58.887 INFO:tasks.workunit.client.1.smithi100.stdout:3/199: write d3/dd/f24 [70035,99100] 0 2024-03-18T14:50:58.889 INFO:tasks.workunit.client.1.smithi100.stdout:9/216: creat da/dd/f42 x:0 0 0 2024-03-18T14:50:58.890 INFO:tasks.workunit.client.1.smithi100.stdout:2/158: rename d3/d1a/f2b to d3/da/f3a 0 2024-03-18T14:50:58.890 INFO:tasks.workunit.client.1.smithi100.stdout:2/159: rename d3/da to d3/da/de/d20/d3b 22 2024-03-18T14:50:58.890 INFO:tasks.workunit.client.1.smithi100.stdout:2/160: stat d3/c7 0 2024-03-18T14:50:58.891 INFO:tasks.workunit.client.0.smithi012.stdout:2/582: creat d0/da2/fb6 x:0 0 0 2024-03-18T14:50:58.891 INFO:tasks.workunit.client.0.smithi012.stdout:2/583: chown d0/d10/d18/d33/d71/d90/fae 14832 1 2024-03-18T14:50:58.892 INFO:tasks.workunit.client.1.smithi100.stdout:5/166: rmdir d29/d2d 39 2024-03-18T14:50:58.892 INFO:tasks.workunit.client.1.smithi100.stdout:5/167: read - d29/f2c zero size 2024-03-18T14:50:58.892 INFO:tasks.workunit.client.1.smithi100.stdout:5/168: creat d29/f37 x:0 0 0 2024-03-18T14:50:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/544: write d3/d29/d36/d9b/d59/fa8 [370944,69175] 0 2024-03-18T14:50:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/545: fdatasync d3/d1e/d33/f74 0 2024-03-18T14:50:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/546: write d3/d15/f37 [711582,83343] 0 2024-03-18T14:50:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/547: fsync d3/d29/d36/d3d/d73/f76 0 2024-03-18T14:50:58.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/548: read - d3/d1e/f8c zero size 2024-03-18T14:50:58.894 INFO:tasks.workunit.client.1.smithi100.stdout:7/190: write d5/f10 [2489531,100678] 0 2024-03-18T14:50:58.894 INFO:tasks.workunit.client.1.smithi100.stdout:7/191: write d5/d30/f3c [941626,57144] 0 2024-03-18T14:50:58.894 INFO:tasks.workunit.client.0.smithi012.stdout:9/596: dwrite d3/f4 [0,4194304] 0 2024-03-18T14:50:58.895 INFO:tasks.workunit.client.1.smithi100.stdout:8/211: write d4/f3a [365317,31067] 0 2024-03-18T14:50:58.895 INFO:tasks.workunit.client.1.smithi100.stdout:9/217: dread da/dd/d29/f2e [0,4194304] 0 2024-03-18T14:50:58.895 INFO:tasks.workunit.client.1.smithi100.stdout:9/218: write da/db/f13 [2480865,73522] 0 2024-03-18T14:50:58.896 INFO:tasks.workunit.client.1.smithi100.stdout:1/177: unlink d8/d1e/c2b 0 2024-03-18T14:50:58.900 INFO:tasks.workunit.client.1.smithi100.stdout:4/196: mkdir d9/d3a 0 2024-03-18T14:50:58.900 INFO:tasks.workunit.client.1.smithi100.stdout:4/197: truncate d9/d17/f24 340036 0 2024-03-18T14:50:58.901 INFO:tasks.workunit.client.0.smithi012.stdout:3/519: creat dc/fa5 x:0 0 0 2024-03-18T14:50:58.901 INFO:tasks.workunit.client.0.smithi012.stdout:3/520: write dc/d30/f51 [1212400,63062] 0 2024-03-18T14:50:58.901 INFO:tasks.workunit.client.0.smithi012.stdout:4/494: dwrite d6/d19/d6c/f88 [0,4194304] 0 2024-03-18T14:50:58.901 INFO:tasks.workunit.client.0.smithi012.stdout:1/503: dwrite d8/d1e/d27/d44/f59 [0,4194304] 0 2024-03-18T14:50:58.902 INFO:tasks.workunit.client.1.smithi100.stdout:2/161: creat d3/da/de/f3c x:0 0 0 2024-03-18T14:50:58.902 INFO:tasks.workunit.client.0.smithi012.stdout:2/584: truncate d0/d3c/fb3 1209459 0 2024-03-18T14:50:58.905 INFO:tasks.workunit.client.1.smithi100.stdout:6/175: rename d6/c19 to d6/d10/d2b/c34 0 2024-03-18T14:50:58.909 INFO:tasks.workunit.client.1.smithi100.stdout:5/169: chown lc 10888030 1 2024-03-18T14:50:58.911 INFO:tasks.workunit.client.0.smithi012.stdout:9/597: creat d3/d20/dbf/d77/d52/fc0 x:0 0 0 2024-03-18T14:50:58.917 INFO:tasks.workunit.client.1.smithi100.stdout:7/192: mkdir d5/d14/d16/d44 0 2024-03-18T14:50:58.918 INFO:tasks.workunit.client.1.smithi100.stdout:7/193: truncate d5/d14/d16/d2c/f43 4815963 0 2024-03-18T14:50:58.918 INFO:tasks.workunit.client.1.smithi100.stdout:7/194: readlink d5/l2f 0 2024-03-18T14:50:58.918 INFO:tasks.workunit.client.1.smithi100.stdout:7/195: chown d5/f35 1744469 1 2024-03-18T14:50:58.918 INFO:tasks.workunit.client.1.smithi100.stdout:8/212: symlink d4/l3f 0 2024-03-18T14:50:58.920 INFO:tasks.workunit.client.1.smithi100.stdout:7/196: dread d5/fa [4194304,4194304] 0 2024-03-18T14:50:58.920 INFO:tasks.workunit.client.1.smithi100.stdout:7/197: fsync d5/d14/d16/d2c/f43 0 2024-03-18T14:50:58.921 INFO:tasks.workunit.client.0.smithi012.stdout:3/521: read dc/d30/f81 [860057,95185] 0 2024-03-18T14:50:58.922 INFO:tasks.workunit.client.0.smithi012.stdout:4/495: creat d6/da/d89/d95/fa4 x:0 0 0 2024-03-18T14:50:58.924 INFO:tasks.workunit.client.1.smithi100.stdout:8/213: dread d4/fa [0,4194304] 0 2024-03-18T14:50:58.926 INFO:tasks.workunit.client.0.smithi012.stdout:1/504: rename d8/db/d10/d16/d1c/f2c to d8/db/d10/d16/d6b/f99 0 2024-03-18T14:50:58.928 INFO:tasks.workunit.client.0.smithi012.stdout:3/522: mknod dc/d30/d35/ca6 0 2024-03-18T14:50:58.929 INFO:tasks.workunit.client.0.smithi012.stdout:4/496: mkdir d6/d19/da5 0 2024-03-18T14:50:58.929 INFO:tasks.workunit.client.1.smithi100.stdout:5/170: write f23 [3355969,76361] 0 2024-03-18T14:50:58.930 INFO:tasks.workunit.client.0.smithi012.stdout:1/505: mknod d8/d1e/d27/c9a 0 2024-03-18T14:50:58.931 INFO:tasks.workunit.client.0.smithi012.stdout:1/506: stat d8/d1e/d27/f38 0 2024-03-18T14:50:58.931 INFO:tasks.workunit.client.0.smithi012.stdout:1/507: truncate d8/db/d10/f94 564564 0 2024-03-18T14:50:58.932 INFO:tasks.workunit.client.1.smithi100.stdout:7/198: read d5/d14/d16/d2c/f43 [1088873,22158] 0 2024-03-18T14:50:58.932 INFO:tasks.workunit.client.1.smithi100.stdout:9/219: mknod da/d2c/d2f/c43 0 2024-03-18T14:50:58.934 INFO:tasks.workunit.client.0.smithi012.stdout:3/523: creat dc/d3c/d59/d86/d9a/fa7 x:0 0 0 2024-03-18T14:50:58.938 INFO:tasks.workunit.client.0.smithi012.stdout:2/585: dwrite d0/f97 [0,4194304] 0 2024-03-18T14:50:58.938 INFO:tasks.workunit.client.1.smithi100.stdout:1/178: creat d8/d1e/f31 x:0 0 0 2024-03-18T14:50:58.939 INFO:tasks.workunit.client.1.smithi100.stdout:4/198: creat d9/f3b x:0 0 0 2024-03-18T14:50:58.942 INFO:tasks.workunit.client.1.smithi100.stdout:2/162: creat d3/d6/d29/f3d x:0 0 0 2024-03-18T14:50:58.942 INFO:tasks.workunit.client.1.smithi100.stdout:2/163: stat d3/d6/d29/f35 0 2024-03-18T14:50:58.942 INFO:tasks.workunit.client.0.smithi012.stdout:5/549: dwrite d3/d15/f1d [0,4194304] 0 2024-03-18T14:50:58.942 INFO:tasks.workunit.client.0.smithi012.stdout:5/550: chown d3/d29/d36/d9b/d59/f7c 2111544 1 2024-03-18T14:50:58.943 INFO:tasks.workunit.client.1.smithi100.stdout:6/176: rmdir d6/d7 39 2024-03-18T14:50:58.943 INFO:tasks.workunit.client.0.smithi012.stdout:9/598: dwrite d3/d20/dbf/f6a [0,4194304] 0 2024-03-18T14:50:58.945 INFO:tasks.workunit.client.1.smithi100.stdout:3/200: dwrite d3/f29 [0,4194304] 0 2024-03-18T14:50:58.948 INFO:tasks.workunit.client.0.smithi012.stdout:2/586: mkdir d0/d6c/db7 0 2024-03-18T14:50:58.950 INFO:tasks.workunit.client.1.smithi100.stdout:5/171: mkdir d29/d2d/d32/d38 0 2024-03-18T14:50:58.950 INFO:tasks.workunit.client.1.smithi100.stdout:5/172: dread - d29/f2a zero size 2024-03-18T14:50:58.953 INFO:tasks.workunit.client.0.smithi012.stdout:5/551: truncate d3/d15/f1b 2433955 0 2024-03-18T14:50:58.953 INFO:tasks.workunit.client.0.smithi012.stdout:5/552: readlink d3/d1e/l60 0 2024-03-18T14:50:58.954 INFO:tasks.workunit.client.0.smithi012.stdout:5/553: chown d3/d29/d36/d3d/d73/l77 37389703 1 2024-03-18T14:50:58.954 INFO:tasks.workunit.client.0.smithi012.stdout:5/554: write d3/db6/fb7 [269909,48762] 0 2024-03-18T14:50:58.955 INFO:tasks.workunit.client.1.smithi100.stdout:7/199: rename d5/f3f to d5/d22/d24/d39/f45 0 2024-03-18T14:50:58.955 INFO:tasks.workunit.client.1.smithi100.stdout:7/200: creat d5/f46 x:0 0 0 2024-03-18T14:50:58.957 INFO:tasks.workunit.client.1.smithi100.stdout:1/179: mknod d8/d30/c32 0 2024-03-18T14:50:58.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/164: symlink d3/d6/d29/d36/l3e 0 2024-03-18T14:50:58.961 INFO:tasks.workunit.client.0.smithi012.stdout:1/508: dwrite d8/db/d10/d56/f8a [0,4194304] 0 2024-03-18T14:50:58.961 INFO:tasks.workunit.client.0.smithi012.stdout:1/509: creat d8/db/d10/d31/d43/d5e/d65/d84/f9b x:0 0 0 2024-03-18T14:50:58.962 INFO:tasks.workunit.client.1.smithi100.stdout:6/177: read d6/d12/f1f [2812674,107799] 0 2024-03-18T14:50:58.972 INFO:tasks.workunit.client.1.smithi100.stdout:6/178: dread f4 [0,4194304] 0 2024-03-18T14:50:58.973 INFO:tasks.workunit.client.0.smithi012.stdout:3/524: dwrite dc/d30/d3a/f4f [0,4194304] 0 2024-03-18T14:50:58.974 INFO:tasks.workunit.client.0.smithi012.stdout:1/510: dread d8/db/d10/d16/d1c/f4c [0,4194304] 0 2024-03-18T14:50:58.974 INFO:tasks.workunit.client.0.smithi012.stdout:1/511: stat d8/db/fd 0 2024-03-18T14:50:58.979 INFO:tasks.workunit.client.0.smithi012.stdout:2/587: dwrite d0/d3c/f41 [0,4194304] 0 2024-03-18T14:50:58.979 INFO:tasks.workunit.client.1.smithi100.stdout:8/214: dwrite d4/fa [0,4194304] 0 2024-03-18T14:50:58.979 INFO:tasks.workunit.client.1.smithi100.stdout:1/180: dwrite d8/d1e/d11/d2d/f25 [0,4194304] 0 2024-03-18T14:50:58.985 INFO:tasks.workunit.client.1.smithi100.stdout:8/215: dread d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:50:58.985 INFO:tasks.workunit.client.1.smithi100.stdout:8/216: chown d4/dc/dd/f11 201949211 1 2024-03-18T14:50:59.046 INFO:tasks.workunit.client.1.smithi100.stdout:3/201: link d3/c7 d3/d16/d13/c44 0 2024-03-18T14:50:59.048 INFO:tasks.workunit.client.1.smithi100.stdout:5/173: mkdir d29/d2d/d39 0 2024-03-18T14:50:59.048 INFO:tasks.workunit.client.1.smithi100.stdout:5/174: write d29/f34 [179226,82220] 0 2024-03-18T14:50:59.052 INFO:tasks.workunit.client.1.smithi100.stdout:4/199: rename d9/d13/f2f to d9/d3a/f3c 0 2024-03-18T14:50:59.057 INFO:tasks.workunit.client.1.smithi100.stdout:7/201: mkdir d5/d22/d3b/d47 0 2024-03-18T14:50:59.057 INFO:tasks.workunit.client.1.smithi100.stdout:7/202: dread - d5/d22/d24/f2d zero size 2024-03-18T14:50:59.059 INFO:tasks.workunit.client.1.smithi100.stdout:6/179: mknod d6/d12/c35 0 2024-03-18T14:50:59.060 INFO:tasks.workunit.client.0.smithi012.stdout:3/525: rename dc/d30/f9c to dc/d3c/d59/d86/fa8 0 2024-03-18T14:50:59.062 INFO:tasks.workunit.client.0.smithi012.stdout:1/512: truncate d8/db/f15 243608 0 2024-03-18T14:50:59.062 INFO:tasks.workunit.client.0.smithi012.stdout:1/513: dread - d8/db/d10/d31/f6e zero size 2024-03-18T14:50:59.069 INFO:tasks.workunit.client.1.smithi100.stdout:4/200: dwrite d9/f22 [0,4194304] 0 2024-03-18T14:50:59.070 INFO:tasks.workunit.client.1.smithi100.stdout:8/217: creat d4/dc/dd/de/d19/f40 x:0 0 0 2024-03-18T14:50:59.071 INFO:tasks.workunit.client.1.smithi100.stdout:3/202: creat d3/d16/f45 x:0 0 0 2024-03-18T14:50:59.075 INFO:tasks.workunit.client.0.smithi012.stdout:2/588: unlink d0/fa 0 2024-03-18T14:50:59.076 INFO:tasks.workunit.client.0.smithi012.stdout:3/526: dwrite dc/d3c/d59/d86/d9a/f5d [0,4194304] 0 2024-03-18T14:50:59.078 INFO:tasks.workunit.client.0.smithi012.stdout:9/599: mkdir d3/d20/dc1 0 2024-03-18T14:50:59.080 INFO:tasks.workunit.client.0.smithi012.stdout:1/514: getdents d8/db/d10/d31/d43 0 2024-03-18T14:50:59.082 INFO:tasks.workunit.client.0.smithi012.stdout:2/589: symlink d0/d3c/d40/d47/lb8 0 2024-03-18T14:50:59.082 INFO:tasks.workunit.client.0.smithi012.stdout:2/590: chown d0/d10/d18/d4f/f7c 72520312 1 2024-03-18T14:50:59.082 INFO:tasks.workunit.client.0.smithi012.stdout:2/591: chown d0/d6c/fa9 661435 1 2024-03-18T14:50:59.083 INFO:tasks.workunit.client.0.smithi012.stdout:3/527: getdents dc/d3c/d59/d86/d9a/d64 0 2024-03-18T14:50:59.086 INFO:tasks.workunit.client.1.smithi100.stdout:1/181: rename d8/d1e/d11/d2d/f14 to d8/d1e/d11/f33 0 2024-03-18T14:50:59.086 INFO:tasks.workunit.client.1.smithi100.stdout:1/182: dread - d8/d1e/d11/f21 zero size 2024-03-18T14:50:59.088 INFO:tasks.workunit.client.0.smithi012.stdout:9/600: creat d3/d20/fc2 x:0 0 0 2024-03-18T14:50:59.088 INFO:tasks.workunit.client.0.smithi012.stdout:9/601: fsync d3/d20/dbf/d77/d52/d65/f86 0 2024-03-18T14:50:59.088 INFO:tasks.workunit.client.0.smithi012.stdout:9/602: readlink d3/d20/dbf/d66/l37 0 2024-03-18T14:50:59.091 INFO:tasks.workunit.client.1.smithi100.stdout:6/180: mkdir d6/d10/d2b/d36 0 2024-03-18T14:50:59.092 INFO:tasks.workunit.client.0.smithi012.stdout:2/592: dread d0/d10/d18/f5b [0,4194304] 0 2024-03-18T14:50:59.093 INFO:tasks.workunit.client.1.smithi100.stdout:1/183: dread f7 [0,4194304] 0 2024-03-18T14:50:59.096 INFO:tasks.workunit.client.1.smithi100.stdout:4/201: truncate d9/d17/f1c 5488576 0 2024-03-18T14:50:59.097 INFO:tasks.workunit.client.0.smithi012.stdout:3/528: mknod dc/d3c/d4c/d57/ca9 0 2024-03-18T14:50:59.097 INFO:tasks.workunit.client.0.smithi012.stdout:3/529: dread - dc/d3c/f74 zero size 2024-03-18T14:50:59.098 INFO:tasks.workunit.client.1.smithi100.stdout:8/218: mkdir d4/dc/dd/de/d21/d23/d41 0 2024-03-18T14:50:59.100 INFO:tasks.workunit.client.0.smithi012.stdout:1/515: dwrite d8/db/d10/d16/d1c/f4c [0,4194304] 0 2024-03-18T14:50:59.101 INFO:tasks.workunit.client.0.smithi012.stdout:9/603: unlink d3/f83 0 2024-03-18T14:50:59.102 INFO:tasks.workunit.client.1.smithi100.stdout:3/203: mknod d3/d16/d13/d1f/d3f/c46 0 2024-03-18T14:50:59.104 INFO:tasks.workunit.client.1.smithi100.stdout:8/219: dread d4/dc/dd/f34 [0,4194304] 0 2024-03-18T14:50:59.104 INFO:tasks.workunit.client.1.smithi100.stdout:8/220: creat d4/dc/dd/de/d19/f42 x:0 0 0 2024-03-18T14:50:59.105 INFO:tasks.workunit.client.0.smithi012.stdout:2/593: link d0/d3c/f46 d0/db4/fb9 0 2024-03-18T14:50:59.105 INFO:tasks.workunit.client.0.smithi012.stdout:2/594: stat d0/d10/d18/d33/d71/lb0 0 2024-03-18T14:50:59.105 INFO:tasks.workunit.client.1.smithi100.stdout:3/204: dread d3/dd/f24 [0,4194304] 0 2024-03-18T14:50:59.105 INFO:tasks.workunit.client.1.smithi100.stdout:5/175: rmdir d29 39 2024-03-18T14:50:59.107 INFO:tasks.workunit.client.0.smithi012.stdout:3/530: creat dc/d3c/d4c/faa x:0 0 0 2024-03-18T14:50:59.107 INFO:tasks.workunit.client.0.smithi012.stdout:3/531: readlink dc/d91/la2 0 2024-03-18T14:50:59.110 INFO:tasks.workunit.client.0.smithi012.stdout:1/516: mkdir d8/db/d10/d16/d1c/d2e/d9c 0 2024-03-18T14:50:59.110 INFO:tasks.workunit.client.0.smithi012.stdout:1/517: stat d8/db/d10/d31/d43/d5e/d65/c85 0 2024-03-18T14:50:59.111 INFO:tasks.workunit.client.1.smithi100.stdout:7/203: getdents d5/d30 0 2024-03-18T14:50:59.112 INFO:tasks.workunit.client.0.smithi012.stdout:9/604: symlink d3/d20/dbf/d77/d52/d65/lc3 0 2024-03-18T14:50:59.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/181: rmdir d6/d7/df 39 2024-03-18T14:50:59.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/182: chown d6/f2f 237 1 2024-03-18T14:50:59.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/183: dread - d6/d10/d2b/f2d zero size 2024-03-18T14:50:59.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/184: write d6/f15 [1441792,89300] 0 2024-03-18T14:50:59.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/185: write d6/d10/f22 [4644531,120079] 0 2024-03-18T14:50:59.115 INFO:tasks.workunit.client.0.smithi012.stdout:6/529: sync 2024-03-18T14:50:59.115 INFO:tasks.workunit.client.0.smithi012.stdout:9/605: dread d3/f3b [0,4194304] 0 2024-03-18T14:50:59.116 INFO:tasks.workunit.client.1.smithi100.stdout:1/184: symlink d8/d1e/d11/d2d/l34 0 2024-03-18T14:50:59.116 INFO:tasks.workunit.client.1.smithi100.stdout:1/185: write f5 [3675573,81066] 0 2024-03-18T14:50:59.116 INFO:tasks.workunit.client.1.smithi100.stdout:1/186: write f5 [1885624,12497] 0 2024-03-18T14:50:59.120 INFO:tasks.workunit.client.1.smithi100.stdout:1/187: dread f7 [0,4194304] 0 2024-03-18T14:50:59.120 INFO:tasks.workunit.client.1.smithi100.stdout:1/188: truncate d8/d1e/f28 310713 0 2024-03-18T14:50:59.121 INFO:tasks.workunit.client.0.smithi012.stdout:3/532: mkdir dc/d30/dab 0 2024-03-18T14:50:59.121 INFO:tasks.workunit.client.0.smithi012.stdout:3/533: dread - dc/d3c/f74 zero size 2024-03-18T14:50:59.124 INFO:tasks.workunit.client.1.smithi100.stdout:0/245: sync 2024-03-18T14:50:59.124 INFO:tasks.workunit.client.0.smithi012.stdout:1/518: mkdir d8/db/d10/d16/d1c/d2e/d9c/d9d 0 2024-03-18T14:50:59.126 INFO:tasks.workunit.client.1.smithi100.stdout:3/205: creat d3/d16/d13/f47 x:0 0 0 2024-03-18T14:50:59.127 INFO:tasks.workunit.client.0.smithi012.stdout:6/530: truncate fb 1349872 0 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:0/548: sync 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:7/544: sync 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:8/735: sync 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:9/606: dread d3/d9/f39 [0,4194304] 0 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:9/607: write d3/d20/dbf/d66/d21/d35/d48/fa4 [327273,45620] 0 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:9/608: stat d3/d20/dbf/d66/d21/d35/d48/l2d 0 2024-03-18T14:50:59.130 INFO:tasks.workunit.client.0.smithi012.stdout:9/609: stat d3/d20/dbf/d77/d52/d78 0 2024-03-18T14:50:59.133 INFO:tasks.workunit.client.0.smithi012.stdout:3/534: truncate dc/d3c/d4c/f94 349247 0 2024-03-18T14:50:59.134 INFO:tasks.workunit.client.0.smithi012.stdout:8/736: dread d0/d5a/fa2 [0,4194304] 0 2024-03-18T14:50:59.136 INFO:tasks.workunit.client.0.smithi012.stdout:1/519: mknod d8/c9e 0 2024-03-18T14:50:59.137 INFO:tasks.workunit.client.0.smithi012.stdout:1/520: creat d8/d1e/f9f x:0 0 0 2024-03-18T14:50:59.138 INFO:tasks.workunit.client.1.smithi100.stdout:6/186: rmdir d6/d7/df 39 2024-03-18T14:50:59.139 INFO:tasks.workunit.client.1.smithi100.stdout:7/204: dwrite d5/d22/d24/d39/f45 [0,4194304] 0 2024-03-18T14:50:59.142 INFO:tasks.workunit.client.0.smithi012.stdout:2/595: dwrite d0/d10/d18/d4f/f7c [0,4194304] 0 2024-03-18T14:50:59.142 INFO:tasks.workunit.client.0.smithi012.stdout:9/610: creat d3/d20/dbf/d77/d52/d65/fc4 x:0 0 0 2024-03-18T14:50:59.142 INFO:tasks.workunit.client.0.smithi012.stdout:7/545: write dd/d13/d23/d33/f78 [1727513,5502] 0 2024-03-18T14:50:59.142 INFO:tasks.workunit.client.0.smithi012.stdout:7/546: dread - dd/d13/f57 zero size 2024-03-18T14:50:59.144 INFO:tasks.workunit.client.0.smithi012.stdout:8/737: dread d0/f6 [0,4194304] 0 2024-03-18T14:50:59.144 INFO:tasks.workunit.client.1.smithi100.stdout:1/189: dread f5 [0,4194304] 0 2024-03-18T14:50:59.145 INFO:tasks.workunit.client.1.smithi100.stdout:1/190: truncate d8/f1a 4782437 0 2024-03-18T14:50:59.145 INFO:tasks.workunit.client.1.smithi100.stdout:1/191: write d8/d1e/f31 [967098,76168] 0 2024-03-18T14:50:59.145 INFO:tasks.workunit.client.0.smithi012.stdout:3/535: truncate dc/f12 2512159 0 2024-03-18T14:50:59.147 INFO:tasks.workunit.client.1.smithi100.stdout:9/220: sync 2024-03-18T14:50:59.147 INFO:tasks.workunit.client.1.smithi100.stdout:2/165: sync 2024-03-18T14:50:59.150 INFO:tasks.workunit.client.1.smithi100.stdout:4/202: dread d9/d17/f1c [0,4194304] 0 2024-03-18T14:50:59.150 INFO:tasks.workunit.client.1.smithi100.stdout:4/203: fsync d9/f22 0 2024-03-18T14:50:59.150 INFO:tasks.workunit.client.1.smithi100.stdout:4/204: chown d9/d13/d39 7568100 1 2024-03-18T14:50:59.152 INFO:tasks.workunit.client.1.smithi100.stdout:8/221: rename d4/dc/dd/de/d19/f42 to d4/f43 0 2024-03-18T14:50:59.153 INFO:tasks.workunit.client.0.smithi012.stdout:2/596: dread d0/d10/f6a [0,4194304] 0 2024-03-18T14:50:59.155 INFO:tasks.workunit.client.1.smithi100.stdout:2/166: dread d3/da/de/d20/f24 [0,4194304] 0 2024-03-18T14:50:59.155 INFO:tasks.workunit.client.1.smithi100.stdout:2/167: truncate d3/f4 1959945 0 2024-03-18T14:50:59.155 INFO:tasks.workunit.client.1.smithi100.stdout:0/246: symlink dd/d31/l60 0 2024-03-18T14:50:59.158 INFO:tasks.workunit.client.0.smithi012.stdout:1/521: link d8/db/f7d d8/d1e/d27/d44/fa0 0 2024-03-18T14:50:59.158 INFO:tasks.workunit.client.0.smithi012.stdout:1/522: stat d8/db/d10/d16/d54/l58 0 2024-03-18T14:50:59.158 INFO:tasks.workunit.client.0.smithi012.stdout:0/549: dwrite d1/d20/d23/d6f/f75 [0,4194304] 0 2024-03-18T14:50:59.159 INFO:tasks.workunit.client.1.smithi100.stdout:3/206: mknod d3/d16/d13/d1f/d37/c48 0 2024-03-18T14:50:59.161 INFO:tasks.workunit.client.0.smithi012.stdout:9/611: mknod d3/d20/dbf/d66/d69/d7f/cc5 0 2024-03-18T14:50:59.161 INFO:tasks.workunit.client.0.smithi012.stdout:9/612: write d3/d20/dbf/d77/d52/f6d [4987606,88714] 0 2024-03-18T14:50:59.161 INFO:tasks.workunit.client.0.smithi012.stdout:9/613: chown d3/d20/dbf/d77/d52/d99/lab 127 1 2024-03-18T14:50:59.161 INFO:tasks.workunit.client.0.smithi012.stdout:9/614: fdatasync d3/d49/f57 0 2024-03-18T14:50:59.162 INFO:tasks.workunit.client.0.smithi012.stdout:4/497: sync 2024-03-18T14:50:59.162 INFO:tasks.workunit.client.1.smithi100.stdout:6/187: rmdir d6/d7/df 39 2024-03-18T14:50:59.164 INFO:tasks.workunit.client.0.smithi012.stdout:8/738: rename d0/d15/f38 to d0/d5a/d19/fe5 0 2024-03-18T14:50:59.167 INFO:tasks.workunit.client.0.smithi012.stdout:3/536: creat dc/d30/fac x:0 0 0 2024-03-18T14:50:59.170 INFO:tasks.workunit.client.0.smithi012.stdout:3/537: write dc/d30/f99 [125870,117346] 0 2024-03-18T14:50:59.170 INFO:tasks.workunit.client.0.smithi012.stdout:2/597: creat d0/d3c/d40/d47/d85/fba x:0 0 0 2024-03-18T14:50:59.172 INFO:tasks.workunit.client.1.smithi100.stdout:7/205: mkdir d5/d48 0 2024-03-18T14:50:59.177 INFO:tasks.workunit.client.1.smithi100.stdout:1/192: creat d8/d30/f35 x:0 0 0 2024-03-18T14:50:59.177 INFO:tasks.workunit.client.1.smithi100.stdout:1/193: creat d8/d1e/f36 x:0 0 0 2024-03-18T14:50:59.180 INFO:tasks.workunit.client.0.smithi012.stdout:6/531: dwrite d17/d19/d27/d58/f77 [0,4194304] 0 2024-03-18T14:50:59.184 INFO:tasks.workunit.client.1.smithi100.stdout:4/205: dwrite d9/f1f [0,4194304] 0 2024-03-18T14:50:59.184 INFO:tasks.workunit.client.1.smithi100.stdout:4/206: fsync d9/d13/f1e 0 2024-03-18T14:50:59.186 INFO:tasks.workunit.client.1.smithi100.stdout:5/176: rename l13 to d29/l3a 0 2024-03-18T14:50:59.187 INFO:tasks.workunit.client.0.smithi012.stdout:2/598: read d0/d10/d18/d4f/f7c [3198422,116652] 0 2024-03-18T14:50:59.189 INFO:tasks.workunit.client.1.smithi100.stdout:3/207: mkdir d3/dd/d49 0 2024-03-18T14:50:59.190 INFO:tasks.workunit.client.0.smithi012.stdout:0/550: truncate d1/f9 2586298 0 2024-03-18T14:50:59.190 INFO:tasks.workunit.client.0.smithi012.stdout:0/551: read - d1/d16/d19/d8d/fab zero size 2024-03-18T14:50:59.192 INFO:tasks.workunit.client.1.smithi100.stdout:6/188: link d6/d7/df/d26/c20 d6/d7/df/c37 0 2024-03-18T14:50:59.192 INFO:tasks.workunit.client.1.smithi100.stdout:6/189: creat d6/d10/f38 x:0 0 0 2024-03-18T14:50:59.192 INFO:tasks.workunit.client.0.smithi012.stdout:9/615: write d3/d20/dbf/d77/da8/f87 [544605,2975] 0 2024-03-18T14:50:59.195 INFO:tasks.workunit.client.1.smithi100.stdout:7/206: link d5/d14/f1e d5/d22/d3b/d47/f49 0 2024-03-18T14:50:59.195 INFO:tasks.workunit.client.1.smithi100.stdout:7/207: write d5/f46 [832491,118638] 0 2024-03-18T14:50:59.197 INFO:tasks.workunit.client.0.smithi012.stdout:8/739: link d0/d5a/d5e/ddf/f97 d0/d47/d59/dc9/fe6 0 2024-03-18T14:50:59.197 INFO:tasks.workunit.client.0.smithi012.stdout:8/740: readlink d0/l34 0 2024-03-18T14:50:59.200 INFO:tasks.workunit.client.1.smithi100.stdout:1/194: mkdir d8/d2a/d37 0 2024-03-18T14:50:59.200 INFO:tasks.workunit.client.0.smithi012.stdout:8/741: dread d0/d15/f94 [0,4194304] 0 2024-03-18T14:50:59.200 INFO:tasks.workunit.client.0.smithi012.stdout:8/742: write d0/d5a/d5e/ddf/f44 [398734,130871] 0 2024-03-18T14:50:59.201 INFO:tasks.workunit.client.1.smithi100.stdout:9/221: mkdir da/db/d44 0 2024-03-18T14:50:59.202 INFO:tasks.workunit.client.1.smithi100.stdout:8/222: rename d4/dc/d1f/f20 to d4/dc/dd/f44 0 2024-03-18T14:50:59.202 INFO:tasks.workunit.client.1.smithi100.stdout:8/223: chown d4/dc/dd/de/d19 182 1 2024-03-18T14:50:59.204 INFO:tasks.workunit.client.1.smithi100.stdout:3/208: creat d3/dd/d49/f4a x:0 0 0 2024-03-18T14:50:59.206 INFO:tasks.workunit.client.1.smithi100.stdout:6/190: mknod d6/d10/d2b/c39 0 2024-03-18T14:50:59.208 INFO:tasks.workunit.client.1.smithi100.stdout:7/208: creat d5/d30/f4a x:0 0 0 2024-03-18T14:50:59.209 INFO:tasks.workunit.client.1.smithi100.stdout:7/209: chown d5/f46 332 1 2024-03-18T14:50:59.210 INFO:tasks.workunit.client.1.smithi100.stdout:1/195: symlink d8/d30/l38 0 2024-03-18T14:50:59.210 INFO:tasks.workunit.client.1.smithi100.stdout:1/196: chown d8/dd/ff 601756232 1 2024-03-18T14:50:59.210 INFO:tasks.workunit.client.1.smithi100.stdout:1/197: write f5 [763660,8238] 0 2024-03-18T14:50:59.211 INFO:tasks.workunit.client.1.smithi100.stdout:9/222: symlink da/dd/d3b/l45 0 2024-03-18T14:50:59.211 INFO:tasks.workunit.client.1.smithi100.stdout:9/223: chown da/dd/c39 4589415 1 2024-03-18T14:50:59.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/532: dread d17/f2c [0,4194304] 0 2024-03-18T14:50:59.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/533: dread - d17/d26/f8c zero size 2024-03-18T14:50:59.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/534: readlink d17/d26/l9c 0 2024-03-18T14:50:59.212 INFO:tasks.workunit.client.0.smithi012.stdout:1/523: rmdir d8/db/d10 39 2024-03-18T14:50:59.213 INFO:tasks.workunit.client.1.smithi100.stdout:8/224: truncate d4/f15 28544 0 2024-03-18T14:50:59.215 INFO:tasks.workunit.client.1.smithi100.stdout:6/191: creat d6/d12/f3a x:0 0 0 2024-03-18T14:50:59.216 INFO:tasks.workunit.client.1.smithi100.stdout:6/192: stat d6/d10/f38 0 2024-03-18T14:50:59.216 INFO:tasks.workunit.client.1.smithi100.stdout:6/193: creat d6/d12/f3b x:0 0 0 2024-03-18T14:50:59.217 INFO:tasks.workunit.client.0.smithi012.stdout:2/599: unlink d0/c45 0 2024-03-18T14:50:59.218 INFO:tasks.workunit.client.1.smithi100.stdout:7/210: symlink d5/d14/d16/d44/l4b 0 2024-03-18T14:50:59.219 INFO:tasks.workunit.client.0.smithi012.stdout:9/616: mkdir d3/d20/dbf/d66/d6e/dc6 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/198: link f7 d8/d2a/d37/f39 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/199: write d8/d1e/f36 [510992,14019] 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/200: chown d8/dd/c1d 393912 1 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:9/224: dread da/d25/f35 [0,4194304] 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/201: write d8/d30/f35 [20642,78223] 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/202: rename d8/d1e to d8/d1e/d11/d3a 22 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/203: chown d8/f29 633504769 1 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/204: write d8/d1e/f31 [1128996,4718] 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:1/205: write d8/d1e/f28 [588710,87879] 0 2024-03-18T14:50:59.221 INFO:tasks.workunit.client.1.smithi100.stdout:9/225: write da/dd/f10 [9298282,32550] 0 2024-03-18T14:50:59.222 INFO:tasks.workunit.client.1.smithi100.stdout:8/225: dread d4/f9 [0,4194304] 0 2024-03-18T14:50:59.222 INFO:tasks.workunit.client.1.smithi100.stdout:8/226: dread - d4/dc/d1f/f32 zero size 2024-03-18T14:50:59.222 INFO:tasks.workunit.client.0.smithi012.stdout:3/538: mknod dc/cad 0 2024-03-18T14:50:59.224 INFO:tasks.workunit.client.0.smithi012.stdout:7/547: dwrite dd/d13/f72 [0,4194304] 0 2024-03-18T14:50:59.225 INFO:tasks.workunit.client.0.smithi012.stdout:7/548: chown dd/d13/d23/c83 9 1 2024-03-18T14:50:59.225 INFO:tasks.workunit.client.1.smithi100.stdout:3/209: symlink d3/l4b 0 2024-03-18T14:50:59.225 INFO:tasks.workunit.client.1.smithi100.stdout:3/210: write d3/d16/f14 [139409,97893] 0 2024-03-18T14:50:59.225 INFO:tasks.workunit.client.1.smithi100.stdout:3/211: chown d3/l4b 108 1 2024-03-18T14:50:59.228 INFO:tasks.workunit.client.0.smithi012.stdout:6/535: getdents d17/d19 0 2024-03-18T14:50:59.228 INFO:tasks.workunit.client.1.smithi100.stdout:6/194: symlink d6/d10/l3c 0 2024-03-18T14:50:59.230 INFO:tasks.workunit.client.1.smithi100.stdout:7/211: link d5/d14/d16/f41 d5/d14/d16/d44/f4c 0 2024-03-18T14:50:59.233 INFO:tasks.workunit.client.0.smithi012.stdout:1/524: rename d8/db/d10/d16/d1c to d8/db/d10/d16/da1 0 2024-03-18T14:50:59.233 INFO:tasks.workunit.client.0.smithi012.stdout:1/525: write d8/db/d10/d31/f6e [771083,50823] 0 2024-03-18T14:50:59.233 INFO:tasks.workunit.client.0.smithi012.stdout:1/526: readlink d8/db/d10/d31/d45/l5d 0 2024-03-18T14:50:59.235 INFO:tasks.workunit.client.0.smithi012.stdout:9/617: link d3/d20/dbf/d66/d21/d35/l3d d3/d20/dbf/d77/d52/d78/lc7 0 2024-03-18T14:50:59.237 INFO:tasks.workunit.client.0.smithi012.stdout:3/539: mknod dc/d3c/d4c/d57/cae 0 2024-03-18T14:50:59.237 INFO:tasks.workunit.client.0.smithi012.stdout:3/540: readlink dc/d30/d35/l8c 0 2024-03-18T14:50:59.237 INFO:tasks.workunit.client.0.smithi012.stdout:3/541: write dc/d30/d35/f2c [586487,24369] 0 2024-03-18T14:50:59.237 INFO:tasks.workunit.client.0.smithi012.stdout:0/552: dwrite d1/d16/d19/d8d/fab [0,4194304] 0 2024-03-18T14:50:59.239 INFO:tasks.workunit.client.1.smithi100.stdout:5/177: dwrite f20 [0,4194304] 0 2024-03-18T14:50:59.239 INFO:tasks.workunit.client.1.smithi100.stdout:5/178: getdents d29/d2d/d32/d38 0 2024-03-18T14:50:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/549: rename dd/d13/d23/d76/d6f/la2 to dd/d13/d23/d2f/d7a/db6/lbf 0 2024-03-18T14:50:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/550: chown dd/d13/f57 10 1 2024-03-18T14:50:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/551: write dd/d13/d23/d76/fb2 [48910,67944] 0 2024-03-18T14:50:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:7/552: readlink dd/d13/d23/d2f/d7a/dab/lba 0 2024-03-18T14:50:59.240 INFO:tasks.workunit.client.0.smithi012.stdout:4/498: dwrite d6/d1a/d92/f5e [0,4194304] 0 2024-03-18T14:50:59.241 INFO:tasks.workunit.client.1.smithi100.stdout:4/207: dwrite d9/f2b [0,4194304] 0 2024-03-18T14:50:59.242 INFO:tasks.workunit.client.0.smithi012.stdout:2/600: dwrite d0/f79 [0,4194304] 0 2024-03-18T14:50:59.242 INFO:tasks.workunit.client.1.smithi100.stdout:4/208: truncate d9/f1b 4840569 0 2024-03-18T14:50:59.242 INFO:tasks.workunit.client.1.smithi100.stdout:2/168: dwrite d3/d13/f37 [0,4194304] 0 2024-03-18T14:50:59.242 INFO:tasks.workunit.client.1.smithi100.stdout:0/247: dwrite dd/d31/f57 [0,4194304] 0 2024-03-18T14:50:59.243 INFO:tasks.workunit.client.1.smithi100.stdout:6/195: write d6/d10/f22 [3568694,49173] 0 2024-03-18T14:50:59.243 INFO:tasks.workunit.client.1.smithi100.stdout:8/227: unlink d4/dc/dd/de/d19/l1d 0 2024-03-18T14:50:59.244 INFO:tasks.workunit.client.1.smithi100.stdout:8/228: write f1 [836749,52742] 0 2024-03-18T14:50:59.244 INFO:tasks.workunit.client.1.smithi100.stdout:9/226: dwrite da/d25/f3a [0,4194304] 0 2024-03-18T14:50:59.244 INFO:tasks.workunit.client.0.smithi012.stdout:8/743: link d0/d5a/d5e/dd7/d84/c95 d0/d5a/d19/d63/dbe/ce7 0 2024-03-18T14:50:59.246 INFO:tasks.workunit.client.1.smithi100.stdout:2/169: dread d3/f4 [0,4194304] 0 2024-03-18T14:50:59.247 INFO:tasks.workunit.client.1.smithi100.stdout:3/212: link d3/dd/d49/f4a d3/d16/d13/d17/f4c 0 2024-03-18T14:50:59.247 INFO:tasks.workunit.client.0.smithi012.stdout:6/536: write d17/d38/d5b/f87 [3533255,62139] 0 2024-03-18T14:50:59.247 INFO:tasks.workunit.client.0.smithi012.stdout:6/537: stat d17/d33/l3a 0 2024-03-18T14:50:59.247 INFO:tasks.workunit.client.0.smithi012.stdout:6/538: chown d17/d38/d3f/l61 11 1 2024-03-18T14:50:59.250 INFO:tasks.workunit.client.0.smithi012.stdout:9/618: rmdir d3/d20/dbf/d77/d52/d78 39 2024-03-18T14:50:59.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/248: dread fa [0,4194304] 0 2024-03-18T14:50:59.255 INFO:tasks.workunit.client.1.smithi100.stdout:8/229: write d4/dc/dd/f11 [3500371,21934] 0 2024-03-18T14:50:59.255 INFO:tasks.workunit.client.1.smithi100.stdout:7/212: rmdir d5/d22/d3b 39 2024-03-18T14:50:59.256 INFO:tasks.workunit.client.1.smithi100.stdout:0/249: dread dd/f1a [0,4194304] 0 2024-03-18T14:50:59.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/209: write d9/fe [3356804,39660] 0 2024-03-18T14:50:59.263 INFO:tasks.workunit.client.1.smithi100.stdout:7/213: read d5/f46 [476382,5217] 0 2024-03-18T14:50:59.270 INFO:tasks.workunit.client.1.smithi100.stdout:5/179: write f1b [440870,119970] 0 2024-03-18T14:50:59.271 INFO:tasks.workunit.client.0.smithi012.stdout:0/553: mkdir d1/d20/d23/d6f/d93/dba 0 2024-03-18T14:50:59.274 INFO:tasks.workunit.client.0.smithi012.stdout:7/553: symlink dd/d13/d23/d2f/d7a/db6/lc0 0 2024-03-18T14:50:59.275 INFO:tasks.workunit.client.0.smithi012.stdout:7/554: chown dd/d13/d23/d33/l63 3 1 2024-03-18T14:50:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:7/555: dread dd/d13/d23/d2f/f3d [0,4194304] 0 2024-03-18T14:50:59.289 INFO:tasks.workunit.client.1.smithi100.stdout:9/227: mkdir da/d46 0 2024-03-18T14:50:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:9/619: dwrite d3/d20/dbf/d77/d52/fc0 [0,4194304] 0 2024-03-18T14:50:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:9/620: write d3/d20/dbf/d66/d69/d7f/d8c/dac/fb8 [188583,75825] 0 2024-03-18T14:50:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:1/527: dwrite d8/db/f41 [0,4194304] 0 2024-03-18T14:50:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:6/539: dwrite d17/d19/f20 [0,4194304] 0 2024-03-18T14:50:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:6/540: write d17/d19/f3c [654129,47652] 0 2024-03-18T14:50:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:6/541: creat d17/d19/d27/d58/d91/fa4 x:0 0 0 2024-03-18T14:50:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:6/542: fdatasync d17/f22 0 2024-03-18T14:50:59.293 INFO:tasks.workunit.client.1.smithi100.stdout:4/210: dwrite f8 [0,4194304] 0 2024-03-18T14:50:59.295 INFO:tasks.workunit.client.1.smithi100.stdout:2/170: rename d3/d13/c39 to d3/d6/d29/c3f 0 2024-03-18T14:50:59.295 INFO:tasks.workunit.client.1.smithi100.stdout:9/228: dread da/d25/f3a [0,4194304] 0 2024-03-18T14:50:59.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/229: write da/d25/f3f [485931,64486] 0 2024-03-18T14:50:59.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/230: chown da/fc 4 1 2024-03-18T14:50:59.303 INFO:tasks.workunit.client.0.smithi012.stdout:4/499: symlink d6/d1a/d30/d46/d6f/la6 0 2024-03-18T14:50:59.303 INFO:tasks.workunit.client.0.smithi012.stdout:4/500: write d6/da/d58/f8d [437990,21174] 0 2024-03-18T14:50:59.303 INFO:tasks.workunit.client.0.smithi012.stdout:4/501: write d6/d19/fa0 [555737,103686] 0 2024-03-18T14:50:59.306 INFO:tasks.workunit.client.1.smithi100.stdout:0/250: mkdir dd/d61 0 2024-03-18T14:50:59.307 INFO:tasks.workunit.client.1.smithi100.stdout:0/251: dread - dd/d20/f4d zero size 2024-03-18T14:50:59.307 INFO:tasks.workunit.client.1.smithi100.stdout:0/252: chown dd/f1c 3200 1 2024-03-18T14:50:59.309 INFO:tasks.workunit.client.0.smithi012.stdout:2/601: rename d0/d10/d18/d33/l66 to d0/d10/d18/d33/d77/lbb 0 2024-03-18T14:50:59.310 INFO:tasks.workunit.client.0.smithi012.stdout:2/602: fsync d0/da2/f67 0 2024-03-18T14:50:59.310 INFO:tasks.workunit.client.1.smithi100.stdout:4/211: dwrite d9/d17/f24 [0,4194304] 0 2024-03-18T14:50:59.314 INFO:tasks.workunit.client.0.smithi012.stdout:0/554: symlink d1/d2d/lbb 0 2024-03-18T14:50:59.315 INFO:tasks.workunit.client.0.smithi012.stdout:3/542: truncate dc/d30/d35/f2c 1443343 0 2024-03-18T14:50:59.315 INFO:tasks.workunit.client.0.smithi012.stdout:3/543: read - dc/d30/f98 zero size 2024-03-18T14:50:59.316 INFO:tasks.workunit.client.1.smithi100.stdout:1/206: truncate d8/f1a 2162537 0 2024-03-18T14:50:59.319 INFO:tasks.workunit.client.1.smithi100.stdout:7/214: mkdir d5/d30/d4d 0 2024-03-18T14:50:59.319 INFO:tasks.workunit.client.1.smithi100.stdout:1/207: dread d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:50:59.320 INFO:tasks.workunit.client.1.smithi100.stdout:1/208: fsync d8/d1e/d11/f21 0 2024-03-18T14:50:59.320 INFO:tasks.workunit.client.0.smithi012.stdout:9/621: truncate d3/f12 612285 0 2024-03-18T14:50:59.322 INFO:tasks.workunit.client.1.smithi100.stdout:5/180: write f10 [258192,820] 0 2024-03-18T14:50:59.327 INFO:tasks.workunit.client.1.smithi100.stdout:3/213: rename d3/d16/c2c to d3/d16/d13/d17/c4d 0 2024-03-18T14:50:59.328 INFO:tasks.workunit.client.0.smithi012.stdout:4/502: mknod d6/d27/ca7 0 2024-03-18T14:50:59.328 INFO:tasks.workunit.client.0.smithi012.stdout:4/503: dread - d6/da/d58/f9e zero size 2024-03-18T14:50:59.329 INFO:tasks.workunit.client.1.smithi100.stdout:2/171: mkdir d3/d13/d40 0 2024-03-18T14:50:59.331 INFO:tasks.workunit.client.1.smithi100.stdout:2/172: write d3/f4 [813994,75283] 0 2024-03-18T14:50:59.331 INFO:tasks.workunit.client.1.smithi100.stdout:2/173: stat d3/fc 0 2024-03-18T14:50:59.331 INFO:tasks.workunit.client.1.smithi100.stdout:2/174: chown d3/d6/f9 106497 1 2024-03-18T14:50:59.331 INFO:tasks.workunit.client.0.smithi012.stdout:8/744: rename d0/d5a/d5e/dd7/d84/dc8 to d0/d5a/d5e/dd7/d84/d54/d62/daf/de8 0 2024-03-18T14:50:59.331 INFO:tasks.workunit.client.0.smithi012.stdout:8/745: chown d0/d5a/f8b 9 1 2024-03-18T14:50:59.333 INFO:tasks.workunit.client.1.smithi100.stdout:1/209: dwrite d8/dd/ff [0,4194304] 0 2024-03-18T14:50:59.334 INFO:tasks.workunit.client.0.smithi012.stdout:0/555: getdents d1/d20/d23/d3b 0 2024-03-18T14:50:59.334 INFO:tasks.workunit.client.0.smithi012.stdout:0/556: dread - d1/d16/d6e/f9b zero size 2024-03-18T14:50:59.337 INFO:tasks.workunit.client.0.smithi012.stdout:3/544: mknod dc/d30/dab/caf 0 2024-03-18T14:50:59.337 INFO:tasks.workunit.client.0.smithi012.stdout:3/545: readlink dc/d30/d35/l89 0 2024-03-18T14:50:59.337 INFO:tasks.workunit.client.1.smithi100.stdout:2/175: dread d3/da/de/f28 [0,4194304] 0 2024-03-18T14:50:59.343 INFO:tasks.workunit.client.1.smithi100.stdout:9/231: truncate da/dd/d29/f2e 6033 0 2024-03-18T14:50:59.343 INFO:tasks.workunit.client.1.smithi100.stdout:9/232: chown da/d2c/c2d 425527 1 2024-03-18T14:50:59.346 INFO:tasks.workunit.client.1.smithi100.stdout:8/230: truncate d4/f15 760534 0 2024-03-18T14:50:59.347 INFO:tasks.workunit.client.1.smithi100.stdout:9/233: dread da/d25/f3a [0,4194304] 0 2024-03-18T14:50:59.347 INFO:tasks.workunit.client.0.smithi012.stdout:9/622: mknod d3/d20/dbf/d77/d52/d65/cc8 0 2024-03-18T14:50:59.348 INFO:tasks.workunit.client.1.smithi100.stdout:2/176: dread d3/f17 [0,4194304] 0 2024-03-18T14:50:59.348 INFO:tasks.workunit.client.1.smithi100.stdout:0/253: mkdir dd/d39/d56/d62 0 2024-03-18T14:50:59.349 INFO:tasks.workunit.client.1.smithi100.stdout:2/177: dread d3/d13/f38 [0,4194304] 0 2024-03-18T14:50:59.349 INFO:tasks.workunit.client.0.smithi012.stdout:6/543: getdents d17/d18 0 2024-03-18T14:50:59.350 INFO:tasks.workunit.client.0.smithi012.stdout:6/544: write d17/d33/f64 [1483069,73240] 0 2024-03-18T14:50:59.352 INFO:tasks.workunit.client.0.smithi012.stdout:4/504: creat d6/d1a/d30/d46/d6f/d7c/fa8 x:0 0 0 2024-03-18T14:50:59.356 INFO:tasks.workunit.client.0.smithi012.stdout:3/546: dwrite dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:50:59.356 INFO:tasks.workunit.client.0.smithi012.stdout:3/547: write dc/f43 [377559,34351] 0 2024-03-18T14:50:59.357 INFO:tasks.workunit.client.0.smithi012.stdout:7/556: rename dd/d13/d23/d76/l35 to dd/d13/d23/d2f/d7a/dab/dbc/lc1 0 2024-03-18T14:50:59.358 INFO:tasks.workunit.client.1.smithi100.stdout:4/212: getdents d9/d3a 0 2024-03-18T14:50:59.358 INFO:tasks.workunit.client.0.smithi012.stdout:4/505: dread d6/f21 [0,4194304] 0 2024-03-18T14:50:59.362 INFO:tasks.workunit.client.0.smithi012.stdout:8/746: symlink d0/d47/d59/dc9/d87/le9 0 2024-03-18T14:50:59.365 INFO:tasks.workunit.client.0.smithi012.stdout:2/603: rmdir d0/d6b 39 2024-03-18T14:50:59.366 INFO:tasks.workunit.client.0.smithi012.stdout:2/604: write d0/d10/d18/d33/f62 [4044261,32845] 0 2024-03-18T14:50:59.368 INFO:tasks.workunit.client.0.smithi012.stdout:0/557: stat d1/d16/d31/c7a 0 2024-03-18T14:50:59.371 INFO:tasks.workunit.client.0.smithi012.stdout:6/545: truncate d17/d26/d36/d4d/d82/f8b 621539 0 2024-03-18T14:50:59.371 INFO:tasks.workunit.client.0.smithi012.stdout:6/546: dread - d17/d38/fa2 zero size 2024-03-18T14:50:59.371 INFO:tasks.workunit.client.0.smithi012.stdout:5/555: sync 2024-03-18T14:50:59.372 INFO:tasks.workunit.client.0.smithi012.stdout:5/556: chown d3/d29/d36/d3d/d86 6 1 2024-03-18T14:50:59.372 INFO:tasks.workunit.client.0.smithi012.stdout:6/547: dread d17/d26/d36/d4d/f7e [0,4194304] 0 2024-03-18T14:50:59.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/254: dwrite dd/d2b/f4c [4194304,4194304] 0 2024-03-18T14:50:59.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/255: write dd/d20/d44/f5e [372394,106878] 0 2024-03-18T14:50:59.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/256: fsync dd/d20/f27 0 2024-03-18T14:50:59.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/257: creat dd/f63 x:0 0 0 2024-03-18T14:50:59.375 INFO:tasks.workunit.client.1.smithi100.stdout:7/215: unlink d5/d30/f4a 0 2024-03-18T14:50:59.377 INFO:tasks.workunit.client.1.smithi100.stdout:5/181: readlink l1c 0 2024-03-18T14:50:59.380 INFO:tasks.workunit.client.1.smithi100.stdout:3/214: unlink d3/d16/d13/c44 0 2024-03-18T14:50:59.382 INFO:tasks.workunit.client.0.smithi012.stdout:3/548: dwrite dc/f44 [0,4194304] 0 2024-03-18T14:50:59.382 INFO:tasks.workunit.client.0.smithi012.stdout:1/528: rename d8/db/d10/d16/l49 to d8/db/d10/d31/d43/d5e/d65/d70/la2 0 2024-03-18T14:50:59.384 INFO:tasks.workunit.client.0.smithi012.stdout:7/557: truncate dd/d13/d23/d2f/f9f 497837 0 2024-03-18T14:50:59.387 INFO:tasks.workunit.client.0.smithi012.stdout:1/529: dread d8/db/d10/f34 [0,4194304] 0 2024-03-18T14:50:59.389 INFO:tasks.workunit.client.0.smithi012.stdout:8/747: creat d0/d47/d59/dc9/d87/fea x:0 0 0 2024-03-18T14:50:59.389 INFO:tasks.workunit.client.0.smithi012.stdout:8/748: chown d0/d5a/d5e/dd7/d84/l5b 2056 1 2024-03-18T14:50:59.389 INFO:tasks.workunit.client.0.smithi012.stdout:8/749: write d0/d5a/d5e/dd7/d84/fc6 [1673688,74221] 0 2024-03-18T14:50:59.389 INFO:tasks.workunit.client.0.smithi012.stdout:8/750: read - d0/d5a/d19/d63/db0/fe1 zero size 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.0.smithi012.stdout:5/557: write d3/d1e/fa4 [2869920,78494] 0 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.0.smithi012.stdout:4/506: dwrite d6/d1a/f2d [0,4194304] 0 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.0.smithi012.stdout:4/507: fdatasync d6/d1a/d30/d46/d6f/d7c/f81 0 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.0.smithi012.stdout:4/508: creat d6/da/d89/d97/fa9 x:0 0 0 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.0.smithi012.stdout:4/509: creat d6/d1a/d30/d46/d6f/d7c/faa x:0 0 0 2024-03-18T14:50:59.390 INFO:tasks.workunit.client.1.smithi100.stdout:1/210: mknod d8/d2a/d37/c3b 0 2024-03-18T14:50:59.392 INFO:tasks.workunit.client.0.smithi012.stdout:2/605: symlink d0/d10/d18/d33/dac/lbc 0 2024-03-18T14:50:59.392 INFO:tasks.workunit.client.0.smithi012.stdout:2/606: readlink d0/d10/d18/l89 0 2024-03-18T14:50:59.392 INFO:tasks.workunit.client.0.smithi012.stdout:2/607: write d0/db4/fb9 [1497154,109786] 0 2024-03-18T14:50:59.394 INFO:tasks.workunit.client.1.smithi100.stdout:9/234: symlink da/db/d44/l47 0 2024-03-18T14:50:59.396 INFO:tasks.workunit.client.1.smithi100.stdout:3/215: write d3/dd/f27 [7305031,83759] 0 2024-03-18T14:50:59.400 INFO:tasks.workunit.client.1.smithi100.stdout:5/182: dwrite d29/f2c [0,4194304] 0 2024-03-18T14:50:59.400 INFO:tasks.workunit.client.1.smithi100.stdout:5/183: read f1b [546933,77375] 0 2024-03-18T14:50:59.403 INFO:tasks.workunit.client.0.smithi012.stdout:0/558: unlink d1/f3 0 2024-03-18T14:50:59.404 INFO:tasks.workunit.client.1.smithi100.stdout:2/178: symlink d3/d1a/l41 0 2024-03-18T14:50:59.407 INFO:tasks.workunit.client.0.smithi012.stdout:6/548: unlink d17/d38/c52 0 2024-03-18T14:50:59.407 INFO:tasks.workunit.client.0.smithi012.stdout:6/549: fdatasync f4 0 2024-03-18T14:50:59.409 INFO:tasks.workunit.client.1.smithi100.stdout:4/213: rmdir d9/d17 39 2024-03-18T14:50:59.409 INFO:tasks.workunit.client.1.smithi100.stdout:4/214: creat d9/d27/f3d x:0 0 0 2024-03-18T14:50:59.415 INFO:tasks.workunit.client.0.smithi012.stdout:7/558: dwrite dd/d13/f15 [0,4194304] 0 2024-03-18T14:50:59.415 INFO:tasks.workunit.client.0.smithi012.stdout:7/559: fsync dd/d13/f75 0 2024-03-18T14:50:59.417 INFO:tasks.workunit.client.0.smithi012.stdout:9/623: rename d3/d20/dbf/d66/d21/d35/l38 to d3/d20/dbf/d66/d69/d7f/lc9 0 2024-03-18T14:50:59.418 INFO:tasks.workunit.client.0.smithi012.stdout:0/559: dread d1/f9c [0,4194304] 0 2024-03-18T14:50:59.419 INFO:tasks.workunit.client.1.smithi100.stdout:0/258: link dd/d31/c46 dd/d20/c64 0 2024-03-18T14:50:59.420 INFO:tasks.workunit.client.1.smithi100.stdout:0/259: write dd/f1d [261320,44272] 0 2024-03-18T14:50:59.422 INFO:tasks.workunit.client.1.smithi100.stdout:7/216: symlink d5/d34/l4e 0 2024-03-18T14:50:59.422 INFO:tasks.workunit.client.1.smithi100.stdout:7/217: fdatasync d5/d14/d16/f1a 0 2024-03-18T14:50:59.422 INFO:tasks.workunit.client.0.smithi012.stdout:1/530: getdents d8/db/d10/d31/d45 0 2024-03-18T14:50:59.428 INFO:tasks.workunit.client.0.smithi012.stdout:5/558: dwrite d3/d29/d36/d3d/d65/f6c [0,4194304] 0 2024-03-18T14:50:59.428 INFO:tasks.workunit.client.0.smithi012.stdout:5/559: fdatasync d3/d1e/fa4 0 2024-03-18T14:50:59.431 INFO:tasks.workunit.client.1.smithi100.stdout:9/235: symlink da/d2c/d2f/l48 0 2024-03-18T14:50:59.436 INFO:tasks.workunit.client.1.smithi100.stdout:3/216: symlink d3/dd/d49/l4e 0 2024-03-18T14:50:59.437 INFO:tasks.workunit.client.0.smithi012.stdout:7/560: write dd/d13/f15 [499781,66122] 0 2024-03-18T14:50:59.438 INFO:tasks.workunit.client.0.smithi012.stdout:4/510: symlink d6/da/d89/d94/da2/lab 0 2024-03-18T14:50:59.442 INFO:tasks.workunit.client.1.smithi100.stdout:4/215: dwrite d9/d17/f34 [0,4194304] 0 2024-03-18T14:50:59.442 INFO:tasks.workunit.client.1.smithi100.stdout:6/196: sync 2024-03-18T14:50:59.443 INFO:tasks.workunit.client.0.smithi012.stdout:6/550: dwrite d17/d38/d51/f85 [0,4194304] 0 2024-03-18T14:50:59.443 INFO:tasks.workunit.client.0.smithi012.stdout:6/551: creat d17/d33/fa5 x:0 0 0 2024-03-18T14:50:59.443 INFO:tasks.workunit.client.0.smithi012.stdout:2/608: symlink d0/d10/d18/d33/d71/lbd 0 2024-03-18T14:50:59.443 INFO:tasks.workunit.client.0.smithi012.stdout:2/609: chown d0/d10/f16 95982327 1 2024-03-18T14:50:59.443 INFO:tasks.workunit.client.1.smithi100.stdout:2/179: dwrite d3/f17 [0,4194304] 0 2024-03-18T14:50:59.445 INFO:tasks.workunit.client.1.smithi100.stdout:5/184: truncate f15 3892968 0 2024-03-18T14:50:59.446 INFO:tasks.workunit.client.1.smithi100.stdout:7/218: dwrite d5/d22/d24/f2d [0,4194304] 0 2024-03-18T14:50:59.447 INFO:tasks.workunit.client.0.smithi012.stdout:4/511: dread d6/d27/f67 [0,4194304] 0 2024-03-18T14:50:59.447 INFO:tasks.workunit.client.0.smithi012.stdout:4/512: chown d6/c8e 176 1 2024-03-18T14:50:59.450 INFO:tasks.workunit.client.1.smithi100.stdout:0/260: mkdir dd/d39/d65 0 2024-03-18T14:50:59.450 INFO:tasks.workunit.client.1.smithi100.stdout:0/261: write dd/d20/f5c [1569813,64402] 0 2024-03-18T14:50:59.453 INFO:tasks.workunit.client.1.smithi100.stdout:8/231: rmdir d4/dc/dd/de/d19 39 2024-03-18T14:50:59.456 INFO:tasks.workunit.client.1.smithi100.stdout:1/211: getdents d8 0 2024-03-18T14:50:59.456 INFO:tasks.workunit.client.1.smithi100.stdout:1/212: truncate d8/d1e/f28 1019275 0 2024-03-18T14:50:59.458 INFO:tasks.workunit.client.1.smithi100.stdout:9/236: link da/dd/d29/c2b da/d2c/c49 0 2024-03-18T14:50:59.458 INFO:tasks.workunit.client.1.smithi100.stdout:9/237: fsync da/fc 0 2024-03-18T14:50:59.458 INFO:tasks.workunit.client.1.smithi100.stdout:9/238: creat da/d2c/d2f/f4a x:0 0 0 2024-03-18T14:50:59.459 INFO:tasks.workunit.client.0.smithi012.stdout:3/549: rename dc/d3c/d59/d86/d9a/f52 to dc/d3c/d4c/d5f/fb0 0 2024-03-18T14:50:59.460 INFO:tasks.workunit.client.1.smithi100.stdout:3/217: getdents d3/d16/d13/d1f/d37 0 2024-03-18T14:50:59.468 INFO:tasks.workunit.client.1.smithi100.stdout:4/216: symlink d9/d27/l3e 0 2024-03-18T14:50:59.469 INFO:tasks.workunit.client.0.smithi012.stdout:0/560: mkdir d1/d20/d4e/d54/d67/dbc 0 2024-03-18T14:50:59.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/180: mkdir d3/d1a/d42 0 2024-03-18T14:50:59.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/181: fdatasync d3/da/de/d20/f23 0 2024-03-18T14:50:59.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/182: chown d3/da 24964 1 2024-03-18T14:50:59.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/183: creat d3/da/f43 x:0 0 0 2024-03-18T14:50:59.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/184: chown f0 3390920 1 2024-03-18T14:50:59.476 INFO:tasks.workunit.client.1.smithi100.stdout:5/185: mknod d29/d2d/c3b 0 2024-03-18T14:50:59.476 INFO:tasks.workunit.client.1.smithi100.stdout:2/185: read d3/da/de/f28 [529132,21086] 0 2024-03-18T14:50:59.476 INFO:tasks.workunit.client.1.smithi100.stdout:2/186: truncate d3/d6/d29/f3d 402316 0 2024-03-18T14:50:59.478 INFO:tasks.workunit.client.0.smithi012.stdout:7/561: link dd/d13/d23/d60/f89 dd/d13/d23/d2f/d7a/dab/fc2 0 2024-03-18T14:50:59.478 INFO:tasks.workunit.client.1.smithi100.stdout:4/217: write d9/f2b [3542047,41069] 0 2024-03-18T14:50:59.483 INFO:tasks.workunit.client.1.smithi100.stdout:7/219: mkdir d5/d34/d4f 0 2024-03-18T14:50:59.485 INFO:tasks.workunit.client.1.smithi100.stdout:9/239: dwrite da/d2c/d2f/f31 [0,4194304] 0 2024-03-18T14:50:59.486 INFO:tasks.workunit.client.1.smithi100.stdout:9/240: truncate da/d2c/d2f/f30 1194734 0 2024-03-18T14:50:59.487 INFO:tasks.workunit.client.0.smithi012.stdout:6/552: symlink d17/la6 0 2024-03-18T14:50:59.489 INFO:tasks.workunit.client.1.smithi100.stdout:0/262: rename dd/d2b/f38 to dd/d39/f66 0 2024-03-18T14:50:59.489 INFO:tasks.workunit.client.1.smithi100.stdout:0/263: stat dd 0 2024-03-18T14:50:59.489 INFO:tasks.workunit.client.1.smithi100.stdout:3/218: dwrite d3/f30 [0,4194304] 0 2024-03-18T14:50:59.490 INFO:tasks.workunit.client.1.smithi100.stdout:5/186: write d29/f2c [4123085,99731] 0 2024-03-18T14:50:59.490 INFO:tasks.workunit.client.1.smithi100.stdout:5/187: fsync f7 0 2024-03-18T14:50:59.492 INFO:tasks.workunit.client.0.smithi012.stdout:2/610: unlink d0/da2/fb6 0 2024-03-18T14:50:59.494 INFO:tasks.workunit.client.1.smithi100.stdout:7/220: read d5/fa [2565943,94099] 0 2024-03-18T14:50:59.497 INFO:tasks.workunit.client.0.smithi012.stdout:7/562: dread dd/d13/d23/d60/f6c [0,4194304] 0 2024-03-18T14:50:59.497 INFO:tasks.workunit.client.0.smithi012.stdout:7/563: chown dd/d13/f57 0 1 2024-03-18T14:50:59.497 INFO:tasks.workunit.client.1.smithi100.stdout:8/232: write d4/f3a [2913279,121195] 0 2024-03-18T14:50:59.501 INFO:tasks.workunit.client.1.smithi100.stdout:1/213: unlink d8/d1e/d11/d2d/l26 0 2024-03-18T14:50:59.503 INFO:tasks.workunit.client.0.smithi012.stdout:2/611: write d0/da2/d59/f8c [3228941,26896] 0 2024-03-18T14:50:59.503 INFO:tasks.workunit.client.0.smithi012.stdout:1/531: rename d8/f7a to d8/db/d10/d16/d54/fa3 0 2024-03-18T14:50:59.504 INFO:tasks.workunit.client.1.smithi100.stdout:7/221: write d5/f35 [3894696,39514] 0 2024-03-18T14:50:59.506 INFO:tasks.workunit.client.0.smithi012.stdout:3/550: creat dc/d30/d35/fb1 x:0 0 0 2024-03-18T14:50:59.507 INFO:tasks.workunit.client.1.smithi100.stdout:6/197: rmdir d6/d12 39 2024-03-18T14:50:59.509 INFO:tasks.workunit.client.0.smithi012.stdout:9/624: mknod d3/d20/dbf/d77/da8/cca 0 2024-03-18T14:50:59.511 INFO:tasks.workunit.client.0.smithi012.stdout:6/553: symlink d17/d19/d27/d6e/la7 0 2024-03-18T14:50:59.513 INFO:tasks.workunit.client.1.smithi100.stdout:9/241: dwrite da/d25/f35 [0,4194304] 0 2024-03-18T14:50:59.514 INFO:tasks.workunit.client.0.smithi012.stdout:7/564: creat dd/d13/d23/d2f/d7a/dab/dbc/fc3 x:0 0 0 2024-03-18T14:50:59.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/222: write d5/d22/d24/f2d [1041116,129761] 0 2024-03-18T14:50:59.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/223: truncate d5/d14/f40 333266 0 2024-03-18T14:50:59.517 INFO:tasks.workunit.client.0.smithi012.stdout:2/612: mknod d0/d10/d18/d33/d77/da0/cbe 0 2024-03-18T14:50:59.520 INFO:tasks.workunit.client.0.smithi012.stdout:0/561: rename d1/d20/d23/d6f/fb0 to d1/d20/d23/d3b/d33/fbd 0 2024-03-18T14:50:59.523 INFO:tasks.workunit.client.0.smithi012.stdout:1/532: creat d8/db/d10/d31/d81/fa4 x:0 0 0 2024-03-18T14:50:59.524 INFO:tasks.workunit.client.1.smithi100.stdout:7/224: dread d5/f10 [0,4194304] 0 2024-03-18T14:50:59.524 INFO:tasks.workunit.client.1.smithi100.stdout:7/225: chown d5/d14/d16/c19 24274685 1 2024-03-18T14:50:59.525 INFO:tasks.workunit.client.1.smithi100.stdout:3/219: dwrite d3/dd/d21/d3e/f32 [0,4194304] 0 2024-03-18T14:50:59.526 INFO:tasks.workunit.client.1.smithi100.stdout:0/264: mkdir dd/d39/d56/d59/d67 0 2024-03-18T14:50:59.527 INFO:tasks.workunit.client.0.smithi012.stdout:3/551: mkdir dc/d3c/d59/db2 0 2024-03-18T14:50:59.527 INFO:tasks.workunit.client.0.smithi012.stdout:3/552: write dc/d30/f51 [2635733,106854] 0 2024-03-18T14:50:59.531 INFO:tasks.workunit.client.0.smithi012.stdout:2/613: dread d0/d10/d18/f73 [0,4194304] 0 2024-03-18T14:50:59.531 INFO:tasks.workunit.client.0.smithi012.stdout:9/625: dwrite d3/f4 [0,4194304] 0 2024-03-18T14:50:59.532 INFO:tasks.workunit.client.0.smithi012.stdout:6/554: mkdir d17/d38/d51/da8 0 2024-03-18T14:50:59.534 INFO:tasks.workunit.client.1.smithi100.stdout:5/188: rename d29/c31 to d29/d2d/d32/c3c 0 2024-03-18T14:50:59.536 INFO:tasks.workunit.client.0.smithi012.stdout:7/565: truncate dd/f5a 2072736 0 2024-03-18T14:50:59.537 INFO:tasks.workunit.client.1.smithi100.stdout:7/226: write d5/d22/d24/d39/f45 [2717886,32311] 0 2024-03-18T14:50:59.537 INFO:tasks.workunit.client.0.smithi012.stdout:1/533: write d8/db/d10/d56/f8a [4100482,35336] 0 2024-03-18T14:50:59.538 INFO:tasks.workunit.client.1.smithi100.stdout:9/242: dwrite da/db/f13 [0,4194304] 0 2024-03-18T14:50:59.538 INFO:tasks.workunit.client.1.smithi100.stdout:9/243: write da/d25/f3f [867810,21412] 0 2024-03-18T14:50:59.538 INFO:tasks.workunit.client.0.smithi012.stdout:0/562: mknod d1/d20/d23/d26/cbe 0 2024-03-18T14:50:59.546 INFO:tasks.workunit.client.1.smithi100.stdout:1/214: mkdir d8/d1e/d11/d27/d3c 0 2024-03-18T14:50:59.546 INFO:tasks.workunit.client.1.smithi100.stdout:1/215: readlink d8/d1e/l13 0 2024-03-18T14:50:59.546 INFO:tasks.workunit.client.1.smithi100.stdout:1/216: chown f5 12980045 1 2024-03-18T14:50:59.548 INFO:tasks.workunit.client.0.smithi012.stdout:3/553: stat dc/d30/d3a/c68 0 2024-03-18T14:50:59.552 INFO:tasks.workunit.client.0.smithi012.stdout:0/563: dread d1/d16/d31/f35 [0,4194304] 0 2024-03-18T14:50:59.553 INFO:tasks.workunit.client.1.smithi100.stdout:6/198: truncate d6/d7/f8 625241 0 2024-03-18T14:50:59.553 INFO:tasks.workunit.client.1.smithi100.stdout:6/199: chown d6/d10/d27/c23 6 1 2024-03-18T14:50:59.555 INFO:tasks.workunit.client.0.smithi012.stdout:9/626: creat d3/d20/dbf/d66/d69/d7f/d8c/dac/fcb x:0 0 0 2024-03-18T14:50:59.556 INFO:tasks.workunit.client.0.smithi012.stdout:9/627: chown d3/d20/dbf/d77/d52/c79 30784743 1 2024-03-18T14:50:59.556 INFO:tasks.workunit.client.0.smithi012.stdout:9/628: chown d3/d9/fa2 0 1 2024-03-18T14:50:59.556 INFO:tasks.workunit.client.0.smithi012.stdout:9/629: dread - d3/d20/dbf/d66/d69/d7f/d8c/dac/fae zero size 2024-03-18T14:50:59.556 INFO:tasks.workunit.client.0.smithi012.stdout:1/534: dwrite d8/db/d10/d16/d54/f89 [0,4194304] 0 2024-03-18T14:50:59.556 INFO:tasks.workunit.client.1.smithi100.stdout:4/218: rmdir d9/d13/d25 39 2024-03-18T14:50:59.558 INFO:tasks.workunit.client.0.smithi012.stdout:7/566: mknod dd/d13/d23/d2f/d53/d8d/cc4 0 2024-03-18T14:50:59.559 INFO:tasks.workunit.client.0.smithi012.stdout:7/567: chown dd/d13/d23/d60/l70 1464156 1 2024-03-18T14:50:59.559 INFO:tasks.workunit.client.0.smithi012.stdout:7/568: dread dd/d13/d23/f55 [0,4194304] 0 2024-03-18T14:50:59.564 INFO:tasks.workunit.client.1.smithi100.stdout:3/220: mkdir d3/d16/d13/d1f/d3f/d4f 0 2024-03-18T14:50:59.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/554: mknod dc/d91/cb3 0 2024-03-18T14:50:59.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/555: chown dc/d30/d35/f65 9 1 2024-03-18T14:50:59.566 INFO:tasks.workunit.client.1.smithi100.stdout:3/221: dread d3/dd/f2a [0,4194304] 0 2024-03-18T14:50:59.568 INFO:tasks.workunit.client.0.smithi012.stdout:7/569: dread dd/d13/f75 [0,4194304] 0 2024-03-18T14:50:59.569 INFO:tasks.workunit.client.1.smithi100.stdout:0/265: rename c4 to dd/d39/d56/c68 0 2024-03-18T14:50:59.569 INFO:tasks.workunit.client.1.smithi100.stdout:0/266: fdatasync dd/f2e 0 2024-03-18T14:50:59.570 INFO:tasks.workunit.client.1.smithi100.stdout:5/189: symlink d29/l3d 0 2024-03-18T14:50:59.575 INFO:tasks.workunit.client.0.smithi012.stdout:1/535: dwrite d8/d1e/d27/f74 [0,4194304] 0 2024-03-18T14:50:59.576 INFO:tasks.workunit.client.1.smithi100.stdout:7/227: creat d5/d14/d16/d21/f50 x:0 0 0 2024-03-18T14:50:59.576 INFO:tasks.workunit.client.1.smithi100.stdout:7/228: stat l2 0 2024-03-18T14:50:59.576 INFO:tasks.workunit.client.1.smithi100.stdout:7/229: fsync d5/d22/d24/d39/f45 0 2024-03-18T14:50:59.576 INFO:tasks.workunit.client.1.smithi100.stdout:7/230: fsync d5/f35 0 2024-03-18T14:50:59.578 INFO:tasks.workunit.client.1.smithi100.stdout:8/233: getdents d4/d3c 0 2024-03-18T14:50:59.580 INFO:tasks.workunit.client.1.smithi100.stdout:1/217: creat d8/f3d x:0 0 0 2024-03-18T14:50:59.582 INFO:tasks.workunit.client.0.smithi012.stdout:0/564: rmdir d1/d20/d23 39 2024-03-18T14:50:59.585 INFO:tasks.workunit.client.1.smithi100.stdout:2/187: truncate d3/f4 887038 0 2024-03-18T14:50:59.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/222: dwrite d3/d16/d13/d17/f33 [0,4194304] 0 2024-03-18T14:50:59.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/223: chown d3/d16/d13/d17/c4d 0 1 2024-03-18T14:50:59.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/224: fsync d3/dd/f28 0 2024-03-18T14:50:59.594 INFO:tasks.workunit.client.0.smithi012.stdout:3/556: rmdir dc/d3c/d72/d8f 0 2024-03-18T14:50:59.596 INFO:tasks.workunit.client.0.smithi012.stdout:7/570: stat dd/d13/d23/d2f/d7a/dab/dbc/lc1 0 2024-03-18T14:50:59.596 INFO:tasks.workunit.client.0.smithi012.stdout:7/571: truncate dd/d13/d23/d2f/d7a/dab/fbe 704800 0 2024-03-18T14:50:59.596 INFO:tasks.workunit.client.0.smithi012.stdout:7/572: getdents dd/d13/db7 0 2024-03-18T14:50:59.601 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:59 smithi012 ceph-mon[29182]: Deploying cephadm binary to smithi100 2024-03-18T14:50:59.601 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:59 smithi012 ceph-mon[29182]: Deploying cephadm binary to smithi012 2024-03-18T14:50:59.601 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:59 smithi012 ceph-mon[29182]: mgrmap e22: smithi100.rwnrcn(active, since 1.04485s) 2024-03-18T14:50:59.601 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:50:59 smithi012 ceph-mon[29182]: pgmap v3: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:50:59.603 INFO:tasks.workunit.client.1.smithi100.stdout:6/200: unlink d6/d12/d2c/f30 0 2024-03-18T14:50:59.603 INFO:tasks.workunit.client.1.smithi100.stdout:6/201: chown d6 0 1 2024-03-18T14:50:59.603 INFO:tasks.workunit.client.0.smithi012.stdout:0/565: dwrite d1/faa [0,4194304] 0 2024-03-18T14:50:59.603 INFO:tasks.workunit.client.1.smithi100.stdout:6/202: dread d6/d7/df/d26/f1c [0,4194304] 0 2024-03-18T14:50:59.608 INFO:tasks.workunit.client.0.smithi012.stdout:1/536: mkdir d8/db/d10/d31/d43/d5e/da5 0 2024-03-18T14:50:59.608 INFO:tasks.workunit.client.0.smithi012.stdout:1/537: write d8/d1e/f63 [1116546,116003] 0 2024-03-18T14:50:59.608 INFO:tasks.workunit.client.0.smithi012.stdout:1/538: write d8/db/d10/d16/da1/d2e/f4b [820616,69684] 0 2024-03-18T14:50:59.610 INFO:tasks.workunit.client.0.smithi012.stdout:2/614: rename d0/d10/d18 to d0/d6b/dbf 0 2024-03-18T14:50:59.610 INFO:tasks.workunit.client.0.smithi012.stdout:2/615: write d0/d6b/dbf/d57/d61/f6d [487043,77971] 0 2024-03-18T14:50:59.612 INFO:tasks.workunit.client.0.smithi012.stdout:3/557: dwrite dc/f44 [0,4194304] 0 2024-03-18T14:50:59.612 INFO:tasks.workunit.client.0.smithi012.stdout:9/630: truncate d3/f12 1597247 0 2024-03-18T14:50:59.618 INFO:tasks.workunit.client.0.smithi012.stdout:0/566: creat d1/d20/d4e/d54/d67/fbf x:0 0 0 2024-03-18T14:50:59.618 INFO:tasks.workunit.client.0.smithi012.stdout:7/573: dwrite dd/d13/d23/d33/d48/f5c [0,4194304] 0 2024-03-18T14:50:59.620 INFO:tasks.workunit.client.0.smithi012.stdout:1/539: creat d8/db/d10/d16/d54/fa6 x:0 0 0 2024-03-18T14:50:59.620 INFO:tasks.workunit.client.0.smithi012.stdout:1/540: dread - d8/db/d10/f60 zero size 2024-03-18T14:50:59.622 INFO:tasks.workunit.client.0.smithi012.stdout:2/616: mknod d0/db4/cc0 0 2024-03-18T14:50:59.625 INFO:tasks.workunit.client.1.smithi100.stdout:0/267: rename dd/l22 to dd/d20/d44/l69 0 2024-03-18T14:50:59.625 INFO:tasks.workunit.client.1.smithi100.stdout:0/268: fdatasync f6 0 2024-03-18T14:50:59.625 INFO:tasks.workunit.client.1.smithi100.stdout:0/269: dread - dd/f43 zero size 2024-03-18T14:50:59.627 INFO:tasks.workunit.client.1.smithi100.stdout:5/190: creat d29/d2d/d39/f3e x:0 0 0 2024-03-18T14:50:59.629 INFO:tasks.workunit.client.0.smithi012.stdout:1/541: write d8/db/d10/f73 [1905505,42972] 0 2024-03-18T14:50:59.630 INFO:tasks.workunit.client.0.smithi012.stdout:3/558: rename dc/f11 to dc/d3c/d72/fb4 0 2024-03-18T14:50:59.630 INFO:tasks.workunit.client.0.smithi012.stdout:3/559: getdents dc/d3c/d59/d86/d9a/d64/d93 0 2024-03-18T14:50:59.632 INFO:tasks.workunit.client.1.smithi100.stdout:7/231: creat d5/d14/d16/d2c/f51 x:0 0 0 2024-03-18T14:50:59.633 INFO:tasks.workunit.client.0.smithi012.stdout:9/631: symlink d3/d20/dbf/lcc 0 2024-03-18T14:50:59.634 INFO:tasks.workunit.client.1.smithi100.stdout:5/191: dread f17 [0,4194304] 0 2024-03-18T14:50:59.635 INFO:tasks.workunit.client.1.smithi100.stdout:8/234: dread d4/f15 [0,4194304] 0 2024-03-18T14:50:59.636 INFO:tasks.workunit.client.1.smithi100.stdout:0/270: dread dd/d2b/f4c [4194304,4194304] 0 2024-03-18T14:50:59.636 INFO:tasks.workunit.client.1.smithi100.stdout:0/271: stat dd/d39/d56 0 2024-03-18T14:50:59.636 INFO:tasks.workunit.client.0.smithi012.stdout:0/567: creat d1/d20/d23/d82/fc0 x:0 0 0 2024-03-18T14:50:59.636 INFO:tasks.workunit.client.0.smithi012.stdout:0/568: stat d1/d16/d19/f56 0 2024-03-18T14:50:59.638 INFO:tasks.workunit.client.1.smithi100.stdout:1/218: unlink d8/d1e/d11/d27/c2f 0 2024-03-18T14:50:59.642 INFO:tasks.workunit.client.1.smithi100.stdout:2/188: creat d3/d6/f44 x:0 0 0 2024-03-18T14:50:59.649 INFO:tasks.workunit.client.1.smithi100.stdout:3/225: mknod d3/d16/d13/d1f/d3f/d4f/c50 0 2024-03-18T14:50:59.651 INFO:tasks.workunit.client.0.smithi012.stdout:2/617: unlink d0/d6b/dbf/l89 0 2024-03-18T14:50:59.651 INFO:tasks.workunit.client.0.smithi012.stdout:2/618: chown d0/d6b/dbf/d57/c7d 48149 1 2024-03-18T14:50:59.651 INFO:tasks.workunit.client.0.smithi012.stdout:2/619: write d0/d10/f58 [2257879,76793] 0 2024-03-18T14:50:59.653 INFO:tasks.workunit.client.1.smithi100.stdout:6/203: rename d6/f15 to d6/d7/f3d 0 2024-03-18T14:50:59.655 INFO:tasks.workunit.client.0.smithi012.stdout:3/560: rename dc/d3c/d59/d86/f96 to dc/d3c/d59/db2/fb5 0 2024-03-18T14:50:59.656 INFO:tasks.workunit.client.1.smithi100.stdout:5/192: mknod d29/d2d/d32/d38/c3f 0 2024-03-18T14:50:59.656 INFO:tasks.workunit.client.1.smithi100.stdout:5/193: write f7 [6139152,5566] 0 2024-03-18T14:50:59.657 INFO:tasks.workunit.client.0.smithi012.stdout:0/569: mkdir d1/d20/d23/d3b/dc1 0 2024-03-18T14:50:59.657 INFO:tasks.workunit.client.0.smithi012.stdout:9/632: dwrite d3/d20/f31 [0,4194304] 0 2024-03-18T14:50:59.658 INFO:tasks.workunit.client.1.smithi100.stdout:0/272: mknod dd/d39/d56/d62/c6a 0 2024-03-18T14:50:59.658 INFO:tasks.workunit.client.1.smithi100.stdout:0/273: fsync dd/f55 0 2024-03-18T14:50:59.658 INFO:tasks.workunit.client.1.smithi100.stdout:0/274: chown f6 16 1 2024-03-18T14:50:59.661 INFO:tasks.workunit.client.1.smithi100.stdout:8/235: dwrite f1 [0,4194304] 0 2024-03-18T14:50:59.661 INFO:tasks.workunit.client.1.smithi100.stdout:8/236: dread d4/f15 [0,4194304] 0 2024-03-18T14:50:59.664 INFO:tasks.workunit.client.1.smithi100.stdout:2/189: mknod d3/da/c45 0 2024-03-18T14:50:59.664 INFO:tasks.workunit.client.1.smithi100.stdout:1/219: dwrite d8/d1e/f36 [0,4194304] 0 2024-03-18T14:50:59.667 INFO:tasks.workunit.client.0.smithi012.stdout:2/620: creat d0/d6b/dbf/d33/d71/fc1 x:0 0 0 2024-03-18T14:50:59.669 INFO:tasks.workunit.client.1.smithi100.stdout:1/220: dread f7 [0,4194304] 0 2024-03-18T14:50:59.669 INFO:tasks.workunit.client.1.smithi100.stdout:3/226: creat d3/d16/d13/d1f/d3f/d4f/f51 x:0 0 0 2024-03-18T14:50:59.672 INFO:tasks.workunit.client.0.smithi012.stdout:2/621: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:50:59.672 INFO:tasks.workunit.client.0.smithi012.stdout:2/622: stat d0/d6b/dbf/d4f 0 2024-03-18T14:50:59.673 INFO:tasks.workunit.client.0.smithi012.stdout:3/561: mknod dc/d30/d35/cb6 0 2024-03-18T14:50:59.674 INFO:tasks.workunit.client.1.smithi100.stdout:0/275: mkdir dd/d39/d56/d62/d6b 0 2024-03-18T14:50:59.676 INFO:tasks.workunit.client.0.smithi012.stdout:2/623: link d0/d6b/dbf/d33/d71/f87 d0/d6b/fc2 0 2024-03-18T14:50:59.676 INFO:tasks.workunit.client.0.smithi012.stdout:2/624: dread - d0/f6e zero size 2024-03-18T14:50:59.677 INFO:tasks.workunit.client.1.smithi100.stdout:8/237: mknod d4/dc/dd/de/d19/c45 0 2024-03-18T14:50:59.677 INFO:tasks.workunit.client.1.smithi100.stdout:8/238: creat d4/dc/d31/f46 x:0 0 0 2024-03-18T14:50:59.677 INFO:tasks.workunit.client.1.smithi100.stdout:8/239: chown d4/dc/dd/de/d19/d2c 2236 1 2024-03-18T14:50:59.678 INFO:tasks.workunit.client.0.smithi012.stdout:3/562: mkdir dc/d30/d3a/db7 0 2024-03-18T14:50:59.680 INFO:tasks.workunit.client.1.smithi100.stdout:2/190: mkdir d3/d6/d29/d36/d46 0 2024-03-18T14:50:59.680 INFO:tasks.workunit.client.1.smithi100.stdout:2/191: creat d3/d6/d29/f47 x:0 0 0 2024-03-18T14:50:59.680 INFO:tasks.workunit.client.1.smithi100.stdout:2/192: read d3/d6/f9 [382329,61147] 0 2024-03-18T14:50:59.681 INFO:tasks.workunit.client.0.smithi012.stdout:2/625: creat d0/d6b/dbf/d33/dac/fc3 x:0 0 0 2024-03-18T14:50:59.683 INFO:tasks.workunit.client.0.smithi012.stdout:3/563: rmdir dc/d30/dab 39 2024-03-18T14:50:59.686 INFO:tasks.workunit.client.1.smithi100.stdout:1/221: mkdir d8/d1e/d11/d27/d3e 0 2024-03-18T14:50:59.687 INFO:tasks.workunit.client.1.smithi100.stdout:3/227: mkdir d3/d16/d13/d1f/d37/d52 0 2024-03-18T14:50:59.689 INFO:tasks.workunit.client.1.smithi100.stdout:0/276: rename dd/l4a to dd/d20/d44/l6c 0 2024-03-18T14:50:59.689 INFO:tasks.workunit.client.1.smithi100.stdout:0/277: write dd/f43 [193775,99368] 0 2024-03-18T14:50:59.691 INFO:tasks.workunit.client.0.smithi012.stdout:9/633: dwrite d3/d9/d5c/f8f [0,4194304] 0 2024-03-18T14:50:59.691 INFO:tasks.workunit.client.0.smithi012.stdout:0/570: dwrite d1/d20/d7b/fa1 [0,4194304] 0 2024-03-18T14:50:59.692 INFO:tasks.workunit.client.1.smithi100.stdout:6/204: dwrite d6/d12/f3a [0,4194304] 0 2024-03-18T14:50:59.692 INFO:tasks.workunit.client.1.smithi100.stdout:5/194: dwrite d29/f2c [0,4194304] 0 2024-03-18T14:50:59.692 INFO:tasks.workunit.client.1.smithi100.stdout:8/240: write d4/f6 [4899212,106312] 0 2024-03-18T14:50:59.695 INFO:tasks.workunit.client.1.smithi100.stdout:2/193: mkdir d3/da/de/d48 0 2024-03-18T14:50:59.696 INFO:tasks.workunit.client.0.smithi012.stdout:2/626: dread d0/da2/f67 [0,4194304] 0 2024-03-18T14:50:59.698 INFO:tasks.workunit.client.1.smithi100.stdout:1/222: mkdir d8/d1e/d11/d27/d3f 0 2024-03-18T14:50:59.698 INFO:tasks.workunit.client.1.smithi100.stdout:1/223: creat d8/d30/f40 x:0 0 0 2024-03-18T14:50:59.702 INFO:tasks.workunit.client.1.smithi100.stdout:3/228: unlink d3/d16/d13/d1f/l2e 0 2024-03-18T14:50:59.703 INFO:tasks.workunit.client.0.smithi012.stdout:3/564: dwrite dc/d30/f99 [0,4194304] 0 2024-03-18T14:50:59.705 INFO:tasks.workunit.client.1.smithi100.stdout:0/278: creat dd/d2b/d54/f6d x:0 0 0 2024-03-18T14:50:59.705 INFO:tasks.workunit.client.1.smithi100.stdout:0/279: chown dd/f11 53 1 2024-03-18T14:50:59.709 INFO:tasks.workunit.client.0.smithi012.stdout:9/634: rename d3/d20/f5f to d3/d20/dbf/d66/d69/d7f/d8c/fcd 0 2024-03-18T14:50:59.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/565: dread dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:50:59.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/566: dread - dc/d3c/d59/d86/d9a/d64/f79 zero size 2024-03-18T14:50:59.712 INFO:tasks.workunit.client.0.smithi012.stdout:0/571: dwrite d1/d20/d4e/d54/f57 [4194304,4194304] 0 2024-03-18T14:50:59.712 INFO:tasks.workunit.client.1.smithi100.stdout:6/205: mknod d6/d7/df/d26/c3e 0 2024-03-18T14:50:59.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/206: dread d6/d10/f22 [0,4194304] 0 2024-03-18T14:50:59.718 INFO:tasks.workunit.client.1.smithi100.stdout:8/241: fdatasync d4/fa 0 2024-03-18T14:50:59.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/627: rename d0/d6b/dbf/d33/dac/fc3 to d0/d6b/dbf/d57/fc4 0 2024-03-18T14:50:59.721 INFO:tasks.workunit.client.1.smithi100.stdout:2/194: mkdir d3/d6/d29/d36/d49 0 2024-03-18T14:50:59.721 INFO:tasks.workunit.client.1.smithi100.stdout:2/195: stat d3/d6 0 2024-03-18T14:50:59.722 INFO:tasks.workunit.client.1.smithi100.stdout:2/196: dread d3/d6/d29/f3d [0,4194304] 0 2024-03-18T14:50:59.722 INFO:tasks.workunit.client.1.smithi100.stdout:2/197: truncate d3/da/de/ff 4651308 0 2024-03-18T14:50:59.722 INFO:tasks.workunit.client.0.smithi012.stdout:9/635: mknod d3/d20/dbf/cce 0 2024-03-18T14:50:59.722 INFO:tasks.workunit.client.0.smithi012.stdout:9/636: dread - d3/d20/fc2 zero size 2024-03-18T14:50:59.725 INFO:tasks.workunit.client.0.smithi012.stdout:3/567: creat dc/d30/dab/fb8 x:0 0 0 2024-03-18T14:50:59.727 INFO:tasks.workunit.client.1.smithi100.stdout:3/229: symlink d3/dd/d21/l53 0 2024-03-18T14:50:59.730 INFO:tasks.workunit.client.1.smithi100.stdout:0/280: mkdir dd/d39/d56/d62/d6e 0 2024-03-18T14:50:59.731 INFO:tasks.workunit.client.1.smithi100.stdout:0/281: fsync dd/d20/f32 0 2024-03-18T14:50:59.732 INFO:tasks.workunit.client.0.smithi012.stdout:0/572: link d1/d16/d19/c2b d1/d20/d23/d26/cc2 0 2024-03-18T14:50:59.734 INFO:tasks.workunit.client.0.smithi012.stdout:2/628: creat d0/d6b/d9f/fc5 x:0 0 0 2024-03-18T14:50:59.736 INFO:tasks.workunit.client.0.smithi012.stdout:0/573: truncate d1/fa 1588623 0 2024-03-18T14:50:59.737 INFO:tasks.workunit.client.0.smithi012.stdout:9/637: dwrite d3/f64 [0,4194304] 0 2024-03-18T14:50:59.740 INFO:tasks.workunit.client.1.smithi100.stdout:5/195: rename l28 to d29/d2d/d32/l40 0 2024-03-18T14:50:59.740 INFO:tasks.workunit.client.1.smithi100.stdout:5/196: read - d29/f37 zero size 2024-03-18T14:50:59.743 INFO:tasks.workunit.client.1.smithi100.stdout:8/242: mkdir d4/dc/dd/de/d21/d23/d47 0 2024-03-18T14:50:59.744 INFO:tasks.workunit.client.0.smithi012.stdout:9/638: rename d3/d20/dbf/d66/d69/d7f/f9b to d3/d20/dbf/d77/d52/d95/fcf 0 2024-03-18T14:50:59.744 INFO:tasks.workunit.client.0.smithi012.stdout:9/639: dread - d3/d20/dbf/d77/d52/d65/f86 zero size 2024-03-18T14:50:59.746 INFO:tasks.workunit.client.0.smithi012.stdout:2/629: dread d0/d3c/d40/d47/f49 [0,4194304] 0 2024-03-18T14:50:59.750 INFO:tasks.workunit.client.0.smithi012.stdout:2/630: dread d0/d6b/dbf/f3a [0,4194304] 0 2024-03-18T14:50:59.750 INFO:tasks.workunit.client.0.smithi012.stdout:2/631: read d0/d6b/dbf/d33/d77/f86 [2264151,48034] 0 2024-03-18T14:50:59.752 INFO:tasks.workunit.client.0.smithi012.stdout:0/574: dwrite d1/f5f [0,4194304] 0 2024-03-18T14:50:59.752 INFO:tasks.workunit.client.0.smithi012.stdout:0/575: chown d1/d20/d7b/l11 516362 1 2024-03-18T14:50:59.756 INFO:tasks.workunit.client.0.smithi012.stdout:2/632: dread d0/d6b/dbf/d4f/f6f [0,4194304] 0 2024-03-18T14:50:59.756 INFO:tasks.workunit.client.0.smithi012.stdout:2/633: fsync d0/da2/f5c 0 2024-03-18T14:50:59.763 INFO:tasks.workunit.client.0.smithi012.stdout:0/576: dread d1/d16/d19/d8d/fab [0,4194304] 0 2024-03-18T14:50:59.771 INFO:tasks.workunit.client.0.smithi012.stdout:0/577: dread d1/d20/f2e [4194304,4194304] 0 2024-03-18T14:50:59.771 INFO:tasks.workunit.client.0.smithi012.stdout:0/578: fdatasync d1/d20/d23/d6f/f75 0 2024-03-18T14:50:59.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:59 smithi100 ceph-mon[37807]: Deploying cephadm binary to smithi100 2024-03-18T14:50:59.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:59 smithi100 ceph-mon[37807]: Deploying cephadm binary to smithi012 2024-03-18T14:50:59.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:59 smithi100 ceph-mon[37807]: mgrmap e22: smithi100.rwnrcn(active, since 1.04485s) 2024-03-18T14:50:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:50:59 smithi100 ceph-mon[37807]: pgmap v3: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:50:59.796 INFO:tasks.workunit.client.1.smithi100.stdout:3/230: creat d3/d16/d13/d1f/d37/f54 x:0 0 0 2024-03-18T14:50:59.798 INFO:tasks.workunit.client.1.smithi100.stdout:0/282: symlink dd/d39/d65/l6f 0 2024-03-18T14:50:59.803 INFO:tasks.workunit.client.0.smithi012.stdout:0/579: symlink d1/d16/d19/d8d/lc3 0 2024-03-18T14:50:59.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/224: rename d8/dd/ff to d8/d2a/f41 0 2024-03-18T14:50:59.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/225: write f7 [3956386,83816] 0 2024-03-18T14:50:59.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/226: fdatasync d8/d30/f35 0 2024-03-18T14:50:59.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/227: readlink d8/d1e/d11/d27/l2e 0 2024-03-18T14:50:59.806 INFO:tasks.workunit.client.0.smithi012.stdout:9/640: mknod d3/cd0 0 2024-03-18T14:50:59.808 INFO:tasks.workunit.client.1.smithi100.stdout:5/197: getdents d29/d2d/d32/d38 0 2024-03-18T14:50:59.809 INFO:tasks.workunit.client.0.smithi012.stdout:2/634: rmdir d0/d6c 39 2024-03-18T14:50:59.811 INFO:tasks.workunit.client.1.smithi100.stdout:8/243: mkdir d4/dc/dd/d48 0 2024-03-18T14:50:59.813 INFO:tasks.workunit.client.0.smithi012.stdout:9/641: mkdir d3/d20/dbf/d66/d69/d7f/d8c/dac/dd1 0 2024-03-18T14:50:59.813 INFO:tasks.workunit.client.0.smithi012.stdout:9/642: stat d3/d20/dbf 0 2024-03-18T14:50:59.814 INFO:tasks.workunit.client.0.smithi012.stdout:0/580: write d1/d16/d31/f50 [6498737,120422] 0 2024-03-18T14:50:59.815 INFO:tasks.workunit.client.0.smithi012.stdout:9/643: read d3/d20/dbf/d77/d52/f6d [4718745,12337] 0 2024-03-18T14:50:59.817 INFO:tasks.workunit.client.1.smithi100.stdout:3/231: creat d3/dd/d21/f55 x:0 0 0 2024-03-18T14:50:59.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/283: dwrite dd/d20/f5f [0,4194304] 0 2024-03-18T14:50:59.818 INFO:tasks.workunit.client.0.smithi012.stdout:2/635: creat d0/d6b/dbf/d33/d80/fc6 x:0 0 0 2024-03-18T14:50:59.820 INFO:tasks.workunit.client.1.smithi100.stdout:6/207: rename d6/c17 to d6/d7/df/c3f 0 2024-03-18T14:50:59.821 INFO:tasks.workunit.client.1.smithi100.stdout:6/208: dread d6/d7/df/d26/f1c [0,4194304] 0 2024-03-18T14:50:59.821 INFO:tasks.workunit.client.0.smithi012.stdout:0/581: symlink d1/d16/d6e/lc4 0 2024-03-18T14:50:59.825 INFO:tasks.workunit.client.0.smithi012.stdout:9/644: creat d3/d20/dbf/d77/d52/d99/da0/fd2 x:0 0 0 2024-03-18T14:50:59.825 INFO:tasks.workunit.client.1.smithi100.stdout:6/209: dread d6/d7/df/f1e [0,4194304] 0 2024-03-18T14:50:59.826 INFO:tasks.workunit.client.1.smithi100.stdout:9/244: sync 2024-03-18T14:50:59.827 INFO:tasks.workunit.client.1.smithi100.stdout:9/245: fsync da/f16 0 2024-03-18T14:50:59.827 INFO:tasks.workunit.client.1.smithi100.stdout:1/228: dwrite f5 [0,4194304] 0 2024-03-18T14:50:59.827 INFO:tasks.workunit.client.1.smithi100.stdout:1/229: fdatasync d8/d1e/d11/f1b 0 2024-03-18T14:50:59.827 INFO:tasks.workunit.client.1.smithi100.stdout:1/230: creat d8/d30/f42 x:0 0 0 2024-03-18T14:50:59.828 INFO:tasks.workunit.client.0.smithi012.stdout:2/636: dread d0/f97 [0,4194304] 0 2024-03-18T14:50:59.828 INFO:tasks.workunit.client.0.smithi012.stdout:0/582: symlink d1/d16/d79/lc5 0 2024-03-18T14:50:59.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/244: creat d4/d3c/f49 x:0 0 0 2024-03-18T14:50:59.831 INFO:tasks.workunit.client.1.smithi100.stdout:4/219: sync 2024-03-18T14:50:59.831 INFO:tasks.workunit.client.1.smithi100.stdout:4/220: fdatasync d9/f22 0 2024-03-18T14:50:59.833 INFO:tasks.workunit.client.1.smithi100.stdout:7/232: sync 2024-03-18T14:50:59.835 INFO:tasks.workunit.client.0.smithi012.stdout:2/637: write d0/d6b/dbf/f5b [1615625,115843] 0 2024-03-18T14:50:59.835 INFO:tasks.workunit.client.0.smithi012.stdout:2/638: fsync d0/d3c/fb3 0 2024-03-18T14:50:59.838 INFO:tasks.workunit.client.1.smithi100.stdout:2/198: rename d3/d1a/d42 to d3/d6/d29/d36/d4a 0 2024-03-18T14:50:59.838 INFO:tasks.workunit.client.1.smithi100.stdout:9/246: write da/db/f1a [3389870,73764] 0 2024-03-18T14:50:59.839 INFO:tasks.workunit.client.0.smithi012.stdout:0/583: getdents d1/d16/d19/d8d 0 2024-03-18T14:50:59.839 INFO:tasks.workunit.client.0.smithi012.stdout:0/584: truncate d1/fa 2485018 0 2024-03-18T14:50:59.839 INFO:tasks.workunit.client.1.smithi100.stdout:5/198: dread fa [0,4194304] 0 2024-03-18T14:50:59.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/639: rename d0/d6b/f9c to d0/d6c/db7/fc7 0 2024-03-18T14:50:59.842 INFO:tasks.workunit.client.0.smithi012.stdout:0/585: dread d1/d20/d23/d26/fa4 [0,4194304] 0 2024-03-18T14:50:59.842 INFO:tasks.workunit.client.0.smithi012.stdout:0/586: creat d1/d20/d4e/d54/fc6 x:0 0 0 2024-03-18T14:50:59.843 INFO:tasks.workunit.client.0.smithi012.stdout:0/587: chown d1/d20/d7b 973715 1 2024-03-18T14:50:59.843 INFO:tasks.workunit.client.1.smithi100.stdout:6/210: creat d6/d7/f40 x:0 0 0 2024-03-18T14:50:59.844 INFO:tasks.workunit.client.0.smithi012.stdout:9/645: dwrite d3/d20/dbf/d66/fba [0,4194304] 0 2024-03-18T14:50:59.844 INFO:tasks.workunit.client.0.smithi012.stdout:9/646: truncate d3/d20/dbf/d66/f51 1316982 0 2024-03-18T14:50:59.846 INFO:tasks.workunit.client.0.smithi012.stdout:9/647: dread d3/d9/f39 [4194304,4194304] 0 2024-03-18T14:50:59.846 INFO:tasks.workunit.client.0.smithi012.stdout:9/648: fsync d3/d9/f6c 0 2024-03-18T14:50:59.846 INFO:tasks.workunit.client.0.smithi012.stdout:2/640: dread d0/da2/d59/f8c [0,4194304] 0 2024-03-18T14:50:59.846 INFO:tasks.workunit.client.0.smithi012.stdout:2/641: write d0/d6b/dbf/d33/f62 [2792391,102175] 0 2024-03-18T14:50:59.847 INFO:tasks.workunit.client.1.smithi100.stdout:1/231: mknod d8/d1e/d11/d27/d3f/c43 0 2024-03-18T14:50:59.850 INFO:tasks.workunit.client.1.smithi100.stdout:8/245: rmdir d4/dc/dd/de/d21/d23/d25 39 2024-03-18T14:50:59.854 INFO:tasks.workunit.client.0.smithi012.stdout:2/642: write d0/d6b/dbf/f73 [541528,89646] 0 2024-03-18T14:50:59.854 INFO:tasks.workunit.client.0.smithi012.stdout:2/643: creat d0/d6b/dbf/d33/d80/fc8 x:0 0 0 2024-03-18T14:50:59.854 INFO:tasks.workunit.client.0.smithi012.stdout:2/644: chown d0/d6b/dbf/d33/d80 9386 1 2024-03-18T14:50:59.854 INFO:tasks.workunit.client.0.smithi012.stdout:0/588: write d1/f5f [4071661,28029] 0 2024-03-18T14:50:59.855 INFO:tasks.workunit.client.1.smithi100.stdout:4/221: creat d9/d13/f3f x:0 0 0 2024-03-18T14:50:59.855 INFO:tasks.workunit.client.0.smithi012.stdout:9/649: symlink d3/d20/dbf/d66/d6e/ld3 0 2024-03-18T14:50:59.855 INFO:tasks.workunit.client.1.smithi100.stdout:3/232: getdents d3/d16/d13/d1f/d37 0 2024-03-18T14:50:59.856 INFO:tasks.workunit.client.1.smithi100.stdout:0/284: rmdir dd/d20/d44 39 2024-03-18T14:50:59.858 INFO:tasks.workunit.client.0.smithi012.stdout:0/589: dread d1/d2d/f62 [0,4194304] 0 2024-03-18T14:50:59.858 INFO:tasks.workunit.client.0.smithi012.stdout:0/590: fsync d1/d20/d23/d7f/da7/fb6 0 2024-03-18T14:50:59.860 INFO:tasks.workunit.client.1.smithi100.stdout:1/232: write d8/d1e/d11/d2d/f25 [571136,104625] 0 2024-03-18T14:50:59.861 INFO:tasks.workunit.client.1.smithi100.stdout:7/233: rename d5/d14/d16/f2b to d5/f52 0 2024-03-18T14:50:59.863 INFO:tasks.workunit.client.1.smithi100.stdout:2/199: symlink d3/d6/d29/d36/d49/l4b 0 2024-03-18T14:50:59.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/285: write dd/f41 [1923388,38782] 0 2024-03-18T14:50:59.866 INFO:tasks.workunit.client.1.smithi100.stdout:9/247: mkdir da/dd/d3c/d4b 0 2024-03-18T14:50:59.866 INFO:tasks.workunit.client.1.smithi100.stdout:9/248: write da/d2c/d2f/f30 [2076092,129880] 0 2024-03-18T14:50:59.867 INFO:tasks.workunit.client.1.smithi100.stdout:6/211: symlink d6/d7/l41 0 2024-03-18T14:50:59.867 INFO:tasks.workunit.client.1.smithi100.stdout:6/212: dread - d6/d10/d2b/f2d zero size 2024-03-18T14:50:59.868 INFO:tasks.workunit.client.1.smithi100.stdout:6/213: write d6/fe [48686,56857] 0 2024-03-18T14:50:59.870 INFO:tasks.workunit.client.1.smithi100.stdout:8/246: mknod d4/dc/dd/de/d21/d23/d41/c4a 0 2024-03-18T14:50:59.872 INFO:tasks.workunit.client.1.smithi100.stdout:4/222: mknod d9/c40 0 2024-03-18T14:50:59.872 INFO:tasks.workunit.client.1.smithi100.stdout:4/223: read d9/d17/f1c [5227947,19708] 0 2024-03-18T14:50:59.875 INFO:tasks.workunit.client.1.smithi100.stdout:3/233: creat d3/dd/f56 x:0 0 0 2024-03-18T14:50:59.875 INFO:tasks.workunit.client.1.smithi100.stdout:3/234: write d3/d16/d13/d17/f1a [750996,58518] 0 2024-03-18T14:50:59.875 INFO:tasks.workunit.client.1.smithi100.stdout:3/235: chown d3/dd/d21/f55 62791 1 2024-03-18T14:50:59.879 INFO:tasks.workunit.client.1.smithi100.stdout:1/233: truncate d8/d30/f35 41784 0 2024-03-18T14:50:59.881 INFO:tasks.workunit.client.0.smithi012.stdout:9/650: link d3/d20/f31 d3/d49/fd4 0 2024-03-18T14:50:59.882 INFO:tasks.workunit.client.1.smithi100.stdout:2/200: mknod d3/d6/d29/d36/d46/c4c 0 2024-03-18T14:50:59.883 INFO:tasks.workunit.client.0.smithi012.stdout:5/560: sync 2024-03-18T14:50:59.884 INFO:tasks.workunit.client.0.smithi012.stdout:6/555: sync 2024-03-18T14:50:59.884 INFO:tasks.workunit.client.0.smithi012.stdout:4/513: sync 2024-03-18T14:50:59.884 INFO:tasks.workunit.client.0.smithi012.stdout:8/751: sync 2024-03-18T14:50:59.884 INFO:tasks.workunit.client.1.smithi100.stdout:0/286: symlink dd/d20/l70 0 2024-03-18T14:50:59.884 INFO:tasks.workunit.client.1.smithi100.stdout:0/287: truncate dd/f5b 427062 0 2024-03-18T14:50:59.885 INFO:tasks.workunit.client.0.smithi012.stdout:0/591: creat d1/d16/d6e/fc7 x:0 0 0 2024-03-18T14:50:59.886 INFO:tasks.workunit.client.1.smithi100.stdout:9/249: creat da/db/f4c x:0 0 0 2024-03-18T14:50:59.887 INFO:tasks.workunit.client.0.smithi012.stdout:2/645: getdents d0/d3c/d40/d47/d85 0 2024-03-18T14:50:59.887 INFO:tasks.workunit.client.0.smithi012.stdout:2/646: chown d0/d10/f13 210251 1 2024-03-18T14:50:59.891 INFO:tasks.workunit.client.1.smithi100.stdout:7/234: dwrite d5/f52 [4194304,4194304] 0 2024-03-18T14:50:59.891 INFO:tasks.workunit.client.1.smithi100.stdout:6/214: unlink d6/d10/f38 0 2024-03-18T14:50:59.892 INFO:tasks.workunit.client.1.smithi100.stdout:2/201: read d3/fc [576301,36629] 0 2024-03-18T14:50:59.892 INFO:tasks.workunit.client.1.smithi100.stdout:8/247: rmdir d4/dc/dd/de/d21/d23/d25 39 2024-03-18T14:50:59.894 INFO:tasks.workunit.client.1.smithi100.stdout:4/224: mkdir d9/d13/d25/d41 0 2024-03-18T14:50:59.896 INFO:tasks.workunit.client.0.smithi012.stdout:5/561: mknod d3/d1e/cb9 0 2024-03-18T14:50:59.897 INFO:tasks.workunit.client.1.smithi100.stdout:9/250: unlink da/dd/c33 0 2024-03-18T14:50:59.898 INFO:tasks.workunit.client.1.smithi100.stdout:1/234: dwrite d8/d1e/d11/f33 [0,4194304] 0 2024-03-18T14:50:59.899 INFO:tasks.workunit.client.0.smithi012.stdout:8/752: mknod d0/d5a/ceb 0 2024-03-18T14:50:59.901 INFO:tasks.workunit.client.0.smithi012.stdout:4/514: symlink d6/da/d89/d97/lac 0 2024-03-18T14:50:59.902 INFO:tasks.workunit.client.1.smithi100.stdout:0/288: dread f3 [0,4194304] 0 2024-03-18T14:50:59.903 INFO:tasks.workunit.client.0.smithi012.stdout:8/753: dread d0/d5a/d19/dad/f3c [0,4194304] 0 2024-03-18T14:50:59.904 INFO:tasks.workunit.client.1.smithi100.stdout:7/235: symlink d5/d14/d16/l53 0 2024-03-18T14:50:59.906 INFO:tasks.workunit.client.0.smithi012.stdout:0/592: rename d1/d20/c55 to d1/d20/d23/d6f/d93/dba/cc8 0 2024-03-18T14:50:59.906 INFO:tasks.workunit.client.0.smithi012.stdout:0/593: chown d1/d16/d79 2 1 2024-03-18T14:50:59.909 INFO:tasks.workunit.client.0.smithi012.stdout:4/515: read d6/d27/f67 [256659,81283] 0 2024-03-18T14:50:59.910 INFO:tasks.workunit.client.0.smithi012.stdout:5/562: mkdir d3/d29/d9d/dba 0 2024-03-18T14:50:59.910 INFO:tasks.workunit.client.1.smithi100.stdout:6/215: link d6/d7/df/d26/c3e d6/d7/c42 0 2024-03-18T14:50:59.910 INFO:tasks.workunit.client.1.smithi100.stdout:7/236: dread d5/d22/d24/f2d [0,4194304] 0 2024-03-18T14:50:59.911 INFO:tasks.workunit.client.1.smithi100.stdout:6/216: truncate d6/d12/f1f 5168572 0 2024-03-18T14:50:59.911 INFO:tasks.workunit.client.1.smithi100.stdout:6/217: chown d6/d7/df/d26/d32/f33 55415605 1 2024-03-18T14:50:59.914 INFO:tasks.workunit.client.1.smithi100.stdout:2/202: symlink d3/d1a/l4d 0 2024-03-18T14:50:59.919 INFO:tasks.workunit.client.0.smithi012.stdout:6/556: dwrite d17/d33/f88 [0,4194304] 0 2024-03-18T14:50:59.919 INFO:tasks.workunit.client.0.smithi012.stdout:6/557: fsync d17/d19/d27/d58/d91/fa4 0 2024-03-18T14:50:59.919 INFO:tasks.workunit.client.0.smithi012.stdout:8/754: dread d0/fc [0,4194304] 0 2024-03-18T14:50:59.920 INFO:tasks.workunit.client.0.smithi012.stdout:9/651: write d3/d20/dbf/d77/da8/f76 [155654,17947] 0 2024-03-18T14:50:59.921 INFO:tasks.workunit.client.0.smithi012.stdout:0/594: getdents d1/d20/d23/d6f/d93 0 2024-03-18T14:50:59.925 INFO:tasks.workunit.client.0.smithi012.stdout:8/755: dread d0/d5a/d5e/ddf/f44 [0,4194304] 0 2024-03-18T14:50:59.925 INFO:tasks.workunit.client.0.smithi012.stdout:8/756: read - d0/d5a/d5e/dd7/d84/d54/f96 zero size 2024-03-18T14:50:59.925 INFO:tasks.workunit.client.0.smithi012.stdout:8/757: creat d0/d15/fec x:0 0 0 2024-03-18T14:50:59.926 INFO:tasks.workunit.client.1.smithi100.stdout:3/236: rename d3/dd/d21/f35 to d3/f57 0 2024-03-18T14:50:59.926 INFO:tasks.workunit.client.1.smithi100.stdout:3/237: dread - d3/d16/d13/f3b zero size 2024-03-18T14:50:59.929 INFO:tasks.workunit.client.0.smithi012.stdout:4/516: rename d6/d27/f70 to d6/d19/fad 0 2024-03-18T14:50:59.933 INFO:tasks.workunit.client.0.smithi012.stdout:5/563: unlink d3/d29/d36/d3d/d73/f58 0 2024-03-18T14:50:59.933 INFO:tasks.workunit.client.0.smithi012.stdout:5/564: readlink d3/l52 0 2024-03-18T14:50:59.933 INFO:tasks.workunit.client.1.smithi100.stdout:8/248: read d4/f9 [320325,10134] 0 2024-03-18T14:50:59.936 INFO:tasks.workunit.client.1.smithi100.stdout:0/289: mkdir dd/d39/d56/d71 0 2024-03-18T14:50:59.939 INFO:tasks.workunit.client.1.smithi100.stdout:4/225: dwrite d9/d17/f23 [0,4194304] 0 2024-03-18T14:50:59.940 INFO:tasks.workunit.client.1.smithi100.stdout:4/226: chown d9/c2a 0 1 2024-03-18T14:50:59.940 INFO:tasks.workunit.client.0.smithi012.stdout:9/652: getdents d3/d20/dbf/d77/d52/d95 0 2024-03-18T14:50:59.940 INFO:tasks.workunit.client.0.smithi012.stdout:9/653: dread - d3/d20/dbf/d66/d69/d7f/d8c/dac/fcb zero size 2024-03-18T14:50:59.940 INFO:tasks.workunit.client.1.smithi100.stdout:9/251: rename da/db/d44 to da/d46/d4d 0 2024-03-18T14:50:59.942 INFO:tasks.workunit.client.1.smithi100.stdout:3/238: unlink d3/d16/d13/d17/c42 0 2024-03-18T14:50:59.943 INFO:tasks.workunit.client.1.smithi100.stdout:8/249: stat d4/l26 0 2024-03-18T14:50:59.945 INFO:tasks.workunit.client.0.smithi012.stdout:8/758: mknod d0/d5a/d5e/dd7/d84/d54/d62/daf/ced 0 2024-03-18T14:50:59.947 INFO:tasks.workunit.client.1.smithi100.stdout:0/290: creat dd/d61/f72 x:0 0 0 2024-03-18T14:50:59.948 INFO:tasks.workunit.client.0.smithi012.stdout:4/517: symlink d6/d1a/d64/lae 0 2024-03-18T14:50:59.948 INFO:tasks.workunit.client.0.smithi012.stdout:4/518: write d6/d1a/d30/d46/d6f/d7c/faa [409778,57805] 0 2024-03-18T14:50:59.949 INFO:tasks.workunit.client.0.smithi012.stdout:5/565: mknod d3/d29/d36/d9b/cbb 0 2024-03-18T14:50:59.949 INFO:tasks.workunit.client.0.smithi012.stdout:5/566: chown d3/d1e/c20 3687739 1 2024-03-18T14:50:59.949 INFO:tasks.workunit.client.0.smithi012.stdout:5/567: chown d3/d1e/fa4 12 1 2024-03-18T14:50:59.949 INFO:tasks.workunit.client.0.smithi012.stdout:5/568: fdatasync d3/d1e/f1a 0 2024-03-18T14:50:59.949 INFO:tasks.workunit.client.0.smithi012.stdout:5/569: chown d3/d29/d36/d9b/d59/la3 3657545 1 2024-03-18T14:50:59.950 INFO:tasks.workunit.client.0.smithi012.stdout:7/574: sync 2024-03-18T14:50:59.951 INFO:tasks.workunit.client.0.smithi012.stdout:1/542: sync 2024-03-18T14:50:59.951 INFO:tasks.workunit.client.0.smithi012.stdout:7/575: readlink dd/d13/d23/lb4 0 2024-03-18T14:50:59.951 INFO:tasks.workunit.client.0.smithi012.stdout:1/543: dread - d8/db/d10/d16/f98 zero size 2024-03-18T14:50:59.951 INFO:tasks.workunit.client.0.smithi012.stdout:9/654: creat d3/d9/fd5 x:0 0 0 2024-03-18T14:50:59.956 INFO:tasks.workunit.client.1.smithi100.stdout:3/239: rmdir d3/d16/d13/d1f/d37/d52 0 2024-03-18T14:50:59.957 INFO:tasks.workunit.client.0.smithi012.stdout:7/576: dread dd/d13/d23/d2f/f9f [0,4194304] 0 2024-03-18T14:50:59.957 INFO:tasks.workunit.client.0.smithi012.stdout:7/577: write dd/d13/d23/d33/f78 [3448932,95194] 0 2024-03-18T14:50:59.957 INFO:tasks.workunit.client.1.smithi100.stdout:8/250: readlink d4/l18 0 2024-03-18T14:50:59.958 INFO:tasks.workunit.client.0.smithi012.stdout:1/544: symlink d8/db/d10/d31/la7 0 2024-03-18T14:50:59.958 INFO:tasks.workunit.client.1.smithi100.stdout:0/291: symlink dd/d20/d44/l73 0 2024-03-18T14:50:59.959 INFO:tasks.workunit.client.0.smithi012.stdout:9/655: creat d3/d20/dbf/d66/d6e/dc6/fd6 x:0 0 0 2024-03-18T14:50:59.960 INFO:tasks.workunit.client.1.smithi100.stdout:8/251: mknod d4/dc/dd/de/d19/d2c/c4b 0 2024-03-18T14:50:59.960 INFO:tasks.workunit.client.1.smithi100.stdout:8/252: stat d4/dc/dd/de/d19/l1b 0 2024-03-18T14:50:59.960 INFO:tasks.workunit.client.0.smithi012.stdout:4/519: truncate d6/d1a/f31 1126977 0 2024-03-18T14:50:59.961 INFO:tasks.workunit.client.1.smithi100.stdout:0/292: creat dd/d39/d65/f74 x:0 0 0 2024-03-18T14:50:59.962 INFO:tasks.workunit.client.0.smithi012.stdout:5/570: getdents d3/d15 0 2024-03-18T14:50:59.962 INFO:tasks.workunit.client.0.smithi012.stdout:5/571: fdatasync d3/f22 0 2024-03-18T14:50:59.964 INFO:tasks.workunit.client.0.smithi012.stdout:7/578: link dd/d13/d23/d60/l66 dd/d13/d23/d2f/d7a/db6/lc5 0 2024-03-18T14:50:59.964 INFO:tasks.workunit.client.1.smithi100.stdout:0/293: mknod dd/d61/c75 0 2024-03-18T14:50:59.965 INFO:tasks.workunit.client.0.smithi012.stdout:1/545: rmdir d8/d1e/d27 39 2024-03-18T14:50:59.966 INFO:tasks.workunit.client.0.smithi012.stdout:9/656: rename d3/d20/dbf/d66/d21/d35/d48/ca1 to d3/d20/dbf/d77/d52/d99/da0/cd7 0 2024-03-18T14:50:59.967 INFO:tasks.workunit.client.1.smithi100.stdout:0/294: mknod dd/c76 0 2024-03-18T14:50:59.968 INFO:tasks.workunit.client.0.smithi012.stdout:4/520: mkdir d6/d1a/d98/daf 0 2024-03-18T14:50:59.970 INFO:tasks.workunit.client.0.smithi012.stdout:7/579: mkdir dd/d13/d23/d2f/d53/dc6 0 2024-03-18T14:50:59.971 INFO:tasks.workunit.client.0.smithi012.stdout:1/546: creat d8/db/d10/d16/fa8 x:0 0 0 2024-03-18T14:50:59.972 INFO:tasks.workunit.client.1.smithi100.stdout:3/240: write d3/dd/f27 [3336261,13422] 0 2024-03-18T14:50:59.972 INFO:tasks.workunit.client.1.smithi100.stdout:3/241: creat d3/d16/f58 x:0 0 0 2024-03-18T14:50:59.974 INFO:tasks.workunit.client.0.smithi012.stdout:5/572: rename d3/d15/f19 to d3/d29/d35/fbc 0 2024-03-18T14:50:59.974 INFO:tasks.workunit.client.0.smithi012.stdout:9/657: stat d3/l8a 0 2024-03-18T14:50:59.976 INFO:tasks.workunit.client.0.smithi012.stdout:1/547: symlink d8/db/la9 0 2024-03-18T14:50:59.979 INFO:tasks.workunit.client.0.smithi012.stdout:9/658: creat d3/d20/dbf/d77/fd8 x:0 0 0 2024-03-18T14:50:59.979 INFO:tasks.workunit.client.0.smithi012.stdout:7/580: write dd/d13/d23/d60/f62 [3756965,120705] 0 2024-03-18T14:50:59.982 INFO:tasks.workunit.client.0.smithi012.stdout:1/548: unlink d8/db/f15 0 2024-03-18T14:50:59.982 INFO:tasks.workunit.client.0.smithi012.stdout:1/549: read - d8/db/d10/f7f zero size 2024-03-18T14:50:59.983 INFO:tasks.workunit.client.0.smithi012.stdout:1/550: chown d8/db/d10/d31/d81/f8c 20577 1 2024-03-18T14:50:59.986 INFO:tasks.workunit.client.0.smithi012.stdout:9/659: mknod d3/d20/dbf/d66/d69/cd9 0 2024-03-18T14:50:59.986 INFO:tasks.workunit.client.0.smithi012.stdout:9/660: getdents d3/d20/dbf/d77/d52/d95/da3 0 2024-03-18T14:50:59.986 INFO:tasks.workunit.client.1.smithi100.stdout:3/242: dread d3/f29 [0,4194304] 0 2024-03-18T14:50:59.988 INFO:tasks.workunit.client.0.smithi012.stdout:5/573: dread d3/d29/d36/d3d/f5c [0,4194304] 0 2024-03-18T14:50:59.989 INFO:tasks.workunit.client.0.smithi012.stdout:5/574: dread - d3/d84/f9c zero size 2024-03-18T14:50:59.990 INFO:tasks.workunit.client.0.smithi012.stdout:0/595: dwrite d1/d20/d23/d3b/d33/d51/fae [0,4194304] 0 2024-03-18T14:50:59.990 INFO:tasks.workunit.client.0.smithi012.stdout:7/581: mkdir dd/d13/d23/d76/d6f/db3/dc7 0 2024-03-18T14:50:59.990 INFO:tasks.workunit.client.0.smithi012.stdout:7/582: write dd/d13/d23/d60/f95 [845079,111911] 0 2024-03-18T14:50:59.994 INFO:tasks.workunit.client.0.smithi012.stdout:1/551: mknod d8/db/d10/d16/caa 0 2024-03-18T14:50:59.995 INFO:tasks.workunit.client.0.smithi012.stdout:2/647: dwrite d0/d6b/fc2 [0,4194304] 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:9/252: dwrite da/db/f4c [0,4194304] 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:9/253: fsync da/db/f36 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:2/203: dwrite d3/d6/d29/f3d [0,4194304] 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:8/253: dwrite d4/f15 [0,4194304] 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:4/227: dwrite d9/d17/f34 [0,4194304] 0 2024-03-18T14:50:59.996 INFO:tasks.workunit.client.1.smithi100.stdout:0/295: dwrite dd/f49 [0,4194304] 0 2024-03-18T14:50:59.997 INFO:tasks.workunit.client.0.smithi012.stdout:6/558: dwrite d17/d33/fa5 [0,4194304] 0 2024-03-18T14:50:59.997 INFO:tasks.workunit.client.0.smithi012.stdout:8/759: dwrite d0/d5a/d5e/dd7/d84/d54/fd6 [0,4194304] 0 2024-03-18T14:50:59.997 INFO:tasks.workunit.client.0.smithi012.stdout:8/760: fdatasync d0/d5a/d19/fe5 0 2024-03-18T14:50:59.997 INFO:tasks.workunit.client.0.smithi012.stdout:2/648: read d0/d6b/dbf/d4f/f60 [956834,14729] 0 2024-03-18T14:50:59.997 INFO:tasks.workunit.client.0.smithi012.stdout:2/649: dread - d0/d6c/db7/fc7 zero size 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:1/235: dwrite d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:7/237: dwrite d5/d14/d16/d2c/f51 [0,4194304] 0 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:1/236: write d8/d1e/d11/d2d/f25 [5224437,16392] 0 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:1/237: chown d8/d1e/d11/f21 594276718 1 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:1/238: chown d8/d1e/d11/d27/d3e 1621 1 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:3/243: unlink d3/dd/f56 0 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:3/244: truncate d3/dd/f11 5030700 0 2024-03-18T14:50:59.999 INFO:tasks.workunit.client.1.smithi100.stdout:3/245: write d3/d16/d13/d17/f40 [322019,29468] 0 2024-03-18T14:51:00.000 INFO:tasks.workunit.client.0.smithi012.stdout:9/661: getdents d3/d20/dbf 0 2024-03-18T14:51:00.001 INFO:tasks.workunit.client.0.smithi012.stdout:0/596: dread d1/d20/d23/d3b/d33/f70 [0,4194304] 0 2024-03-18T14:51:00.001 INFO:tasks.workunit.client.0.smithi012.stdout:0/597: chown d1/d20/d7b/c5e 22589958 1 2024-03-18T14:51:00.001 INFO:tasks.workunit.client.0.smithi012.stdout:0/598: write d1/d20/d23/d3b/f2c [1117597,37953] 0 2024-03-18T14:51:00.001 INFO:tasks.workunit.client.0.smithi012.stdout:0/599: write d1/d16/fe [438527,88746] 0 2024-03-18T14:51:00.004 INFO:tasks.workunit.client.1.smithi100.stdout:4/228: dread d9/f22 [0,4194304] 0 2024-03-18T14:51:00.004 INFO:tasks.workunit.client.0.smithi012.stdout:5/575: creat d3/d29/d36/d9b/fbd x:0 0 0 2024-03-18T14:51:00.004 INFO:tasks.workunit.client.0.smithi012.stdout:5/576: fsync d3/d29/d36/d3d/d65/f6c 0 2024-03-18T14:51:00.012 INFO:tasks.workunit.client.1.smithi100.stdout:9/254: write da/f1d [1499494,9758] 0 2024-03-18T14:51:00.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/204: creat d3/d6/d29/d36/f4e x:0 0 0 2024-03-18T14:51:00.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/205: dread - d3/d6/d29/d36/f4e zero size 2024-03-18T14:51:00.016 INFO:tasks.workunit.client.0.smithi012.stdout:7/583: dwrite dd/d13/d23/d76/d6b/fb0 [0,4194304] 0 2024-03-18T14:51:00.018 INFO:tasks.workunit.client.0.smithi012.stdout:1/552: rename d8/db/d10/d31/d43/d5e/d65/d84/f9b to d8/db/d10/d16/fab 0 2024-03-18T14:51:00.023 INFO:tasks.workunit.client.1.smithi100.stdout:8/254: link d4/dc/dd/de/d21/d23/d25/l3e d4/dc/dd/de/d19/l4c 0 2024-03-18T14:51:00.026 INFO:tasks.workunit.client.0.smithi012.stdout:6/559: creat d17/d38/d51/fa9 x:0 0 0 2024-03-18T14:51:00.028 INFO:tasks.workunit.client.0.smithi012.stdout:5/577: dwrite d3/d1e/f8c [0,4194304] 0 2024-03-18T14:51:00.029 INFO:tasks.workunit.client.1.smithi100.stdout:0/296: dwrite f3 [0,4194304] 0 2024-03-18T14:51:00.029 INFO:tasks.workunit.client.1.smithi100.stdout:0/297: fdatasync dd/f11 0 2024-03-18T14:51:00.038 INFO:tasks.workunit.client.0.smithi012.stdout:2/650: unlink d0/d6b/dbf/d33/d71/f87 0 2024-03-18T14:51:00.038 INFO:tasks.workunit.client.0.smithi012.stdout:2/651: chown d0/d6b/dbf/d31/l38 29061 1 2024-03-18T14:51:00.038 INFO:tasks.workunit.client.0.smithi012.stdout:6/560: write d17/d19/d27/d58/f77 [3034130,39015] 0 2024-03-18T14:51:00.046 INFO:tasks.workunit.client.1.smithi100.stdout:7/238: creat d5/d34/f54 x:0 0 0 2024-03-18T14:51:00.048 INFO:tasks.workunit.client.1.smithi100.stdout:1/239: unlink d8/dd/c20 0 2024-03-18T14:51:00.050 INFO:tasks.workunit.client.1.smithi100.stdout:3/246: creat d3/d16/d13/d1f/f59 x:0 0 0 2024-03-18T14:51:00.053 INFO:tasks.workunit.client.1.smithi100.stdout:4/229: mkdir d9/d27/d42 0 2024-03-18T14:51:00.054 INFO:tasks.workunit.client.0.smithi012.stdout:0/600: link d1/d20/d23/d3b/d33/d64/f74 d1/d16/d6e/fc9 0 2024-03-18T14:51:00.058 INFO:tasks.workunit.client.1.smithi100.stdout:9/255: rename da/dd/c40 to da/d2c/d2f/c4e 0 2024-03-18T14:51:00.058 INFO:tasks.workunit.client.0.smithi012.stdout:2/652: dwrite d0/d6b/dbf/d33/d71/d90/fae [0,4194304] 0 2024-03-18T14:51:00.065 INFO:tasks.workunit.client.0.smithi012.stdout:6/561: dwrite d17/d33/f64 [0,4194304] 0 2024-03-18T14:51:00.065 INFO:tasks.workunit.client.1.smithi100.stdout:1/240: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:51:00.067 INFO:tasks.workunit.client.0.smithi012.stdout:7/584: creat dd/d13/d23/fc8 x:0 0 0 2024-03-18T14:51:00.070 INFO:tasks.workunit.client.0.smithi012.stdout:1/553: mkdir d8/db/d10/d31/d43/d5e/dac 0 2024-03-18T14:51:00.073 INFO:tasks.workunit.client.0.smithi012.stdout:0/601: dwrite d1/d20/d23/d3b/d33/d64/f74 [0,4194304] 0 2024-03-18T14:51:00.074 INFO:tasks.workunit.client.1.smithi100.stdout:0/298: mknod dd/d39/d56/d62/d6e/c77 0 2024-03-18T14:51:00.076 INFO:tasks.workunit.client.0.smithi012.stdout:5/578: rename d3/d15/l44 to d3/d84/lbe 0 2024-03-18T14:51:00.076 INFO:tasks.workunit.client.0.smithi012.stdout:5/579: readlink d3/l52 0 2024-03-18T14:51:00.076 INFO:tasks.workunit.client.0.smithi012.stdout:6/562: dread d17/d19/f3e [0,4194304] 0 2024-03-18T14:51:00.076 INFO:tasks.workunit.client.0.smithi012.stdout:6/563: fdatasync d17/f75 0 2024-03-18T14:51:00.077 INFO:tasks.workunit.client.0.smithi012.stdout:8/761: getdents d0/d47/d59 0 2024-03-18T14:51:00.079 INFO:tasks.workunit.client.1.smithi100.stdout:3/247: link d3/dd/l26 d3/d16/d13/d1f/d3f/l5a 0 2024-03-18T14:51:00.083 INFO:tasks.workunit.client.1.smithi100.stdout:3/248: dread d3/fa [0,4194304] 0 2024-03-18T14:51:00.085 INFO:tasks.workunit.client.1.smithi100.stdout:9/256: rename f7 to da/f4f 0 2024-03-18T14:51:00.085 INFO:tasks.workunit.client.1.smithi100.stdout:9/257: getdents da/dd/d3e 0 2024-03-18T14:51:00.085 INFO:tasks.workunit.client.1.smithi100.stdout:9/258: fdatasync da/db/f13 0 2024-03-18T14:51:00.085 INFO:tasks.workunit.client.1.smithi100.stdout:9/259: truncate da/dd/d29/f2e 874533 0 2024-03-18T14:51:00.085 INFO:tasks.workunit.client.1.smithi100.stdout:9/260: chown da/d25/f35 10 1 2024-03-18T14:51:00.086 INFO:tasks.workunit.client.0.smithi012.stdout:9/662: getdents d3/d20/dbf/d77 0 2024-03-18T14:51:00.086 INFO:tasks.workunit.client.0.smithi012.stdout:9/663: fsync d3/d49/f57 0 2024-03-18T14:51:00.086 INFO:tasks.workunit.client.1.smithi100.stdout:3/249: dread d3/d16/d13/d17/f1a [0,4194304] 0 2024-03-18T14:51:00.087 INFO:tasks.workunit.client.1.smithi100.stdout:3/250: read d3/dd/f24 [113276,125364] 0 2024-03-18T14:51:00.090 INFO:tasks.workunit.client.1.smithi100.stdout:1/241: truncate d8/f29 439040 0 2024-03-18T14:51:00.091 INFO:tasks.workunit.client.1.smithi100.stdout:9/261: dread da/d25/f3a [0,4194304] 0 2024-03-18T14:51:00.093 INFO:tasks.workunit.client.1.smithi100.stdout:0/299: creat dd/d39/d56/d62/f78 x:0 0 0 2024-03-18T14:51:00.095 INFO:tasks.workunit.client.0.smithi012.stdout:9/664: read d3/d20/dbf/d77/d52/fc0 [616591,113995] 0 2024-03-18T14:51:00.095 INFO:tasks.workunit.client.0.smithi012.stdout:2/653: mkdir d0/d6b/dbf/d33/dc9 0 2024-03-18T14:51:00.095 INFO:tasks.workunit.client.0.smithi012.stdout:2/654: stat d0 0 2024-03-18T14:51:00.095 INFO:tasks.workunit.client.0.smithi012.stdout:2/655: write d0/d6b/dbf/d4f/f60 [574234,121990] 0 2024-03-18T14:51:00.096 INFO:tasks.workunit.client.0.smithi012.stdout:2/656: write d0/da2/f5c [167147,8002] 0 2024-03-18T14:51:00.096 INFO:tasks.workunit.client.1.smithi100.stdout:7/239: truncate d5/f52 5359871 0 2024-03-18T14:51:00.096 INFO:tasks.workunit.client.1.smithi100.stdout:7/240: fdatasync d5/d14/d16/d44/f4c 0 2024-03-18T14:51:00.097 INFO:tasks.workunit.client.0.smithi012.stdout:2/657: write d0/db4/fb9 [332988,85873] 0 2024-03-18T14:51:00.098 INFO:tasks.workunit.client.1.smithi100.stdout:5/199: sync 2024-03-18T14:51:00.100 INFO:tasks.workunit.client.1.smithi100.stdout:4/230: getdents d9 0 2024-03-18T14:51:00.100 INFO:tasks.workunit.client.1.smithi100.stdout:4/231: write d9/f22 [4466041,82372] 0 2024-03-18T14:51:00.107 INFO:tasks.workunit.client.1.smithi100.stdout:2/206: rename d3/d6/d29/d36/d46 to d3/d13/d4f 0 2024-03-18T14:51:00.107 INFO:tasks.workunit.client.1.smithi100.stdout:2/207: dread - d3/da/f30 zero size 2024-03-18T14:51:00.109 INFO:tasks.workunit.client.0.smithi012.stdout:7/585: symlink dd/d13/d23/d76/d6f/db3/lc9 0 2024-03-18T14:51:00.111 INFO:tasks.workunit.client.1.smithi100.stdout:3/251: symlink d3/d16/d13/d1f/d3f/d4f/l5b 0 2024-03-18T14:51:00.111 INFO:tasks.workunit.client.1.smithi100.stdout:3/252: read d3/d16/d13/f39 [71192,7147] 0 2024-03-18T14:51:00.111 INFO:tasks.workunit.client.1.smithi100.stdout:3/253: creat d3/d16/d13/d1f/f5c x:0 0 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/554: creat d8/db/d10/d16/d6b/d7e/fad x:0 0 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/555: creat d8/db/d10/d31/d81/fae x:0 0 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/556: stat d8/db/d10/d16/d6b 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/557: write d8/db/d10/d16/d54/f93 [830308,126913] 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/558: truncate d8/db/d10/d31/f6e 1868512 0 2024-03-18T14:51:00.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/559: write d8/d1e/f7c [1474253,90531] 0 2024-03-18T14:51:00.115 INFO:tasks.workunit.client.1.smithi100.stdout:1/242: creat d8/d2a/f44 x:0 0 0 2024-03-18T14:51:00.115 INFO:tasks.workunit.client.0.smithi012.stdout:3/568: sync 2024-03-18T14:51:00.115 INFO:tasks.workunit.client.0.smithi012.stdout:3/569: creat dc/d3c/d4c/d57/fb9 x:0 0 0 2024-03-18T14:51:00.115 INFO:tasks.workunit.client.0.smithi012.stdout:3/570: chown dc/d3c 58 1 2024-03-18T14:51:00.115 INFO:tasks.workunit.client.0.smithi012.stdout:3/571: read - dc/d30/f98 zero size 2024-03-18T14:51:00.116 INFO:tasks.workunit.client.0.smithi012.stdout:3/572: write dc/d3c/d4c/d57/fa1 [79646,75788] 0 2024-03-18T14:51:00.116 INFO:tasks.workunit.client.0.smithi012.stdout:3/573: chown dc/f48 80 1 2024-03-18T14:51:00.116 INFO:tasks.workunit.client.0.smithi012.stdout:9/665: dwrite d3/d20/dbf/d66/d69/d7f/d8c/d8b/f9f [0,4194304] 0 2024-03-18T14:51:00.117 INFO:tasks.workunit.client.1.smithi100.stdout:0/300: mkdir dd/d39/d56/d59/d67/d79 0 2024-03-18T14:51:00.120 INFO:tasks.workunit.client.1.smithi100.stdout:5/200: creat d29/d2d/f41 x:0 0 0 2024-03-18T14:51:00.120 INFO:tasks.workunit.client.1.smithi100.stdout:5/201: chown l1d 21 1 2024-03-18T14:51:00.123 INFO:tasks.workunit.client.0.smithi012.stdout:0/602: rename d1/d20/d4e/f52 to d1/d20/d23/d7f/fca 0 2024-03-18T14:51:00.123 INFO:tasks.workunit.client.0.smithi012.stdout:0/603: dread - d1/d20/d23/d3b/d33/d64/f96 zero size 2024-03-18T14:51:00.125 INFO:tasks.workunit.client.1.smithi100.stdout:4/232: symlink d9/d13/d25/d41/l43 0 2024-03-18T14:51:00.127 INFO:tasks.workunit.client.1.smithi100.stdout:1/243: dwrite d8/d2a/f41 [0,4194304] 0 2024-03-18T14:51:00.128 INFO:tasks.workunit.client.1.smithi100.stdout:1/244: chown d8/d30/f35 1664899712 1 2024-03-18T14:51:00.128 INFO:tasks.workunit.client.0.smithi012.stdout:5/580: mknod d3/d29/d36/cbf 0 2024-03-18T14:51:00.128 INFO:tasks.workunit.client.0.smithi012.stdout:5/581: creat d3/d15/d5f/fc0 x:0 0 0 2024-03-18T14:51:00.131 INFO:tasks.workunit.client.1.smithi100.stdout:8/255: rename d4/dc/dd/c38 to d4/dc/dd/de/d19/c4d 0 2024-03-18T14:51:00.131 INFO:tasks.workunit.client.1.smithi100.stdout:5/202: dread f7 [0,4194304] 0 2024-03-18T14:51:00.131 INFO:tasks.workunit.client.1.smithi100.stdout:5/203: write d29/f34 [642468,130758] 0 2024-03-18T14:51:00.132 INFO:tasks.workunit.client.0.smithi012.stdout:6/564: mknod d17/d38/d51/d66/caa 0 2024-03-18T14:51:00.134 INFO:tasks.workunit.client.1.smithi100.stdout:2/208: truncate d3/da/de/ff 3211409 0 2024-03-18T14:51:00.135 INFO:tasks.workunit.client.0.smithi012.stdout:8/762: mknod d0/d5a/d19/d57/cee 0 2024-03-18T14:51:00.139 INFO:tasks.workunit.client.0.smithi012.stdout:2/658: symlink d0/d6b/dbf/d31/lca 0 2024-03-18T14:51:00.141 INFO:tasks.workunit.client.1.smithi100.stdout:6/218: sync 2024-03-18T14:51:00.142 INFO:tasks.workunit.client.1.smithi100.stdout:2/209: read d3/f1b [4192654,73486] 0 2024-03-18T14:51:00.147 INFO:tasks.workunit.client.0.smithi012.stdout:5/582: dwrite d3/d29/d36/d3d/d73/f7e [0,4194304] 0 2024-03-18T14:51:00.147 INFO:tasks.workunit.client.1.smithi100.stdout:3/254: link d3/dd/d21/f22 d3/dd/d49/f5d 0 2024-03-18T14:51:00.147 INFO:tasks.workunit.client.0.smithi012.stdout:1/560: symlink d8/db/d10/d4e/laf 0 2024-03-18T14:51:00.148 INFO:tasks.workunit.client.1.smithi100.stdout:0/301: symlink dd/d2b/l7a 0 2024-03-18T14:51:00.148 INFO:tasks.workunit.client.1.smithi100.stdout:0/302: write dd/d2b/d54/f6d [538484,58609] 0 2024-03-18T14:51:00.150 INFO:tasks.workunit.client.1.smithi100.stdout:4/233: symlink d9/d13/l44 0 2024-03-18T14:51:00.151 INFO:tasks.workunit.client.0.smithi012.stdout:3/574: symlink dc/d3c/lba 0 2024-03-18T14:51:00.151 INFO:tasks.workunit.client.0.smithi012.stdout:3/575: chown dc/d30/d3a/f5e 184408079 1 2024-03-18T14:51:00.153 INFO:tasks.workunit.client.1.smithi100.stdout:1/245: truncate d8/d1e/d11/f1b 2828425 0 2024-03-18T14:51:00.153 INFO:tasks.workunit.client.1.smithi100.stdout:1/246: creat d8/d1e/f45 x:0 0 0 2024-03-18T14:51:00.155 INFO:tasks.workunit.client.1.smithi100.stdout:9/262: rename da/dd/d29/c2b to da/d46/c50 0 2024-03-18T14:51:00.157 INFO:tasks.workunit.client.0.smithi012.stdout:9/666: mkdir d3/d20/dbf/d66/d21/d35/d48/dda 0 2024-03-18T14:51:00.158 INFO:tasks.workunit.client.1.smithi100.stdout:5/204: truncate f25 37435 0 2024-03-18T14:51:00.160 INFO:tasks.workunit.client.1.smithi100.stdout:6/219: truncate d6/fb 1884322 0 2024-03-18T14:51:00.160 INFO:tasks.workunit.client.1.smithi100.stdout:6/220: dread - d6/d7/f40 zero size 2024-03-18T14:51:00.162 INFO:tasks.workunit.client.0.smithi012.stdout:7/586: rename dd/d13/d23/d2f/d7a/la7 to dd/d13/d23/lca 0 2024-03-18T14:51:00.165 INFO:tasks.workunit.client.1.smithi100.stdout:2/210: link d3/da/f3a d3/da/de/d20/f50 0 2024-03-18T14:51:00.165 INFO:tasks.workunit.client.1.smithi100.stdout:2/211: fdatasync d3/f1b 0 2024-03-18T14:51:00.166 INFO:tasks.workunit.client.0.smithi012.stdout:0/604: mknod d1/d20/d23/d6f/d93/ccb 0 2024-03-18T14:51:00.166 INFO:tasks.workunit.client.0.smithi012.stdout:0/605: dread - d1/d16/d19/d8d/fad zero size 2024-03-18T14:51:00.167 INFO:tasks.workunit.client.1.smithi100.stdout:0/303: mkdir dd/d39/d7b 0 2024-03-18T14:51:00.168 INFO:tasks.workunit.client.1.smithi100.stdout:4/234: rmdir d9/d13/d25/d41 39 2024-03-18T14:51:00.168 INFO:tasks.workunit.client.1.smithi100.stdout:4/235: chown d9/d13/d25/l35 189710 1 2024-03-18T14:51:00.169 INFO:tasks.workunit.client.1.smithi100.stdout:4/236: fdatasync d9/fe 0 2024-03-18T14:51:00.169 INFO:tasks.workunit.client.1.smithi100.stdout:4/237: chown d9/f1b 0 1 2024-03-18T14:51:00.169 INFO:tasks.workunit.client.0.smithi012.stdout:6/565: rmdir d17/d26/d36/d4d/d82 39 2024-03-18T14:51:00.172 INFO:tasks.workunit.client.1.smithi100.stdout:1/247: mknod d8/d1e/d11/c46 0 2024-03-18T14:51:00.174 INFO:tasks.workunit.client.1.smithi100.stdout:9/263: dwrite da/db/f13 [0,4194304] 0 2024-03-18T14:51:00.174 INFO:tasks.workunit.client.1.smithi100.stdout:9/264: stat da/d2c/d2f/f4a 0 2024-03-18T14:51:00.174 INFO:tasks.workunit.client.1.smithi100.stdout:9/265: write da/dd/d29/f2e [1836999,6458] 0 2024-03-18T14:51:00.176 INFO:tasks.workunit.client.0.smithi012.stdout:6/566: dread d17/d38/f5a [0,4194304] 0 2024-03-18T14:51:00.176 INFO:tasks.workunit.client.1.smithi100.stdout:8/256: rename d4/dc/dd/f34 to d4/dc/d1f/d33/f4e 0 2024-03-18T14:51:00.177 INFO:tasks.workunit.client.0.smithi012.stdout:4/521: sync 2024-03-18T14:51:00.178 INFO:tasks.workunit.client.1.smithi100.stdout:5/205: truncate fa 3832908 0 2024-03-18T14:51:00.178 INFO:tasks.workunit.client.1.smithi100.stdout:5/206: write d29/d2d/f41 [164451,29330] 0 2024-03-18T14:51:00.178 INFO:tasks.workunit.client.1.smithi100.stdout:5/207: chown d29/d2d/d32/l40 6412 1 2024-03-18T14:51:00.179 INFO:tasks.workunit.client.0.smithi012.stdout:6/567: dread d17/d38/d5b/f80 [0,4194304] 0 2024-03-18T14:51:00.179 INFO:tasks.workunit.client.0.smithi012.stdout:6/568: dread - d17/d19/d27/d58/f9f zero size 2024-03-18T14:51:00.179 INFO:tasks.workunit.client.0.smithi012.stdout:5/583: mknod d3/d29/d36/d3d/d73/d6b/d8f/cc1 0 2024-03-18T14:51:00.181 INFO:tasks.workunit.client.1.smithi100.stdout:8/257: dread d4/dc/dd/de/f17 [0,4194304] 0 2024-03-18T14:51:00.182 INFO:tasks.workunit.client.0.smithi012.stdout:1/561: unlink d8/db/d10/d16/da1/d2e/f4b 0 2024-03-18T14:51:00.183 INFO:tasks.workunit.client.1.smithi100.stdout:2/212: creat d3/d6/d29/d36/d49/f51 x:0 0 0 2024-03-18T14:51:00.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/266: dread da/dd/d29/f34 [0,4194304] 0 2024-03-18T14:51:00.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/267: read da/fc [1866690,26878] 0 2024-03-18T14:51:00.187 INFO:tasks.workunit.client.1.smithi100.stdout:9/268: truncate f6 886288 0 2024-03-18T14:51:00.187 INFO:tasks.workunit.client.0.smithi012.stdout:3/576: creat dc/d91/fbb x:0 0 0 2024-03-18T14:51:00.187 INFO:tasks.workunit.client.0.smithi012.stdout:4/522: dread d6/da/d58/f59 [0,4194304] 0 2024-03-18T14:51:00.188 INFO:tasks.workunit.client.1.smithi100.stdout:0/304: creat dd/d39/d56/d59/f7c x:0 0 0 2024-03-18T14:51:00.191 INFO:tasks.workunit.client.0.smithi012.stdout:9/667: unlink d3/d20/dbf/d66/d6e/dc6/fd6 0 2024-03-18T14:51:00.191 INFO:tasks.workunit.client.0.smithi012.stdout:9/668: read d3/d9/fd [183796,55087] 0 2024-03-18T14:51:00.191 INFO:tasks.workunit.client.0.smithi012.stdout:9/669: dread - d3/d9/fd5 zero size 2024-03-18T14:51:00.191 INFO:tasks.workunit.client.0.smithi012.stdout:9/670: chown d3/d20/dbf/d77/d52/fc0 3 1 2024-03-18T14:51:00.191 INFO:tasks.workunit.client.1.smithi100.stdout:1/248: dwrite d8/d1e/d11/f33 [0,4194304] 0 2024-03-18T14:51:00.192 INFO:tasks.workunit.client.1.smithi100.stdout:4/238: creat d9/d17/f45 x:0 0 0 2024-03-18T14:51:00.195 INFO:tasks.workunit.client.0.smithi012.stdout:8/763: rename d0/fc to d0/d5a/d55/fef 0 2024-03-18T14:51:00.197 INFO:tasks.workunit.client.1.smithi100.stdout:3/255: rename d3/c7 to d3/d16/d13/d1f/d3f/d4f/c5e 0 2024-03-18T14:51:00.198 INFO:tasks.workunit.client.0.smithi012.stdout:7/587: symlink dd/d13/d23/d76/d6b/lcb 0 2024-03-18T14:51:00.198 INFO:tasks.workunit.client.0.smithi012.stdout:7/588: fdatasync dd/f5b 0 2024-03-18T14:51:00.198 INFO:tasks.workunit.client.0.smithi012.stdout:7/589: write dd/d13/d23/f99 [2058805,39072] 0 2024-03-18T14:51:00.199 INFO:tasks.workunit.client.1.smithi100.stdout:2/213: dwrite d3/f4 [0,4194304] 0 2024-03-18T14:51:00.199 INFO:tasks.workunit.client.1.smithi100.stdout:2/214: chown d3/c7 592819 1 2024-03-18T14:51:00.200 INFO:tasks.workunit.client.1.smithi100.stdout:8/258: mknod d4/dc/dd/de/d21/d23/c4f 0 2024-03-18T14:51:00.204 INFO:tasks.workunit.client.1.smithi100.stdout:0/305: truncate dd/f1c 1087499 0 2024-03-18T14:51:00.204 INFO:tasks.workunit.client.1.smithi100.stdout:0/306: write dd/d20/f4e [136619,116269] 0 2024-03-18T14:51:00.204 INFO:tasks.workunit.client.1.smithi100.stdout:0/307: fsync dd/f49 0 2024-03-18T14:51:00.205 INFO:tasks.workunit.client.0.smithi012.stdout:2/659: symlink d0/d6b/dbf/d33/d80/lcb 0 2024-03-18T14:51:00.207 INFO:tasks.workunit.client.0.smithi012.stdout:6/569: symlink d17/d19/d27/d2f/lab 0 2024-03-18T14:51:00.212 INFO:tasks.workunit.client.0.smithi012.stdout:8/764: dwrite d0/d5a/d5e/ddf/d50/fa5 [0,4194304] 0 2024-03-18T14:51:00.212 INFO:tasks.workunit.client.0.smithi012.stdout:8/765: read d0/d5a/d19/dad/f2d [732503,96343] 0 2024-03-18T14:51:00.215 INFO:tasks.workunit.client.0.smithi012.stdout:4/523: link d6/f33 d6/d1a/d30/d46/d6f/d8b/fb0 0 2024-03-18T14:51:00.218 INFO:tasks.workunit.client.0.smithi012.stdout:9/671: symlink d3/d20/dbf/d66/d69/ldb 0 2024-03-18T14:51:00.218 INFO:tasks.workunit.client.0.smithi012.stdout:9/672: truncate d3/d20/dbf/d66/d21/d35/fb9 383282 0 2024-03-18T14:51:00.223 INFO:tasks.workunit.client.0.smithi012.stdout:5/584: rename d3/d29/d36/d3d/d73/l67 to d3/d29/d35/d5b/lc2 0 2024-03-18T14:51:00.228 INFO:tasks.workunit.client.0.smithi012.stdout:0/606: getdents d1/d20/d23/d3b/d33 0 2024-03-18T14:51:00.231 INFO:tasks.workunit.client.0.smithi012.stdout:2/660: symlink d0/d6b/dbf/d33/dac/lcc 0 2024-03-18T14:51:00.233 INFO:tasks.workunit.client.1.smithi100.stdout:4/239: creat d9/d3a/f46 x:0 0 0 2024-03-18T14:51:00.233 INFO:tasks.workunit.client.1.smithi100.stdout:4/240: truncate d9/d13/f33 1223982 0 2024-03-18T14:51:00.233 INFO:tasks.workunit.client.0.smithi012.stdout:3/577: rmdir dc/d30/dab 39 2024-03-18T14:51:00.233 INFO:tasks.workunit.client.0.smithi012.stdout:3/578: truncate dc/d91/fbb 416635 0 2024-03-18T14:51:00.240 INFO:tasks.workunit.client.1.smithi100.stdout:3/256: unlink d3/f5 0 2024-03-18T14:51:00.240 INFO:tasks.workunit.client.1.smithi100.stdout:3/257: creat d3/d16/d13/f5f x:0 0 0 2024-03-18T14:51:00.247 INFO:tasks.workunit.client.1.smithi100.stdout:2/215: symlink d3/l52 0 2024-03-18T14:51:00.247 INFO:tasks.workunit.client.1.smithi100.stdout:2/216: write d3/da/f3a [63528,77516] 0 2024-03-18T14:51:00.249 INFO:tasks.workunit.client.1.smithi100.stdout:8/259: creat d4/dc/dd/de/d21/d23/d47/f50 x:0 0 0 2024-03-18T14:51:00.251 INFO:tasks.workunit.client.0.smithi012.stdout:3/579: dwrite dc/d3c/d59/d86/d9a/d54/f97 [4194304,4194304] 0 2024-03-18T14:51:00.252 INFO:tasks.workunit.client.1.smithi100.stdout:0/308: mkdir dd/d39/d56/d59/d67/d79/d7d 0 2024-03-18T14:51:00.252 INFO:tasks.workunit.client.1.smithi100.stdout:0/309: fdatasync dd/f42 0 2024-03-18T14:51:00.253 INFO:tasks.workunit.client.1.smithi100.stdout:0/310: read dd/d20/f5c [1545234,111335] 0 2024-03-18T14:51:00.253 INFO:tasks.workunit.client.1.smithi100.stdout:0/311: truncate dd/f1a 991043 0 2024-03-18T14:51:00.254 INFO:tasks.workunit.client.1.smithi100.stdout:4/241: link d9/c28 d9/d13/d25/d41/c47 0 2024-03-18T14:51:00.254 INFO:tasks.workunit.client.1.smithi100.stdout:4/242: read - d9/d3a/f46 zero size 2024-03-18T14:51:00.255 INFO:tasks.workunit.client.1.smithi100.stdout:4/243: fdatasync d9/d13/f3f 0 2024-03-18T14:51:00.257 INFO:tasks.workunit.client.1.smithi100.stdout:6/221: rename d6/fb to d6/d12/f43 0 2024-03-18T14:51:00.259 INFO:tasks.workunit.client.0.smithi012.stdout:4/524: stat d6/d1a/l53 0 2024-03-18T14:51:00.261 INFO:tasks.workunit.client.1.smithi100.stdout:3/258: mkdir d3/d16/d13/d1f/d3f/d4f/d60 0 2024-03-18T14:51:00.261 INFO:tasks.workunit.client.1.smithi100.stdout:3/259: chown d3/d16/d13/d1f 1968 1 2024-03-18T14:51:00.262 INFO:tasks.workunit.client.1.smithi100.stdout:2/217: dwrite d3/f10 [0,4194304] 0 2024-03-18T14:51:00.262 INFO:tasks.workunit.client.1.smithi100.stdout:2/218: dread - d3/f22 zero size 2024-03-18T14:51:00.263 INFO:tasks.workunit.client.0.smithi012.stdout:9/673: mkdir d3/d20/dbf/d66/d69/d7f/d8c/dac/dd1/ddc 0 2024-03-18T14:51:00.263 INFO:tasks.workunit.client.0.smithi012.stdout:9/674: read - d3/d9/d5c/fad zero size 2024-03-18T14:51:00.266 INFO:tasks.workunit.client.0.smithi012.stdout:1/562: rename d8/db/d10/d16/d6b/f99 to d8/db/d10/fb0 0 2024-03-18T14:51:00.266 INFO:tasks.workunit.client.0.smithi012.stdout:1/563: write d8/db/d10/f33 [1103520,87010] 0 2024-03-18T14:51:00.267 INFO:tasks.workunit.client.1.smithi100.stdout:8/260: symlink d4/d3c/l51 0 2024-03-18T14:51:00.268 INFO:tasks.workunit.client.0.smithi012.stdout:5/585: symlink d3/d15/lc3 0 2024-03-18T14:51:00.269 INFO:tasks.workunit.client.0.smithi012.stdout:5/586: truncate d3/d84/f85 84097 0 2024-03-18T14:51:00.269 INFO:tasks.workunit.client.0.smithi012.stdout:5/587: write d3/d29/d36/d3d/d73/f92 [818815,98122] 0 2024-03-18T14:51:00.270 INFO:tasks.workunit.client.0.smithi012.stdout:4/525: dread d6/d19/f1c [0,4194304] 0 2024-03-18T14:51:00.270 INFO:tasks.workunit.client.0.smithi012.stdout:4/526: write d6/d19/f51 [707866,44452] 0 2024-03-18T14:51:00.270 INFO:tasks.workunit.client.1.smithi100.stdout:4/244: rename d9/d13/d25/f38 to d9/d13/d25/f48 0 2024-03-18T14:51:00.271 INFO:tasks.workunit.client.1.smithi100.stdout:7/241: sync 2024-03-18T14:51:00.272 INFO:tasks.workunit.client.1.smithi100.stdout:5/208: rename d29/d2d/l2f to d29/d2d/d32/l42 0 2024-03-18T14:51:00.272 INFO:tasks.workunit.client.1.smithi100.stdout:5/209: write f7 [6996948,6550] 0 2024-03-18T14:51:00.272 INFO:tasks.workunit.client.0.smithi012.stdout:7/590: rmdir dd/d13/d23/d2f/d7a/dab 39 2024-03-18T14:51:00.274 INFO:tasks.workunit.client.1.smithi100.stdout:6/222: mknod d6/d7/c44 0 2024-03-18T14:51:00.274 INFO:tasks.workunit.client.1.smithi100.stdout:6/223: dread - d6/d7/df/f2a zero size 2024-03-18T14:51:00.275 INFO:tasks.workunit.client.0.smithi012.stdout:0/607: unlink d1/d2d/f60 0 2024-03-18T14:51:00.275 INFO:tasks.workunit.client.0.smithi012.stdout:0/608: fdatasync d1/d20/d7b/fa1 0 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/260: creat d3/d16/d13/d17/f61 x:0 0 0 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/261: dread - d3/d16/d13/f5f zero size 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/262: chown d3/d16/d13/d1f/d3f/c46 419 1 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/263: dread - d3/d16/d13/d17/f61 zero size 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/264: creat d3/d16/f62 x:0 0 0 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.1.smithi100.stdout:3/265: truncate d3/dd/d49/f5d 608323 0 2024-03-18T14:51:00.278 INFO:tasks.workunit.client.0.smithi012.stdout:2/661: creat d0/d6b/dbf/d33/dc9/fcd x:0 0 0 2024-03-18T14:51:00.279 INFO:tasks.workunit.client.0.smithi012.stdout:8/766: getdents d0/d5a/d5e/ddf/d50/d6a 0 2024-03-18T14:51:00.280 INFO:tasks.workunit.client.1.smithi100.stdout:4/245: write d9/d3a/f3c [767008,21828] 0 2024-03-18T14:51:00.280 INFO:tasks.workunit.client.1.smithi100.stdout:4/246: chown d9/lf 10154 1 2024-03-18T14:51:00.280 INFO:tasks.workunit.client.1.smithi100.stdout:3/266: dread d3/dd/f38 [0,4194304] 0 2024-03-18T14:51:00.280 INFO:tasks.workunit.client.1.smithi100.stdout:3/267: read - d3/dd/d21/f55 zero size 2024-03-18T14:51:00.282 INFO:tasks.workunit.client.1.smithi100.stdout:2/219: mkdir d3/d13/d40/d53 0 2024-03-18T14:51:00.282 INFO:tasks.workunit.client.1.smithi100.stdout:2/220: creat d3/d6/d29/d36/d49/f54 x:0 0 0 2024-03-18T14:51:00.284 INFO:tasks.workunit.client.0.smithi012.stdout:7/591: write f5 [2340357,110900] 0 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.1.smithi100.stdout:7/242: mknod d5/d22/d3b/c55 0 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.1.smithi100.stdout:4/247: dread d9/d13/f1e [0,4194304] 0 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.1.smithi100.stdout:4/248: readlink l6 0 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.1.smithi100.stdout:4/249: write d9/f2b [3371043,122418] 0 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.1.smithi100.stdout:4/250: dread - d9/d3a/f46 zero size 2024-03-18T14:51:00.285 INFO:tasks.workunit.client.0.smithi012.stdout:3/580: rename dc/d3c/d72/fb4 to dc/d3c/d59/fbc 0 2024-03-18T14:51:00.287 INFO:tasks.workunit.client.1.smithi100.stdout:8/261: dwrite d4/dc/dd/f44 [0,4194304] 0 2024-03-18T14:51:00.287 INFO:tasks.workunit.client.0.smithi012.stdout:1/564: creat d8/db/d10/d16/da1/d2e/d9c/fb1 x:0 0 0 2024-03-18T14:51:00.288 INFO:tasks.workunit.client.0.smithi012.stdout:1/565: fdatasync d8/d1e/f9f 0 2024-03-18T14:51:00.289 INFO:tasks.workunit.client.1.smithi100.stdout:5/210: rmdir d29/d2d/d32/d38 39 2024-03-18T14:51:00.289 INFO:tasks.workunit.client.1.smithi100.stdout:5/211: write f23 [555837,63667] 0 2024-03-18T14:51:00.289 INFO:tasks.workunit.client.1.smithi100.stdout:5/212: write f26 [2029670,123513] 0 2024-03-18T14:51:00.292 INFO:tasks.workunit.client.1.smithi100.stdout:5/213: dread fa [0,4194304] 0 2024-03-18T14:51:00.293 INFO:tasks.workunit.client.1.smithi100.stdout:5/214: fsync f1b 0 2024-03-18T14:51:00.293 INFO:tasks.workunit.client.1.smithi100.stdout:0/312: rename dd/d39/d56/d62/f78 to dd/d39/d56/d59/d67/d79/d7d/f7e 0 2024-03-18T14:51:00.293 INFO:tasks.workunit.client.1.smithi100.stdout:0/313: dread - dd/f3a zero size 2024-03-18T14:51:00.293 INFO:tasks.workunit.client.1.smithi100.stdout:0/314: write dd/d20/d44/f5e [895855,20213] 0 2024-03-18T14:51:00.297 INFO:tasks.workunit.client.0.smithi012.stdout:4/527: creat d6/d76/fb1 x:0 0 0 2024-03-18T14:51:00.299 INFO:tasks.workunit.client.0.smithi012.stdout:0/609: unlink d1/d20/d23/d3b/d33/f70 0 2024-03-18T14:51:00.301 INFO:tasks.workunit.client.1.smithi100.stdout:3/268: unlink d3/d16/d13/d17/f40 0 2024-03-18T14:51:00.302 INFO:tasks.workunit.client.0.smithi012.stdout:8/767: creat d0/d5a/d5e/dd7/d84/d54/d62/ff0 x:0 0 0 2024-03-18T14:51:00.305 INFO:tasks.workunit.client.0.smithi012.stdout:3/581: dwrite dc/d30/f37 [0,4194304] 0 2024-03-18T14:51:00.306 INFO:tasks.workunit.client.1.smithi100.stdout:2/221: link d3/da/c2e d3/d13/d4f/c55 0 2024-03-18T14:51:00.307 INFO:tasks.workunit.client.1.smithi100.stdout:3/269: write d3/f29 [961730,4846] 0 2024-03-18T14:51:00.308 INFO:tasks.workunit.client.1.smithi100.stdout:6/224: dwrite d6/d12/f3b [0,4194304] 0 2024-03-18T14:51:00.308 INFO:tasks.workunit.client.0.smithi012.stdout:7/592: rmdir dd/d13/d23/d2f/d53/d8d 39 2024-03-18T14:51:00.309 INFO:tasks.workunit.client.0.smithi012.stdout:7/593: write dd/d13/d23/d60/f62 [84459,72604] 0 2024-03-18T14:51:00.311 INFO:tasks.workunit.client.1.smithi100.stdout:7/243: unlink d5/d14/d16/l26 0 2024-03-18T14:51:00.313 INFO:tasks.workunit.client.0.smithi012.stdout:9/675: rename d3/d20/dbf/d66/d6e/l81 to d3/d20/dbf/d66/d21/d35/d48/dda/ldd 0 2024-03-18T14:51:00.314 INFO:tasks.workunit.client.1.smithi100.stdout:6/225: write d6/d10/f13 [1854115,79535] 0 2024-03-18T14:51:00.315 INFO:tasks.workunit.client.0.smithi012.stdout:1/566: mknod d8/db/d10/d16/d54/cb2 0 2024-03-18T14:51:00.316 INFO:tasks.workunit.client.1.smithi100.stdout:4/251: link d9/la d9/d17/l49 0 2024-03-18T14:51:00.316 INFO:tasks.workunit.client.1.smithi100.stdout:4/252: read - d9/d13/f3f zero size 2024-03-18T14:51:00.322 INFO:tasks.workunit.client.0.smithi012.stdout:0/610: mkdir d1/d16/d6e/dcc 0 2024-03-18T14:51:00.322 INFO:tasks.workunit.client.0.smithi012.stdout:0/611: read d1/f5f [2082692,36804] 0 2024-03-18T14:51:00.323 INFO:tasks.workunit.client.1.smithi100.stdout:7/244: dread d5/fa [0,4194304] 0 2024-03-18T14:51:00.325 INFO:tasks.workunit.client.0.smithi012.stdout:4/528: dwrite d6/d27/f67 [0,4194304] 0 2024-03-18T14:51:00.325 INFO:tasks.workunit.client.0.smithi012.stdout:8/768: mknod d0/d5a/d19/cf1 0 2024-03-18T14:51:00.325 INFO:tasks.workunit.client.0.smithi012.stdout:8/769: creat d0/d47/ff2 x:0 0 0 2024-03-18T14:51:00.325 INFO:tasks.workunit.client.0.smithi012.stdout:8/770: write d0/d5a/d5e/dd7/f7b [1505860,86555] 0 2024-03-18T14:51:00.326 INFO:tasks.workunit.client.1.smithi100.stdout:4/253: dread d9/d17/f1d [0,4194304] 0 2024-03-18T14:51:00.326 INFO:tasks.workunit.client.0.smithi012.stdout:3/582: rmdir dc/d3c 39 2024-03-18T14:51:00.326 INFO:tasks.workunit.client.0.smithi012.stdout:3/583: chown dc/d30/d35/cb6 61792 1 2024-03-18T14:51:00.328 INFO:tasks.workunit.client.0.smithi012.stdout:8/771: write d0/f10 [1341326,68153] 0 2024-03-18T14:51:00.328 INFO:tasks.workunit.client.0.smithi012.stdout:8/772: fdatasync d0/d47/d59/dc9/fe6 0 2024-03-18T14:51:00.330 INFO:tasks.workunit.client.0.smithi012.stdout:9/676: write d3/f4 [3754439,11009] 0 2024-03-18T14:51:00.333 INFO:tasks.workunit.client.0.smithi012.stdout:7/594: mkdir dd/d13/d23/d33/dcc 0 2024-03-18T14:51:00.335 INFO:tasks.workunit.client.0.smithi012.stdout:0/612: dread d1/d20/d23/d26/f36 [0,4194304] 0 2024-03-18T14:51:00.335 INFO:tasks.workunit.client.0.smithi012.stdout:0/613: stat d1/d16/d6e/fc7 0 2024-03-18T14:51:00.337 INFO:tasks.workunit.client.0.smithi012.stdout:2/662: rename d0/da2/f51 to d0/d6b/dbf/d33/d99/fce 0 2024-03-18T14:51:00.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/584: dread dc/d30/d35/f65 [0,4194304] 0 2024-03-18T14:51:00.344 INFO:tasks.workunit.client.1.smithi100.stdout:8/262: mknod d4/dc/d1f/d33/c52 0 2024-03-18T14:51:00.345 INFO:tasks.workunit.client.0.smithi012.stdout:1/567: dwrite d8/db/d10/d16/da1/d2e/d9c/fb1 [0,4194304] 0 2024-03-18T14:51:00.346 INFO:tasks.workunit.client.0.smithi012.stdout:4/529: dwrite d6/da/d89/d95/fa4 [0,4194304] 0 2024-03-18T14:51:00.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/595: write dd/d13/d23/d33/f51 [3182695,122809] 0 2024-03-18T14:51:00.347 INFO:tasks.workunit.client.0.smithi012.stdout:7/596: readlink dd/d13/d23/d2f/d7a/dab/lba 0 2024-03-18T14:51:00.357 INFO:tasks.workunit.client.1.smithi100.stdout:5/215: creat d29/d2d/d32/d38/f43 x:0 0 0 2024-03-18T14:51:00.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/585: dwrite dc/d3c/d59/d86/d9a/d64/f79 [0,4194304] 0 2024-03-18T14:51:00.359 INFO:tasks.workunit.client.0.smithi012.stdout:2/663: dwrite d0/d6b/dbf/d33/d71/fc1 [0,4194304] 0 2024-03-18T14:51:00.361 INFO:tasks.workunit.client.1.smithi100.stdout:0/315: mknod dd/d61/c7f 0 2024-03-18T14:51:00.370 INFO:tasks.workunit.client.1.smithi100.stdout:8/263: dwrite d4/dc/dd/de/d19/f1e [0,4194304] 0 2024-03-18T14:51:00.371 INFO:tasks.workunit.client.1.smithi100.stdout:3/270: creat d3/d16/d13/d1f/d3f/d4f/d60/f63 x:0 0 0 2024-03-18T14:51:00.371 INFO:tasks.workunit.client.1.smithi100.stdout:9/269: sync 2024-03-18T14:51:00.371 INFO:tasks.workunit.client.1.smithi100.stdout:1/249: sync 2024-03-18T14:51:00.372 INFO:tasks.workunit.client.1.smithi100.stdout:1/250: chown d8/d1e/l13 27458388 1 2024-03-18T14:51:00.372 INFO:tasks.workunit.client.1.smithi100.stdout:1/251: chown d8/d1e/d11/d27/d3c 3 1 2024-03-18T14:51:00.374 INFO:tasks.workunit.client.1.smithi100.stdout:6/226: creat d6/d7/df/d26/f45 x:0 0 0 2024-03-18T14:51:00.378 INFO:tasks.workunit.client.1.smithi100.stdout:7/245: mknod d5/d22/d3b/c56 0 2024-03-18T14:51:00.378 INFO:tasks.workunit.client.1.smithi100.stdout:7/246: chown d5/c25 93694 1 2024-03-18T14:51:00.383 INFO:tasks.workunit.client.0.smithi012.stdout:2/664: dwrite d0/d10/f58 [0,4194304] 0 2024-03-18T14:51:00.383 INFO:tasks.workunit.client.0.smithi012.stdout:2/665: write d0/da2/f67 [2798476,98022] 0 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.1.smithi100.stdout:4/254: creat d9/d27/d42/f4a x:0 0 0 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.1.smithi100.stdout:4/255: write d9/d17/f23 [4468855,104549] 0 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.0.smithi012.stdout:5/588: dwrite d3/d29/d35/f48 [0,4194304] 0 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.0.smithi012.stdout:5/589: dread - d3/d29/d36/d3d/d65/fa1 zero size 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.0.smithi012.stdout:5/590: dread - d3/d29/f9a zero size 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.0.smithi012.stdout:5/591: truncate d3/d29/d36/d9b/d59/d8b/fa0 145224 0 2024-03-18T14:51:00.384 INFO:tasks.workunit.client.0.smithi012.stdout:5/592: write d3/d29/d36/fb3 [915145,22851] 0 2024-03-18T14:51:00.385 INFO:tasks.workunit.client.0.smithi012.stdout:9/677: symlink d3/d20/dbf/d77/d52/d65/lde 0 2024-03-18T14:51:00.386 INFO:tasks.workunit.client.1.smithi100.stdout:5/216: mknod d29/d2d/d39/c44 0 2024-03-18T14:51:00.387 INFO:tasks.workunit.client.0.smithi012.stdout:9/678: write d3/d20/dbf/d77/da8/f87 [534240,108478] 0 2024-03-18T14:51:00.387 INFO:tasks.workunit.client.0.smithi012.stdout:9/679: fdatasync d3/d49/f57 0 2024-03-18T14:51:00.388 INFO:tasks.workunit.client.0.smithi012.stdout:0/614: truncate d1/d16/d19/f21 3144467 0 2024-03-18T14:51:00.388 INFO:tasks.workunit.client.1.smithi100.stdout:5/217: write f7 [6191370,20103] 0 2024-03-18T14:51:00.389 INFO:tasks.workunit.client.1.smithi100.stdout:5/218: creat d29/d2d/d39/f45 x:0 0 0 2024-03-18T14:51:00.391 INFO:tasks.workunit.client.1.smithi100.stdout:0/316: mknod dd/d39/d56/d62/d6e/c80 0 2024-03-18T14:51:00.391 INFO:tasks.workunit.client.1.smithi100.stdout:0/317: write dd/d39/d56/d59/f7c [411235,110699] 0 2024-03-18T14:51:00.392 INFO:tasks.workunit.client.1.smithi100.stdout:7/247: dread d5/f35 [0,4194304] 0 2024-03-18T14:51:00.394 INFO:tasks.workunit.client.0.smithi012.stdout:8/773: rename d0/d47/d59/dc9/d87/le9 to d0/d5a/d5e/dd7/lf3 0 2024-03-18T14:51:00.395 INFO:tasks.workunit.client.1.smithi100.stdout:8/264: creat d4/dc/dd/de/f53 x:0 0 0 2024-03-18T14:51:00.398 INFO:tasks.workunit.client.1.smithi100.stdout:9/270: truncate da/f4f 6265376 0 2024-03-18T14:51:00.400 INFO:tasks.workunit.client.1.smithi100.stdout:1/252: rename d8/f3d to d8/d2a/f47 0 2024-03-18T14:51:00.403 INFO:tasks.workunit.client.1.smithi100.stdout:4/256: dwrite d9/d17/f45 [0,4194304] 0 2024-03-18T14:51:00.403 INFO:tasks.workunit.client.1.smithi100.stdout:4/257: creat d9/d27/f4b x:0 0 0 2024-03-18T14:51:00.403 INFO:tasks.workunit.client.1.smithi100.stdout:4/258: readlink d9/l31 0 2024-03-18T14:51:00.403 INFO:tasks.workunit.client.1.smithi100.stdout:4/259: chown d9/d27/f3d 275 1 2024-03-18T14:51:00.404 INFO:tasks.workunit.client.0.smithi012.stdout:8/774: write d0/d5a/f25 [2795096,82251] 0 2024-03-18T14:51:00.404 INFO:tasks.workunit.client.0.smithi012.stdout:4/530: symlink d6/d76/lb2 0 2024-03-18T14:51:00.404 INFO:tasks.workunit.client.1.smithi100.stdout:4/260: dread d9/d13/f33 [0,4194304] 0 2024-03-18T14:51:00.404 INFO:tasks.workunit.client.1.smithi100.stdout:4/261: chown d9/l31 13236735 1 2024-03-18T14:51:00.405 INFO:tasks.workunit.client.0.smithi012.stdout:7/597: mknod dd/d13/d23/d33/d37/ccd 0 2024-03-18T14:51:00.405 INFO:tasks.workunit.client.0.smithi012.stdout:7/598: write dd/f45 [671086,95527] 0 2024-03-18T14:51:00.407 INFO:tasks.workunit.client.1.smithi100.stdout:0/318: symlink dd/d39/d7b/l81 0 2024-03-18T14:51:00.409 INFO:tasks.workunit.client.1.smithi100.stdout:7/248: creat d5/d22/f57 x:0 0 0 2024-03-18T14:51:00.418 INFO:tasks.workunit.client.1.smithi100.stdout:8/265: chown d4/l3f 0 1 2024-03-18T14:51:00.418 INFO:tasks.workunit.client.1.smithi100.stdout:8/266: truncate d4/dc/d1f/d33/f36 575219 0 2024-03-18T14:51:00.422 INFO:tasks.workunit.client.1.smithi100.stdout:0/319: dwrite dd/d61/f72 [0,4194304] 0 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.0.smithi012.stdout:5/593: truncate d3/d29/d36/d3d/d65/f6c 3819559 0 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.0.smithi012.stdout:5/594: write d3/d29/f61 [579386,54558] 0 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.0.smithi012.stdout:5/595: stat d3/d84 0 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.0.smithi012.stdout:5/596: chown d3/d84/fb2 35 1 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.0.smithi012.stdout:5/597: chown d3/d29/f95 0 1 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/271: rmdir d3/dd 39 2024-03-18T14:51:00.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/272: write d3/d16/d13/d17/f1a [1019790,11971] 0 2024-03-18T14:51:00.426 INFO:tasks.workunit.client.0.smithi012.stdout:9/680: mkdir d3/d20/dbf/d77/d52/d99/da0/ddf 0 2024-03-18T14:51:00.426 INFO:tasks.workunit.client.0.smithi012.stdout:9/681: fsync d3/d20/dbf/d77/d52/d65/f86 0 2024-03-18T14:51:00.426 INFO:tasks.workunit.client.0.smithi012.stdout:9/682: chown d3/cd0 0 1 2024-03-18T14:51:00.427 INFO:tasks.workunit.client.1.smithi100.stdout:9/271: mkdir da/d51 0 2024-03-18T14:51:00.427 INFO:tasks.workunit.client.1.smithi100.stdout:9/272: dread - da/dd/f42 zero size 2024-03-18T14:51:00.427 INFO:tasks.workunit.client.1.smithi100.stdout:9/273: fsync da/f4f 0 2024-03-18T14:51:00.427 INFO:tasks.workunit.client.1.smithi100.stdout:9/274: readlink da/l22 0 2024-03-18T14:51:00.428 INFO:tasks.workunit.client.0.smithi012.stdout:0/615: mkdir d1/d20/dcd 0 2024-03-18T14:51:00.431 INFO:tasks.workunit.client.1.smithi100.stdout:1/253: truncate d8/d1e/f31 512584 0 2024-03-18T14:51:00.432 INFO:tasks.workunit.client.0.smithi012.stdout:3/586: rename dc/d3c/d72/c8e to dc/d3c/d59/d86/cbd 0 2024-03-18T14:51:00.439 INFO:tasks.workunit.client.1.smithi100.stdout:4/262: mkdir d9/d13/d39/d4c 0 2024-03-18T14:51:00.442 INFO:tasks.workunit.client.1.smithi100.stdout:5/219: unlink l22 0 2024-03-18T14:51:00.444 INFO:tasks.workunit.client.1.smithi100.stdout:7/249: symlink d5/d30/d4d/l58 0 2024-03-18T14:51:00.445 INFO:tasks.workunit.client.0.smithi012.stdout:4/531: symlink d6/da/d89/lb3 0 2024-03-18T14:51:00.447 INFO:tasks.workunit.client.1.smithi100.stdout:8/267: mkdir d4/dc/dd/de/d21/d23/d47/d54 0 2024-03-18T14:51:00.447 INFO:tasks.workunit.client.1.smithi100.stdout:8/268: write d4/f15 [4551429,22847] 0 2024-03-18T14:51:00.448 INFO:tasks.workunit.client.1.smithi100.stdout:9/275: dwrite da/d25/f3a [0,4194304] 0 2024-03-18T14:51:00.451 INFO:tasks.workunit.client.0.smithi012.stdout:7/599: unlink dd/d13/d23/l3a 0 2024-03-18T14:51:00.451 INFO:tasks.workunit.client.0.smithi012.stdout:7/600: creat dd/d13/d23/fce x:0 0 0 2024-03-18T14:51:00.453 INFO:tasks.workunit.client.0.smithi012.stdout:2/666: rmdir d0/d10 39 2024-03-18T14:51:00.453 INFO:tasks.workunit.client.0.smithi012.stdout:2/667: fsync d0/d6b/dbf/f5b 0 2024-03-18T14:51:00.455 INFO:tasks.workunit.client.1.smithi100.stdout:0/320: rmdir dd/d39/d56/d59/d67/d79/d7d 39 2024-03-18T14:51:00.455 INFO:tasks.workunit.client.1.smithi100.stdout:0/321: chown dd/d39/d56/d59/f7c 3727 1 2024-03-18T14:51:00.457 INFO:tasks.workunit.client.0.smithi012.stdout:5/598: mkdir d3/d29/d9d/dc4 0 2024-03-18T14:51:00.458 INFO:tasks.workunit.client.1.smithi100.stdout:8/269: write d4/dc/dd/de/d21/d23/f27 [3843343,30431] 0 2024-03-18T14:51:00.460 INFO:tasks.workunit.client.0.smithi012.stdout:9/683: creat d3/d20/dbf/d66/d69/d7f/fe0 x:0 0 0 2024-03-18T14:51:00.462 INFO:tasks.workunit.client.1.smithi100.stdout:1/254: link d8/d30/f42 d8/d1e/f48 0 2024-03-18T14:51:00.462 INFO:tasks.workunit.client.1.smithi100.stdout:1/255: fsync d8/d1e/d11/f21 0 2024-03-18T14:51:00.462 INFO:tasks.workunit.client.1.smithi100.stdout:1/256: truncate d8/d2a/f44 882794 0 2024-03-18T14:51:00.462 INFO:tasks.workunit.client.1.smithi100.stdout:1/257: fdatasync d8/d1e/d11/d2d/f18 0 2024-03-18T14:51:00.464 INFO:tasks.workunit.client.0.smithi012.stdout:0/616: creat d1/d20/d23/d88/fce x:0 0 0 2024-03-18T14:51:00.465 INFO:tasks.workunit.client.1.smithi100.stdout:4/263: creat d9/d13/d25/f4d x:0 0 0 2024-03-18T14:51:00.467 INFO:tasks.workunit.client.0.smithi012.stdout:8/775: rename d0/d5a/d5e/ddf/d50/c74 to d0/d5a/d5e/dd7/cf4 0 2024-03-18T14:51:00.471 INFO:tasks.workunit.client.1.smithi100.stdout:5/220: rename d29/d2d/d39/f3e to d29/d2d/d39/f46 0 2024-03-18T14:51:00.471 INFO:tasks.workunit.client.0.smithi012.stdout:3/587: symlink dc/d3c/d59/d86/d9a/d54/lbe 0 2024-03-18T14:51:00.471 INFO:tasks.workunit.client.0.smithi012.stdout:3/588: write dc/d30/d3a/f5e [4042460,15581] 0 2024-03-18T14:51:00.475 INFO:tasks.workunit.client.1.smithi100.stdout:7/250: symlink d5/d14/d16/d44/l59 0 2024-03-18T14:51:00.475 INFO:tasks.workunit.client.1.smithi100.stdout:7/251: write d5/f35 [4786957,61270] 0 2024-03-18T14:51:00.475 INFO:tasks.workunit.client.1.smithi100.stdout:7/252: chown d5/l28 249672832 1 2024-03-18T14:51:00.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/601: symlink dd/d13/d23/lcf 0 2024-03-18T14:51:00.484 INFO:tasks.workunit.client.1.smithi100.stdout:7/253: dread d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:51:00.484 INFO:tasks.workunit.client.0.smithi012.stdout:7/602: write dd/f5b [159110,127212] 0 2024-03-18T14:51:00.485 INFO:tasks.workunit.client.0.smithi012.stdout:5/599: unlink d3/d15/f1d 0 2024-03-18T14:51:00.486 INFO:tasks.workunit.client.1.smithi100.stdout:0/322: mknod dd/d20/c82 0 2024-03-18T14:51:00.486 INFO:tasks.workunit.client.1.smithi100.stdout:0/323: truncate dd/f3a 744060 0 2024-03-18T14:51:00.487 INFO:tasks.workunit.client.0.smithi012.stdout:0/617: symlink d1/d2d/lcf 0 2024-03-18T14:51:00.487 INFO:tasks.workunit.client.0.smithi012.stdout:0/618: readlink d1/d16/d19/d8d/lc3 0 2024-03-18T14:51:00.487 INFO:tasks.workunit.client.0.smithi012.stdout:0/619: read d1/d16/d19/f21 [2711153,129601] 0 2024-03-18T14:51:00.489 INFO:tasks.workunit.client.0.smithi012.stdout:8/776: truncate d0/d5a/d19/f29 223743 0 2024-03-18T14:51:00.489 INFO:tasks.workunit.client.0.smithi012.stdout:8/777: chown d0/d5a/d5e/dd7/d84/d2a/d3e/lb6 180465 1 2024-03-18T14:51:00.489 INFO:tasks.workunit.client.0.smithi012.stdout:8/778: dread - d0/d5a/d19/d63/db0/fe1 zero size 2024-03-18T14:51:00.489 INFO:tasks.workunit.client.0.smithi012.stdout:3/589: symlink dc/d3c/lbf 0 2024-03-18T14:51:00.490 INFO:tasks.workunit.client.0.smithi012.stdout:4/532: dwrite d6/d1a/d30/d46/d6f/d7c/f87 [0,4194304] 0 2024-03-18T14:51:00.490 INFO:tasks.workunit.client.1.smithi100.stdout:9/276: dwrite da/d2c/f41 [0,4194304] 0 2024-03-18T14:51:00.491 INFO:tasks.workunit.client.0.smithi012.stdout:7/603: creat dd/d13/d23/d76/d6f/db3/fd0 x:0 0 0 2024-03-18T14:51:00.492 INFO:tasks.workunit.client.0.smithi012.stdout:5/600: rename d3/d1e/l63 to d3/d15/d42/lc5 0 2024-03-18T14:51:00.494 INFO:tasks.workunit.client.0.smithi012.stdout:3/590: creat dc/d3c/d72/fc0 x:0 0 0 2024-03-18T14:51:00.494 INFO:tasks.workunit.client.0.smithi012.stdout:3/591: chown dc/ca3 11 1 2024-03-18T14:51:00.495 INFO:tasks.workunit.client.1.smithi100.stdout:3/273: rmdir d3/d16/d13 39 2024-03-18T14:51:00.498 INFO:tasks.workunit.client.1.smithi100.stdout:4/264: mknod d9/d27/c4e 0 2024-03-18T14:51:00.498 INFO:tasks.workunit.client.0.smithi012.stdout:4/533: mknod d6/d1a/d98/daf/cb4 0 2024-03-18T14:51:00.498 INFO:tasks.workunit.client.0.smithi012.stdout:7/604: symlink dd/d13/d23/d33/d48/ld1 0 2024-03-18T14:51:00.501 INFO:tasks.workunit.client.0.smithi012.stdout:8/779: write d0/d15/f28 [384479,10481] 0 2024-03-18T14:51:00.501 INFO:tasks.workunit.client.0.smithi012.stdout:8/780: fsync d0/d47/d59/fca 0 2024-03-18T14:51:00.502 INFO:tasks.workunit.client.1.smithi100.stdout:7/254: mkdir d5/d30/d4d/d5a 0 2024-03-18T14:51:00.503 INFO:tasks.workunit.client.0.smithi012.stdout:7/605: link dd/d13/d23/d76/d6f/f91 dd/d13/d23/d2f/d53/fd2 0 2024-03-18T14:51:00.504 INFO:tasks.workunit.client.1.smithi100.stdout:0/324: link dd/c2a dd/d39/d56/d59/c83 0 2024-03-18T14:51:00.504 INFO:tasks.workunit.client.1.smithi100.stdout:0/325: dread - dd/d20/f2c zero size 2024-03-18T14:51:00.505 INFO:tasks.workunit.client.1.smithi100.stdout:0/326: creat dd/d39/d56/d62/f84 x:0 0 0 2024-03-18T14:51:00.507 INFO:tasks.workunit.client.1.smithi100.stdout:1/258: dwrite d8/d1e/d11/f21 [0,4194304] 0 2024-03-18T14:51:00.507 INFO:tasks.workunit.client.1.smithi100.stdout:1/259: stat d8/d1e/d11/d27/d3f/c43 0 2024-03-18T14:51:00.508 INFO:tasks.workunit.client.0.smithi012.stdout:2/668: dwrite d0/d10/f16 [0,4194304] 0 2024-03-18T14:51:00.509 INFO:tasks.workunit.client.1.smithi100.stdout:8/270: dwrite d4/dc/dd/de/d21/d23/d47/f50 [0,4194304] 0 2024-03-18T14:51:00.509 INFO:tasks.workunit.client.1.smithi100.stdout:8/271: write d4/f9 [2168024,48057] 0 2024-03-18T14:51:00.509 INFO:tasks.workunit.client.1.smithi100.stdout:8/272: fsync d4/dc/d1f/d33/f4e 0 2024-03-18T14:51:00.510 INFO:tasks.workunit.client.0.smithi012.stdout:9/684: dwrite d3/d20/fc2 [0,4194304] 0 2024-03-18T14:51:00.510 INFO:tasks.workunit.client.0.smithi012.stdout:9/685: chown d3/d49/cb5 446 1 2024-03-18T14:51:00.510 INFO:tasks.workunit.client.0.smithi012.stdout:5/601: rename d3/d29/d36/d3d to d3/d15/d42/dc6 0 2024-03-18T14:51:00.510 INFO:tasks.workunit.client.0.smithi012.stdout:5/602: write d3/d1e/f93 [972434,5831] 0 2024-03-18T14:51:00.511 INFO:tasks.workunit.client.1.smithi100.stdout:9/277: chown da/db/c19 83 1 2024-03-18T14:51:00.512 INFO:tasks.workunit.client.0.smithi012.stdout:0/620: dread d1/d20/d4e/d54/f57 [4194304,4194304] 0 2024-03-18T14:51:00.513 INFO:tasks.workunit.client.1.smithi100.stdout:3/274: mknod d3/d16/d13/d17/c64 0 2024-03-18T14:51:00.513 INFO:tasks.workunit.client.1.smithi100.stdout:3/275: write d3/d16/d13/d1f/f5c [757466,46380] 0 2024-03-18T14:51:00.513 INFO:tasks.workunit.client.1.smithi100.stdout:5/221: dwrite f17 [0,4194304] 0 2024-03-18T14:51:00.514 INFO:tasks.workunit.client.0.smithi012.stdout:7/606: link dd/d13/d23/d2f/d53/c88 dd/d13/d23/d33/dcc/cd3 0 2024-03-18T14:51:00.518 INFO:tasks.workunit.client.1.smithi100.stdout:4/265: truncate d9/f1f 2155349 0 2024-03-18T14:51:00.518 INFO:tasks.workunit.client.1.smithi100.stdout:4/266: readlink d9/l15 0 2024-03-18T14:51:00.519 INFO:tasks.workunit.client.0.smithi012.stdout:2/669: dread d0/d3c/f46 [0,4194304] 0 2024-03-18T14:51:00.522 INFO:tasks.workunit.client.1.smithi100.stdout:7/255: rename d5/d14/d16/d44/f4c to d5/d14/d16/d2c/f5b 0 2024-03-18T14:51:00.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/278: dread da/dd/f28 [0,4194304] 0 2024-03-18T14:51:00.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/279: truncate f6 1624547 0 2024-03-18T14:51:00.530 INFO:tasks.workunit.client.0.smithi012.stdout:2/670: dread d0/d3c/fb5 [0,4194304] 0 2024-03-18T14:51:00.532 INFO:tasks.workunit.client.1.smithi100.stdout:5/222: dread f23 [0,4194304] 0 2024-03-18T14:51:00.534 INFO:tasks.workunit.client.0.smithi012.stdout:4/534: dwrite d6/d19/f85 [0,4194304] 0 2024-03-18T14:51:00.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/260: creat d8/d1e/d11/d27/d3f/f49 x:0 0 0 2024-03-18T14:51:00.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/261: dread - d8/d30/f42 zero size 2024-03-18T14:51:00.539 INFO:tasks.workunit.client.0.smithi012.stdout:6/570: sync 2024-03-18T14:51:00.539 INFO:tasks.workunit.client.0.smithi012.stdout:6/571: fdatasync fe 0 2024-03-18T14:51:00.539 INFO:tasks.workunit.client.1.smithi100.stdout:9/280: dread da/f4f [0,4194304] 0 2024-03-18T14:51:00.540 INFO:tasks.workunit.client.0.smithi012.stdout:8/781: dwrite d0/d5a/d19/dad/f2d [4194304,4194304] 0 2024-03-18T14:51:00.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/603: dwrite d3/d15/d42/dc6/f6e [0,4194304] 0 2024-03-18T14:51:00.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/604: chown d3/d15/d42/dc6/d73/f92 0 1 2024-03-18T14:51:00.544 INFO:tasks.workunit.client.1.smithi100.stdout:3/276: mkdir d3/d16/d13/d1f/d3f/d4f/d65 0 2024-03-18T14:51:00.546 INFO:tasks.workunit.client.1.smithi100.stdout:0/327: dwrite dd/d20/f4d [0,4194304] 0 2024-03-18T14:51:00.546 INFO:tasks.workunit.client.1.smithi100.stdout:0/328: creat dd/f85 x:0 0 0 2024-03-18T14:51:00.546 INFO:tasks.workunit.client.1.smithi100.stdout:0/329: write dd/d20/f21 [740402,91588] 0 2024-03-18T14:51:00.546 INFO:tasks.workunit.client.1.smithi100.stdout:2/222: sync 2024-03-18T14:51:00.550 INFO:tasks.workunit.client.1.smithi100.stdout:8/273: rename d4/dc/dd/de/d21/d23/d41 to d4/dc/dd/de/d21/d23/d47/d54/d55 0 2024-03-18T14:51:00.553 INFO:tasks.workunit.client.0.smithi012.stdout:0/621: mkdir d1/d16/d19/dd0 0 2024-03-18T14:51:00.563 INFO:tasks.workunit.client.1.smithi100.stdout:1/262: mknod d8/d1e/c4a 0 2024-03-18T14:51:00.564 INFO:tasks.workunit.client.0.smithi012.stdout:0/622: dread d1/d20/d23/d6f/f75 [0,4194304] 0 2024-03-18T14:51:00.564 INFO:tasks.workunit.client.0.smithi012.stdout:0/623: write d1/d20/d23/d7f/faf [844720,24086] 0 2024-03-18T14:51:00.566 INFO:tasks.workunit.client.0.smithi012.stdout:2/671: truncate d0/d3c/d40/f5e 243641 0 2024-03-18T14:51:00.566 INFO:tasks.workunit.client.0.smithi012.stdout:7/607: fsync dd/d13/d23/d76/d6f/db3/fd0 0 2024-03-18T14:51:00.572 INFO:tasks.workunit.client.0.smithi012.stdout:4/535: mkdir d6/d1a/d98/daf/db5 0 2024-03-18T14:51:00.573 INFO:tasks.workunit.client.1.smithi100.stdout:2/223: creat d3/d6/d29/d36/d4a/f56 x:0 0 0 2024-03-18T14:51:00.576 INFO:tasks.workunit.client.1.smithi100.stdout:9/281: dwrite da/f1d [4194304,4194304] 0 2024-03-18T14:51:00.578 INFO:tasks.workunit.client.1.smithi100.stdout:9/282: write da/d2c/d2f/f30 [808627,123460] 0 2024-03-18T14:51:00.579 INFO:tasks.workunit.client.0.smithi012.stdout:8/782: unlink d0/d5a/d5e/dd7/d84/c2b 0 2024-03-18T14:51:00.579 INFO:tasks.workunit.client.0.smithi012.stdout:8/783: chown d0 407 1 2024-03-18T14:51:00.579 INFO:tasks.workunit.client.0.smithi012.stdout:8/784: readlink d0/d5a/d5e/ddf/d50/la7 0 2024-03-18T14:51:00.580 INFO:tasks.workunit.client.1.smithi100.stdout:8/274: mkdir d4/dc/dd/de/d21/d23/d47/d56 0 2024-03-18T14:51:00.580 INFO:tasks.workunit.client.1.smithi100.stdout:8/275: creat d4/dc/dd/de/f57 x:0 0 0 2024-03-18T14:51:00.581 INFO:tasks.workunit.client.1.smithi100.stdout:6/227: sync 2024-03-18T14:51:00.582 INFO:tasks.workunit.client.1.smithi100.stdout:7/256: truncate d5/f35 3346904 0 2024-03-18T14:51:00.582 INFO:tasks.workunit.client.1.smithi100.stdout:5/223: dwrite f15 [0,4194304] 0 2024-03-18T14:51:00.583 INFO:tasks.workunit.client.1.smithi100.stdout:5/224: readlink d29/d2d/d32/l40 0 2024-03-18T14:51:00.584 INFO:tasks.workunit.client.1.smithi100.stdout:3/277: dwrite d3/dd/d21/f22 [0,4194304] 0 2024-03-18T14:51:00.584 INFO:tasks.workunit.client.0.smithi012.stdout:6/572: dwrite d17/d19/d27/d2f/f62 [0,4194304] 0 2024-03-18T14:51:00.585 INFO:tasks.workunit.client.0.smithi012.stdout:6/573: truncate d17/d18/f45 259180 0 2024-03-18T14:51:00.585 INFO:tasks.workunit.client.0.smithi012.stdout:6/574: creat d17/d38/d51/fac x:0 0 0 2024-03-18T14:51:00.585 INFO:tasks.workunit.client.1.smithi100.stdout:1/263: creat d8/d1e/d11/d27/d3e/f4b x:0 0 0 2024-03-18T14:51:00.586 INFO:tasks.workunit.client.1.smithi100.stdout:1/264: write d8/d2a/f47 [452009,45956] 0 2024-03-18T14:51:00.586 INFO:tasks.workunit.client.1.smithi100.stdout:1/265: chown d8/d30/l38 84 1 2024-03-18T14:51:00.586 INFO:tasks.workunit.client.1.smithi100.stdout:1/266: truncate d8/d1e/f48 473516 0 2024-03-18T14:51:00.586 INFO:tasks.workunit.client.1.smithi100.stdout:0/330: dwrite dd/fe [0,4194304] 0 2024-03-18T14:51:00.590 INFO:tasks.workunit.client.1.smithi100.stdout:2/224: mknod d3/d6/d29/d36/c57 0 2024-03-18T14:51:00.595 INFO:tasks.workunit.client.0.smithi012.stdout:0/624: symlink d1/d20/d23/d88/ld1 0 2024-03-18T14:51:00.596 INFO:tasks.workunit.client.1.smithi100.stdout:9/283: creat da/d51/f52 x:0 0 0 2024-03-18T14:51:00.596 INFO:tasks.workunit.client.1.smithi100.stdout:9/284: fdatasync da/dd/d29/f2e 0 2024-03-18T14:51:00.597 INFO:tasks.workunit.client.0.smithi012.stdout:2/672: symlink d0/d6b/dbf/d33/d77/lcf 0 2024-03-18T14:51:00.598 INFO:tasks.workunit.client.0.smithi012.stdout:1/568: sync 2024-03-18T14:51:00.599 INFO:tasks.workunit.client.1.smithi100.stdout:9/285: dread da/dd/f10 [0,4194304] 0 2024-03-18T14:51:00.599 INFO:tasks.workunit.client.0.smithi012.stdout:7/608: truncate f1 486109 0 2024-03-18T14:51:00.599 INFO:tasks.workunit.client.0.smithi012.stdout:7/609: chown dd/c2d 0 1 2024-03-18T14:51:00.602 INFO:tasks.workunit.client.0.smithi012.stdout:4/536: mkdir d6/db6 0 2024-03-18T14:51:00.603 INFO:tasks.workunit.client.0.smithi012.stdout:1/569: dread d8/db/d10/d31/f6e [0,4194304] 0 2024-03-18T14:51:00.603 INFO:tasks.workunit.client.0.smithi012.stdout:1/570: dread - d8/db/d10/d31/d43/f69 zero size 2024-03-18T14:51:00.603 INFO:tasks.workunit.client.0.smithi012.stdout:1/571: creat d8/db/d10/d31/d81/fb3 x:0 0 0 2024-03-18T14:51:00.603 INFO:tasks.workunit.client.0.smithi012.stdout:1/572: stat d8/db/d10/d31/d43/d5e 0 2024-03-18T14:51:00.604 INFO:tasks.workunit.client.1.smithi100.stdout:6/228: truncate d6/d7/f8 460513 0 2024-03-18T14:51:00.604 INFO:tasks.workunit.client.1.smithi100.stdout:6/229: chown d6/d10 90 1 2024-03-18T14:51:00.604 INFO:tasks.workunit.client.1.smithi100.stdout:6/230: chown d6/d12/f3a 23483 1 2024-03-18T14:51:00.604 INFO:tasks.workunit.client.1.smithi100.stdout:6/231: write d6/d7/f40 [140749,74515] 0 2024-03-18T14:51:00.604 INFO:tasks.workunit.client.0.smithi012.stdout:8/785: truncate d0/d5a/d19/fa3 7339958 0 2024-03-18T14:51:00.605 INFO:tasks.workunit.client.0.smithi012.stdout:8/786: readlink d0/d5a/d5e/dd7/d84/d54/d62/dd0/lb8 0 2024-03-18T14:51:00.607 INFO:tasks.workunit.client.1.smithi100.stdout:7/257: mknod d5/d14/d16/d44/c5c 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/258: write d5/d14/d16/d2c/f5b [5123225,95628] 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/259: creat d5/d30/f5d x:0 0 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/260: getdents d5/d30/d4d/d5a 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:4/267: dread d9/f1f [0,4194304] 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:5/225: dwrite d29/d2d/d39/f45 [0,4194304] 0 2024-03-18T14:51:00.608 INFO:tasks.workunit.client.1.smithi100.stdout:5/226: chown d29/d2d/d32/d38 1 1 2024-03-18T14:51:00.610 INFO:tasks.workunit.client.0.smithi012.stdout:0/625: dread d1/d20/d7b/ff [0,4194304] 0 2024-03-18T14:51:00.610 INFO:tasks.workunit.client.0.smithi012.stdout:0/626: creat d1/d20/d23/d88/fd2 x:0 0 0 2024-03-18T14:51:00.611 INFO:tasks.workunit.client.0.smithi012.stdout:9/686: write d3/f12 [2266180,57905] 0 2024-03-18T14:51:00.611 INFO:tasks.workunit.client.0.smithi012.stdout:9/687: truncate d3/d20/dbf/d66/d69/d7f/d8c/dac/fae 430140 0 2024-03-18T14:51:00.618 INFO:tasks.workunit.client.0.smithi012.stdout:4/537: dwrite d6/d19/d39/f6d [0,4194304] 0 2024-03-18T14:51:00.618 INFO:tasks.workunit.client.0.smithi012.stdout:4/538: write d6/da/f54 [942598,102468] 0 2024-03-18T14:51:00.621 INFO:tasks.workunit.client.1.smithi100.stdout:3/278: mknod d3/d16/d13/d1f/d37/c66 0 2024-03-18T14:51:00.624 INFO:tasks.workunit.client.1.smithi100.stdout:1/267: symlink d8/d1e/d11/d27/d3e/l4c 0 2024-03-18T14:51:00.624 INFO:tasks.workunit.client.1.smithi100.stdout:1/268: chown d8/l15 54 1 2024-03-18T14:51:00.624 INFO:tasks.workunit.client.1.smithi100.stdout:1/269: write d8/f29 [996531,15061] 0 2024-03-18T14:51:00.628 INFO:tasks.workunit.client.0.smithi012.stdout:6/575: rename f14 to d17/d19/d27/d6e/fad 0 2024-03-18T14:51:00.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/225: mknod d3/d6/d29/d36/d49/c58 0 2024-03-18T14:51:00.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/226: write d3/d6/d29/d36/f4e [1028283,18537] 0 2024-03-18T14:51:00.632 INFO:tasks.workunit.client.0.smithi012.stdout:4/539: read d6/da/f18 [1843607,4196] 0 2024-03-18T14:51:00.634 INFO:tasks.workunit.client.0.smithi012.stdout:5/605: getdents d3/d15/d5f 0 2024-03-18T14:51:00.635 INFO:tasks.workunit.client.1.smithi100.stdout:1/270: write d8/d1e/d11/f1b [514170,87143] 0 2024-03-18T14:51:00.636 INFO:tasks.workunit.client.1.smithi100.stdout:4/268: dwrite d9/fb [0,4194304] 0 2024-03-18T14:51:00.637 INFO:tasks.workunit.client.1.smithi100.stdout:4/269: stat d9/d13/d25/d41 0 2024-03-18T14:51:00.637 INFO:tasks.workunit.client.1.smithi100.stdout:4/270: chown d9/la 0 1 2024-03-18T14:51:00.637 INFO:tasks.workunit.client.0.smithi012.stdout:9/688: dwrite d3/d20/dbf/d77/f56 [0,4194304] 0 2024-03-18T14:51:00.637 INFO:tasks.workunit.client.0.smithi012.stdout:5/606: dread d3/f22 [0,4194304] 0 2024-03-18T14:51:00.638 INFO:tasks.workunit.client.0.smithi012.stdout:5/607: truncate d3/d1e/d33/f74 697159 0 2024-03-18T14:51:00.638 INFO:tasks.workunit.client.0.smithi012.stdout:5/608: chown d3/d15/f37 119302 1 2024-03-18T14:51:00.640 INFO:tasks.workunit.client.1.smithi100.stdout:9/286: symlink da/d46/l53 0 2024-03-18T14:51:00.641 INFO:tasks.workunit.client.0.smithi012.stdout:7/610: symlink dd/d13/db7/ld4 0 2024-03-18T14:51:00.642 INFO:tasks.workunit.client.0.smithi012.stdout:7/611: fdatasync dd/d13/d23/d2f/d7a/dab/fc2 0 2024-03-18T14:51:00.643 INFO:tasks.workunit.client.1.smithi100.stdout:8/276: rename d4/l5 to d4/dc/l58 0 2024-03-18T14:51:00.646 INFO:tasks.workunit.client.0.smithi012.stdout:1/573: creat d8/db/d10/d16/da1/d2e/fb4 x:0 0 0 2024-03-18T14:51:00.646 INFO:tasks.workunit.client.0.smithi012.stdout:1/574: fsync d8/f40 0 2024-03-18T14:51:00.646 INFO:tasks.workunit.client.0.smithi012.stdout:1/575: dread - d8/db/d10/f60 zero size 2024-03-18T14:51:00.647 INFO:tasks.workunit.client.0.smithi012.stdout:8/787: getdents d0/d47/d59/dc9 0 2024-03-18T14:51:00.649 INFO:tasks.workunit.client.1.smithi100.stdout:6/232: mknod d6/d10/d27/c46 0 2024-03-18T14:51:00.650 INFO:tasks.workunit.client.0.smithi012.stdout:0/627: symlink d1/d20/d23/d7f/da7/ld3 0 2024-03-18T14:51:00.650 INFO:tasks.workunit.client.0.smithi012.stdout:0/628: read - d1/d20/d4e/d54/fc6 zero size 2024-03-18T14:51:00.651 INFO:tasks.workunit.client.0.smithi012.stdout:0/629: write d1/d16/d6e/fc7 [978543,26186] 0 2024-03-18T14:51:00.651 INFO:tasks.workunit.client.1.smithi100.stdout:8/277: dread d4/dc/dd/f44 [4194304,4194304] 0 2024-03-18T14:51:00.654 INFO:tasks.workunit.client.0.smithi012.stdout:6/576: unlink f12 0 2024-03-18T14:51:00.656 INFO:tasks.workunit.client.0.smithi012.stdout:6/577: dread d17/d19/f3c [0,4194304] 0 2024-03-18T14:51:00.659 INFO:tasks.workunit.client.1.smithi100.stdout:7/261: mknod d5/d22/d24/c5e 0 2024-03-18T14:51:00.662 INFO:tasks.workunit.client.0.smithi012.stdout:2/673: getdents d0/d6b/dbf 0 2024-03-18T14:51:00.667 INFO:tasks.workunit.client.1.smithi100.stdout:5/227: unlink d29/d2d/d32/d38/f43 0 2024-03-18T14:51:00.668 INFO:tasks.workunit.client.1.smithi100.stdout:6/233: dwrite d6/d10/d2b/f2d [0,4194304] 0 2024-03-18T14:51:00.668 INFO:tasks.workunit.client.1.smithi100.stdout:6/234: truncate d6/f2f 303296 0 2024-03-18T14:51:00.668 INFO:tasks.workunit.client.1.smithi100.stdout:6/235: read d6/d7/df/d26/f1c [420554,24956] 0 2024-03-18T14:51:00.669 INFO:tasks.workunit.client.0.smithi012.stdout:9/689: symlink d3/d20/dbf/d66/d21/d35/d48/le1 0 2024-03-18T14:51:00.670 INFO:tasks.workunit.client.0.smithi012.stdout:5/609: getdents d3/d29 0 2024-03-18T14:51:00.673 INFO:tasks.workunit.client.1.smithi100.stdout:7/262: write d5/d22/d3b/d47/f49 [3124694,100292] 0 2024-03-18T14:51:00.674 INFO:tasks.workunit.client.1.smithi100.stdout:7/263: readlink d5/l2f 0 2024-03-18T14:51:00.675 INFO:tasks.workunit.client.0.smithi012.stdout:7/612: mkdir dd/d13/d23/d33/dd5 0 2024-03-18T14:51:00.675 INFO:tasks.workunit.client.0.smithi012.stdout:7/613: truncate dd/d13/d23/d76/fb2 146410 0 2024-03-18T14:51:00.678 INFO:tasks.workunit.client.0.smithi012.stdout:1/576: mknod d8/d1e/d27/d44/cb5 0 2024-03-18T14:51:00.680 INFO:tasks.workunit.client.0.smithi012.stdout:2/674: dread d0/da2/f67 [0,4194304] 0 2024-03-18T14:51:00.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/788: symlink d0/d5a/d55/lf5 0 2024-03-18T14:51:00.683 INFO:tasks.workunit.client.1.smithi100.stdout:3/279: mknod d3/dd/c67 0 2024-03-18T14:51:00.688 INFO:tasks.workunit.client.1.smithi100.stdout:2/227: mknod d3/d1a/c59 0 2024-03-18T14:51:00.688 INFO:tasks.workunit.client.1.smithi100.stdout:2/228: stat d3/f10 0 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:0/630: truncate d1/d16/d31/f1d 2813038 0 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:0/631: readlink d1/d2d/la2 0 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:0/632: dread - d1/d20/d23/d3b/d33/d51/fa8 zero size 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:2/675: dread d0/d6b/dbf/d33/d71/d90/fae [0,4194304] 0 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:0/633: truncate d1/d16/d19/d8d/fad 879752 0 2024-03-18T14:51:00.690 INFO:tasks.workunit.client.0.smithi012.stdout:0/634: fdatasync d1/d16/f17 0 2024-03-18T14:51:00.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: [18/Mar/2024:14:50:58] ENGINE Bus STARTING 2024-03-18T14:51:00.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: [18/Mar/2024:14:50:58] ENGINE Serving on https://172.21.15.100:7150 2024-03-18T14:51:00.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: [18/Mar/2024:14:50:58] ENGINE Serving on http://172.21.15.100:8765 2024-03-18T14:51:00.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: [18/Mar/2024:14:50:58] ENGINE Bus STARTED 2024-03-18T14:51:00.691 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: pgmap v4: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:00.691 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:00 smithi012 ceph-mon[29182]: mgrmap e23: smithi100.rwnrcn(active, since 2s) 2024-03-18T14:51:00.692 INFO:tasks.workunit.client.0.smithi012.stdout:6/578: mknod d17/d33/cae 0 2024-03-18T14:51:00.695 INFO:tasks.workunit.client.1.smithi100.stdout:1/271: link d8/d1e/d11/f33 d8/d1e/d11/d27/f4d 0 2024-03-18T14:51:00.695 INFO:tasks.workunit.client.1.smithi100.stdout:1/272: chown d8/f29 2000412481 1 2024-03-18T14:51:00.695 INFO:tasks.workunit.client.1.smithi100.stdout:5/228: dwrite d29/d2d/f41 [0,4194304] 0 2024-03-18T14:51:00.695 INFO:tasks.workunit.client.0.smithi012.stdout:5/610: dwrite d3/d29/d36/d9b/d59/f7c [0,4194304] 0 2024-03-18T14:51:00.696 INFO:tasks.workunit.client.0.smithi012.stdout:4/540: rmdir d6/d19/d6c 39 2024-03-18T14:51:00.700 INFO:tasks.workunit.client.0.smithi012.stdout:7/614: mknod dd/d13/d23/d2f/d7a/dab/cd6 0 2024-03-18T14:51:00.700 INFO:tasks.workunit.client.1.smithi100.stdout:2/229: write f0 [801967,32161] 0 2024-03-18T14:51:00.700 INFO:tasks.workunit.client.1.smithi100.stdout:3/280: dread d3/dd/d49/f5d [0,4194304] 0 2024-03-18T14:51:00.700 INFO:tasks.workunit.client.1.smithi100.stdout:3/281: truncate d3/d16/f14 1249505 0 2024-03-18T14:51:00.701 INFO:tasks.workunit.client.1.smithi100.stdout:4/271: unlink d9/d3a/f46 0 2024-03-18T14:51:00.703 INFO:tasks.workunit.client.0.smithi012.stdout:1/577: symlink d8/db/d10/d31/d45/lb6 0 2024-03-18T14:51:00.703 INFO:tasks.workunit.client.0.smithi012.stdout:1/578: chown d8/db/l1b 221905667 1 2024-03-18T14:51:00.704 INFO:tasks.workunit.client.1.smithi100.stdout:5/229: write d29/d2d/f41 [91864,72604] 0 2024-03-18T14:51:00.705 INFO:tasks.workunit.client.1.smithi100.stdout:5/230: truncate f25 315005 0 2024-03-18T14:51:00.705 INFO:tasks.workunit.client.1.smithi100.stdout:9/287: rename da/d2c/f41 to da/dd/f54 0 2024-03-18T14:51:00.708 INFO:tasks.workunit.client.0.smithi012.stdout:8/789: rename d0/d5a/d19/fa3 to d0/d5a/d5e/dd7/d84/d54/ff6 0 2024-03-18T14:51:00.710 INFO:tasks.workunit.client.1.smithi100.stdout:8/278: mkdir d4/dc/d1f/d59 0 2024-03-18T14:51:00.711 INFO:tasks.workunit.client.0.smithi012.stdout:4/541: dread d6/d19/f85 [0,4194304] 0 2024-03-18T14:51:00.713 INFO:tasks.workunit.client.0.smithi012.stdout:7/615: dread dd/f10 [0,4194304] 0 2024-03-18T14:51:00.713 INFO:tasks.workunit.client.1.smithi100.stdout:2/230: write d3/d6/d29/f35 [3618219,80857] 0 2024-03-18T14:51:00.714 INFO:tasks.workunit.client.1.smithi100.stdout:2/231: write d3/fc [51289,91440] 0 2024-03-18T14:51:00.714 INFO:tasks.workunit.client.1.smithi100.stdout:8/279: dread d4/f3a [0,4194304] 0 2024-03-18T14:51:00.715 INFO:tasks.workunit.client.1.smithi100.stdout:5/231: dread f7 [4194304,4194304] 0 2024-03-18T14:51:00.715 INFO:tasks.workunit.client.0.smithi012.stdout:2/676: mkdir d0/d6b/dbf/d33/dac/dd0 0 2024-03-18T14:51:00.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/236: mknod d6/d7/df/d26/c47 0 2024-03-18T14:51:00.718 INFO:tasks.workunit.client.1.smithi100.stdout:3/282: dread d3/dd/f27 [4194304,4194304] 0 2024-03-18T14:51:00.718 INFO:tasks.workunit.client.1.smithi100.stdout:3/283: fdatasync d3/d16/f45 0 2024-03-18T14:51:00.719 INFO:tasks.workunit.client.0.smithi012.stdout:0/635: mknod d1/d20/d4e/d54/d67/dbc/cd4 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/237: dread d6/d10/f13 [0,4194304] 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/238: read - d6/d7/df/d26/f45 zero size 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/239: write d6/d7/df/f1e [562573,36716] 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/240: fsync d6/d10/d2b/f2d 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/241: write d6/d7/f40 [803420,80297] 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/242: write d6/d7/f3d [2413098,47894] 0 2024-03-18T14:51:00.723 INFO:tasks.workunit.client.1.smithi100.stdout:6/243: stat d6/d12/f3a 0 2024-03-18T14:51:00.724 INFO:tasks.workunit.client.1.smithi100.stdout:7/264: symlink d5/d22/d24/l5f 0 2024-03-18T14:51:00.725 INFO:tasks.workunit.client.0.smithi012.stdout:6/579: creat d17/d38/d51/d66/faf x:0 0 0 2024-03-18T14:51:00.726 INFO:tasks.workunit.client.1.smithi100.stdout:1/273: symlink d8/d1e/d11/d27/l4e 0 2024-03-18T14:51:00.728 INFO:tasks.workunit.client.1.smithi100.stdout:2/232: write d3/d6/f2f [3747521,112005] 0 2024-03-18T14:51:00.731 INFO:tasks.workunit.client.0.smithi012.stdout:5/611: mknod d3/d15/d42/dc6/d86/cc7 0 2024-03-18T14:51:00.731 INFO:tasks.workunit.client.0.smithi012.stdout:5/612: write d3/d29/f61 [711088,114206] 0 2024-03-18T14:51:00.731 INFO:tasks.workunit.client.0.smithi012.stdout:5/613: creat d3/d29/d36/d9b/fc8 x:0 0 0 2024-03-18T14:51:00.731 INFO:tasks.workunit.client.0.smithi012.stdout:5/614: fdatasync d3/d15/f1c 0 2024-03-18T14:51:00.734 INFO:tasks.workunit.client.0.smithi012.stdout:0/636: dread d1/d20/d23/d7f/fca [0,4194304] 0 2024-03-18T14:51:00.734 INFO:tasks.workunit.client.0.smithi012.stdout:9/690: symlink d3/d20/dbf/d77/da8/le2 0 2024-03-18T14:51:00.734 INFO:tasks.workunit.client.0.smithi012.stdout:9/691: write d3/d20/dbf/d66/d69/fb2 [179683,34576] 0 2024-03-18T14:51:00.736 INFO:tasks.workunit.client.1.smithi100.stdout:4/272: symlink d9/d17/l4f 0 2024-03-18T14:51:00.736 INFO:tasks.workunit.client.1.smithi100.stdout:4/273: write d9/d27/f36 [1459578,8401] 0 2024-03-18T14:51:00.737 INFO:tasks.workunit.client.0.smithi012.stdout:8/790: mknod d0/d5a/d19/cf7 0 2024-03-18T14:51:00.741 INFO:tasks.workunit.client.0.smithi012.stdout:4/542: symlink d6/d1a/d98/daf/lb7 0 2024-03-18T14:51:00.742 INFO:tasks.workunit.client.1.smithi100.stdout:9/288: dwrite da/db/f1a [0,4194304] 0 2024-03-18T14:51:00.746 INFO:tasks.workunit.client.0.smithi012.stdout:7/616: rename dd/d13/d23/d2f/f4c to dd/d13/d23/d60/d9c/fd7 0 2024-03-18T14:51:00.746 INFO:tasks.workunit.client.0.smithi012.stdout:7/617: chown dd/d13/d23/d2f/d53/c64 65 1 2024-03-18T14:51:00.746 INFO:tasks.workunit.client.0.smithi012.stdout:9/692: dread d3/d49/fd4 [0,4194304] 0 2024-03-18T14:51:00.747 INFO:tasks.workunit.client.0.smithi012.stdout:1/579: dwrite d8/d1e/d27/d44/fa0 [0,4194304] 0 2024-03-18T14:51:00.748 INFO:tasks.workunit.client.0.smithi012.stdout:2/677: mknod d0/d3c/d40/d47/d85/cd1 0 2024-03-18T14:51:00.755 INFO:tasks.workunit.client.0.smithi012.stdout:5/615: creat d3/db6/fc9 x:0 0 0 2024-03-18T14:51:00.755 INFO:tasks.workunit.client.0.smithi012.stdout:5/616: fsync d3/d1e/f1a 0 2024-03-18T14:51:00.755 INFO:tasks.workunit.client.0.smithi012.stdout:5/617: creat d3/db6/fca x:0 0 0 2024-03-18T14:51:00.755 INFO:tasks.workunit.client.1.smithi100.stdout:6/244: rmdir d6/d12 39 2024-03-18T14:51:00.756 INFO:tasks.workunit.client.1.smithi100.stdout:6/245: read d6/d7/df/d26/d32/f33 [110907,9710] 0 2024-03-18T14:51:00.757 INFO:tasks.workunit.client.0.smithi012.stdout:8/791: link d0/d5a/d19/l56 d0/d5a/d5e/dd7/d84/d54/da4/lf8 0 2024-03-18T14:51:00.757 INFO:tasks.workunit.client.0.smithi012.stdout:8/792: chown d0/d5a/d19/l39 13992805 1 2024-03-18T14:51:00.757 INFO:tasks.workunit.client.0.smithi012.stdout:8/793: chown d0/d15/c31 0 1 2024-03-18T14:51:00.758 INFO:tasks.workunit.client.0.smithi012.stdout:8/794: write d0/f6 [2561638,116107] 0 2024-03-18T14:51:00.763 INFO:tasks.workunit.client.0.smithi012.stdout:7/618: getdents dd/d13/d23/d33/dcc 0 2024-03-18T14:51:00.765 INFO:tasks.workunit.client.0.smithi012.stdout:9/693: creat d3/d20/dbf/fe3 x:0 0 0 2024-03-18T14:51:00.770 INFO:tasks.workunit.client.0.smithi012.stdout:4/543: dread d6/d1a/f2d [0,4194304] 0 2024-03-18T14:51:00.770 INFO:tasks.workunit.client.0.smithi012.stdout:4/544: fsync d6/d19/d39/f50 0 2024-03-18T14:51:00.772 INFO:tasks.workunit.client.0.smithi012.stdout:7/619: dread dd/d13/f72 [0,4194304] 0 2024-03-18T14:51:00.776 INFO:tasks.workunit.client.0.smithi012.stdout:6/580: dwrite d17/d19/d27/d6e/fad [0,4194304] 0 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:1/274: dwrite d8/d1e/d11/f33 [0,4194304] 0 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:1/275: dread - d8/d1e/d11/d27/d3f/f49 zero size 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:9/289: mkdir da/d51/d55 0 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:9/290: read da/dd/f27 [379933,100033] 0 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:9/291: chown da/dd/c2a 127 1 2024-03-18T14:51:00.777 INFO:tasks.workunit.client.1.smithi100.stdout:9/292: dread - da/dd/f42 zero size 2024-03-18T14:51:00.778 INFO:tasks.workunit.client.0.smithi012.stdout:0/637: dwrite d1/d16/d6e/f9b [0,4194304] 0 2024-03-18T14:51:00.779 INFO:tasks.workunit.client.1.smithi100.stdout:2/233: dwrite d3/d13/f37 [0,4194304] 0 2024-03-18T14:51:00.779 INFO:tasks.workunit.client.1.smithi100.stdout:2/234: getdents d3/d13/d40/d53 0 2024-03-18T14:51:00.779 INFO:tasks.workunit.client.1.smithi100.stdout:8/280: mkdir d4/d5a 0 2024-03-18T14:51:00.779 INFO:tasks.workunit.client.1.smithi100.stdout:5/232: unlink c19 0 2024-03-18T14:51:00.783 INFO:tasks.workunit.client.0.smithi012.stdout:1/580: unlink d8/db/d10/d31/d45/l5d 0 2024-03-18T14:51:00.784 INFO:tasks.workunit.client.1.smithi100.stdout:6/246: mkdir d6/d7/df/d26/d32/d48 0 2024-03-18T14:51:00.786 INFO:tasks.workunit.client.0.smithi012.stdout:2/678: chown d0/d10/c20 2 1 2024-03-18T14:51:00.788 INFO:tasks.workunit.client.1.smithi100.stdout:7/265: rmdir d5/d22 39 2024-03-18T14:51:00.788 INFO:tasks.workunit.client.1.smithi100.stdout:7/266: creat d5/d14/d16/d21/f60 x:0 0 0 2024-03-18T14:51:00.791 INFO:tasks.workunit.client.0.smithi012.stdout:5/618: chown d3/d15/d42/dc6/d73/d6b/f98 6 1 2024-03-18T14:51:00.793 INFO:tasks.workunit.client.1.smithi100.stdout:9/293: creat da/d46/d4d/f56 x:0 0 0 2024-03-18T14:51:00.793 INFO:tasks.workunit.client.1.smithi100.stdout:9/294: chown da/d2c/c49 15119320 1 2024-03-18T14:51:00.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: [18/Mar/2024:14:50:58] ENGINE Bus STARTING 2024-03-18T14:51:00.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: [18/Mar/2024:14:50:58] ENGINE Serving on https://172.21.15.100:7150 2024-03-18T14:51:00.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: [18/Mar/2024:14:50:58] ENGINE Serving on http://172.21.15.100:8765 2024-03-18T14:51:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: [18/Mar/2024:14:50:58] ENGINE Bus STARTED 2024-03-18T14:51:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: pgmap v4: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:00 smithi100 ceph-mon[37807]: mgrmap e23: smithi100.rwnrcn(active, since 2s) 2024-03-18T14:51:00.796 INFO:tasks.workunit.client.0.smithi012.stdout:0/638: dwrite d1/d20/d23/d3b/d33/f8b [0,4194304] 0 2024-03-18T14:51:00.799 INFO:tasks.workunit.client.1.smithi100.stdout:2/235: link d3/d6/c8 d3/d13/d40/d53/c5a 0 2024-03-18T14:51:00.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/281: unlink d4/d3c/f49 0 2024-03-18T14:51:00.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/282: creat d4/dc/dd/de/d21/d23/d47/f5b x:0 0 0 2024-03-18T14:51:00.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/283: creat d4/dc/d1f/f5c x:0 0 0 2024-03-18T14:51:00.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/284: chown d4/dc/d1f/d33 1966 1 2024-03-18T14:51:00.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/285: chown d4/dc/dd/de/d21 3416433 1 2024-03-18T14:51:00.806 INFO:tasks.workunit.client.0.smithi012.stdout:4/545: mknod d6/da/d89/d95/cb8 0 2024-03-18T14:51:00.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/267: dwrite d5/d22/d24/d39/f45 [0,4194304] 0 2024-03-18T14:51:00.810 INFO:tasks.workunit.client.0.smithi012.stdout:7/620: creat dd/d13/d23/d76/d6f/db3/dc7/fd8 x:0 0 0 2024-03-18T14:51:00.811 INFO:tasks.workunit.client.1.smithi100.stdout:6/247: symlink d6/d10/d2b/d36/l49 0 2024-03-18T14:51:00.811 INFO:tasks.workunit.client.1.smithi100.stdout:6/248: fdatasync f4 0 2024-03-18T14:51:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:9/694: dread d3/f6 [0,4194304] 0 2024-03-18T14:51:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:9/695: fdatasync d3/d9/f13 0 2024-03-18T14:51:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:9/696: chown d3/f63 738 1 2024-03-18T14:51:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:6/581: creat d17/d26/d36/d4d/d82/fb0 x:0 0 0 2024-03-18T14:51:00.814 INFO:tasks.workunit.client.0.smithi012.stdout:6/582: chown d17/d26/d36/f4b 13679915 1 2024-03-18T14:51:00.815 INFO:tasks.workunit.client.0.smithi012.stdout:6/583: chown d17/d19/f2b 395643385 1 2024-03-18T14:51:00.818 INFO:tasks.workunit.client.1.smithi100.stdout:9/295: dwrite da/db/f36 [0,4194304] 0 2024-03-18T14:51:00.818 INFO:tasks.workunit.client.1.smithi100.stdout:9/296: readlink da/l22 0 2024-03-18T14:51:00.822 INFO:tasks.workunit.client.0.smithi012.stdout:1/581: rename d8/db/d10/f7f to d8/d1e/d27/fb7 0 2024-03-18T14:51:00.824 INFO:tasks.workunit.client.0.smithi012.stdout:2/679: mknod d0/d3c/cd2 0 2024-03-18T14:51:00.826 INFO:tasks.workunit.client.0.smithi012.stdout:9/697: write d3/d20/dbf/d77/f56 [13796,36729] 0 2024-03-18T14:51:00.828 INFO:tasks.workunit.client.1.smithi100.stdout:8/286: rename f3 to d4/dc/d1f/f5d 0 2024-03-18T14:51:00.828 INFO:tasks.workunit.client.0.smithi012.stdout:7/621: dwrite dd/d13/d23/d33/f78 [4194304,4194304] 0 2024-03-18T14:51:00.828 INFO:tasks.workunit.client.0.smithi012.stdout:7/622: chown dd/d13/d23/d76/d6b 176708787 1 2024-03-18T14:51:00.828 INFO:tasks.workunit.client.0.smithi012.stdout:7/623: dread - dd/d13/f57 zero size 2024-03-18T14:51:00.829 INFO:tasks.workunit.client.0.smithi012.stdout:7/624: chown dd/d13/d23/d76/d6f/db3/lc9 0 1 2024-03-18T14:51:00.829 INFO:tasks.workunit.client.0.smithi012.stdout:7/625: chown dd/d13/d23/d60 16046787 1 2024-03-18T14:51:00.829 INFO:tasks.workunit.client.0.smithi012.stdout:5/619: mknod d3/d1e/ccb 0 2024-03-18T14:51:00.830 INFO:tasks.workunit.client.0.smithi012.stdout:9/698: read d3/d20/f31 [2402135,80023] 0 2024-03-18T14:51:00.831 INFO:tasks.workunit.client.1.smithi100.stdout:5/233: rmdir d29/d2d/d39 39 2024-03-18T14:51:00.835 INFO:tasks.workunit.client.1.smithi100.stdout:7/268: symlink d5/d14/d16/d21/l61 0 2024-03-18T14:51:00.837 INFO:tasks.workunit.client.1.smithi100.stdout:6/249: creat d6/d7/df/d26/d32/d48/f4a x:0 0 0 2024-03-18T14:51:00.838 INFO:tasks.workunit.client.1.smithi100.stdout:6/250: chown d6/d7/l41 35455 1 2024-03-18T14:51:00.839 INFO:tasks.workunit.client.1.smithi100.stdout:9/297: dwrite f6 [0,4194304] 0 2024-03-18T14:51:00.839 INFO:tasks.workunit.client.0.smithi012.stdout:4/546: symlink d6/d19/d39/lb9 0 2024-03-18T14:51:00.846 INFO:tasks.workunit.client.0.smithi012.stdout:1/582: rename d8/db/d10/f73 to d8/db/d10/d31/d43/d5e/da5/fb8 0 2024-03-18T14:51:00.848 INFO:tasks.workunit.client.1.smithi100.stdout:8/287: mknod d4/d3c/c5e 0 2024-03-18T14:51:00.848 INFO:tasks.workunit.client.1.smithi100.stdout:8/288: chown d4/dc/c10 1 1 2024-03-18T14:51:00.849 INFO:tasks.workunit.client.0.smithi012.stdout:2/680: symlink d0/db4/ld3 0 2024-03-18T14:51:00.851 INFO:tasks.workunit.client.1.smithi100.stdout:6/251: symlink d6/d7/df/d26/l4b 0 2024-03-18T14:51:00.855 INFO:tasks.workunit.client.0.smithi012.stdout:7/626: mkdir dd/d13/d23/d76/d6f/dd9 0 2024-03-18T14:51:00.855 INFO:tasks.workunit.client.0.smithi012.stdout:7/627: dread - dd/d13/d23/d2f/fb9 zero size 2024-03-18T14:51:00.857 INFO:tasks.workunit.client.1.smithi100.stdout:6/252: creat d6/d12/d2c/f4c x:0 0 0 2024-03-18T14:51:00.857 INFO:tasks.workunit.client.0.smithi012.stdout:3/592: sync 2024-03-18T14:51:00.857 INFO:tasks.workunit.client.0.smithi012.stdout:3/593: chown dc/d91/la2 4 1 2024-03-18T14:51:00.859 INFO:tasks.workunit.client.0.smithi012.stdout:5/620: unlink d3/d1e/c20 0 2024-03-18T14:51:00.861 INFO:tasks.workunit.client.0.smithi012.stdout:9/699: creat d3/d20/dbf/d77/d52/d95/da9/fe4 x:0 0 0 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:6/253: symlink d6/d12/l4d 0 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:6/254: read - d6/d7/df/d26/f45 zero size 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:6/255: chown d6/d7/df/f1e 1546229 1 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/234: dwrite d29/f2b [0,4194304] 0 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/235: stat d29/d2d/d32/d38/c3f 0 2024-03-18T14:51:00.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/236: dread - d29/f2a zero size 2024-03-18T14:51:00.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/331: sync 2024-03-18T14:51:00.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/332: chown dd/d20/f4d 0 1 2024-03-18T14:51:00.870 INFO:tasks.workunit.client.0.smithi012.stdout:7/628: write dd/d13/d23/d60/f89 [451975,16003] 0 2024-03-18T14:51:00.870 INFO:tasks.workunit.client.0.smithi012.stdout:7/629: stat dd/d13/d23/d2f/d53/d8d 0 2024-03-18T14:51:00.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/639: getdents d1/d20/d4e/d54 0 2024-03-18T14:51:00.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/640: write d1/d20/d4e/d54/d67/fbf [507958,96193] 0 2024-03-18T14:51:00.872 INFO:tasks.workunit.client.1.smithi100.stdout:5/237: mknod d29/d2d/d32/c47 0 2024-03-18T14:51:00.872 INFO:tasks.workunit.client.1.smithi100.stdout:5/238: fdatasync fa 0 2024-03-18T14:51:00.874 INFO:tasks.workunit.client.1.smithi100.stdout:9/298: dwrite da/d46/d4d/f56 [0,4194304] 0 2024-03-18T14:51:00.875 INFO:tasks.workunit.client.1.smithi100.stdout:0/333: mknod dd/d39/c86 0 2024-03-18T14:51:00.875 INFO:tasks.workunit.client.1.smithi100.stdout:0/334: readlink dd/d2b/l36 0 2024-03-18T14:51:00.876 INFO:tasks.workunit.client.1.smithi100.stdout:0/335: read dd/d20/f21 [104959,103218] 0 2024-03-18T14:51:00.877 INFO:tasks.workunit.client.1.smithi100.stdout:5/239: mknod d29/d2d/d32/d38/c48 0 2024-03-18T14:51:00.877 INFO:tasks.workunit.client.0.smithi012.stdout:6/584: rmdir d17/d19/d27/d58 39 2024-03-18T14:51:00.879 INFO:tasks.workunit.client.0.smithi012.stdout:7/630: dread dd/d13/d23/d76/d6b/f9a [0,4194304] 0 2024-03-18T14:51:00.879 INFO:tasks.workunit.client.0.smithi012.stdout:7/631: fsync dd/f5b 0 2024-03-18T14:51:00.879 INFO:tasks.workunit.client.0.smithi012.stdout:7/632: chown dd/d13/d23/d60/d9c/fd7 1934161074 1 2024-03-18T14:51:00.882 INFO:tasks.workunit.client.0.smithi012.stdout:1/583: mkdir d8/db/d10/d16/da1/db9 0 2024-03-18T14:51:00.885 INFO:tasks.workunit.client.1.smithi100.stdout:0/336: rename c0 to dd/d2b/d54/c87 0 2024-03-18T14:51:00.886 INFO:tasks.workunit.client.0.smithi012.stdout:2/681: creat d0/d6b/d9f/fd4 x:0 0 0 2024-03-18T14:51:00.886 INFO:tasks.workunit.client.0.smithi012.stdout:2/682: chown d0/da2/d59 69223 1 2024-03-18T14:51:00.886 INFO:tasks.workunit.client.0.smithi012.stdout:2/683: chown d0/d6b/dbf/d57/d61 121287904 1 2024-03-18T14:51:00.887 INFO:tasks.workunit.client.1.smithi100.stdout:7/269: dwrite d5/f52 [4194304,4194304] 0 2024-03-18T14:51:00.887 INFO:tasks.workunit.client.1.smithi100.stdout:8/289: dwrite d4/fa [0,4194304] 0 2024-03-18T14:51:00.887 INFO:tasks.workunit.client.1.smithi100.stdout:5/240: rmdir d29/d2d/d32 39 2024-03-18T14:51:00.888 INFO:tasks.workunit.client.0.smithi012.stdout:3/594: stat dc/l1d 0 2024-03-18T14:51:00.890 INFO:tasks.workunit.client.1.smithi100.stdout:7/270: mkdir d5/d14/d16/d2e/d62 0 2024-03-18T14:51:00.890 INFO:tasks.workunit.client.1.smithi100.stdout:7/271: stat d5/d14/d16/f1a 0 2024-03-18T14:51:00.891 INFO:tasks.workunit.client.0.smithi012.stdout:5/621: unlink d3/d15/d5f/fa6 0 2024-03-18T14:51:00.891 INFO:tasks.workunit.client.0.smithi012.stdout:5/622: fdatasync d3/fc 0 2024-03-18T14:51:00.892 INFO:tasks.workunit.client.1.smithi100.stdout:6/256: dwrite d6/d7/df/d26/f1c [0,4194304] 0 2024-03-18T14:51:00.893 INFO:tasks.workunit.client.1.smithi100.stdout:5/241: dread f1b [0,4194304] 0 2024-03-18T14:51:00.894 INFO:tasks.workunit.client.1.smithi100.stdout:9/299: dwrite da/dd/f27 [0,4194304] 0 2024-03-18T14:51:00.895 INFO:tasks.workunit.client.0.smithi012.stdout:9/700: unlink d3/d20/dbf/d77/c94 0 2024-03-18T14:51:00.900 INFO:tasks.workunit.client.1.smithi100.stdout:5/242: dread f26 [0,4194304] 0 2024-03-18T14:51:00.901 INFO:tasks.workunit.client.1.smithi100.stdout:8/290: dread d4/fa [0,4194304] 0 2024-03-18T14:51:00.901 INFO:tasks.workunit.client.1.smithi100.stdout:6/257: dread d6/f2f [0,4194304] 0 2024-03-18T14:51:00.901 INFO:tasks.workunit.client.1.smithi100.stdout:8/291: fsync d4/dc/f2d 0 2024-03-18T14:51:00.902 INFO:tasks.workunit.client.1.smithi100.stdout:3/284: sync 2024-03-18T14:51:00.905 INFO:tasks.workunit.client.1.smithi100.stdout:0/337: dread dd/f41 [4194304,4194304] 0 2024-03-18T14:51:00.905 INFO:tasks.workunit.client.1.smithi100.stdout:0/338: rename dd/d39/d56/d62/d6e to dd/d39/d56/d62/d6e/d88 22 2024-03-18T14:51:00.908 INFO:tasks.workunit.client.0.smithi012.stdout:4/547: mknod d6/d1a/cba 0 2024-03-18T14:51:00.910 INFO:tasks.workunit.client.0.smithi012.stdout:6/585: mkdir d17/d19/d27/db1 0 2024-03-18T14:51:00.914 INFO:tasks.workunit.client.0.smithi012.stdout:7/633: link dd/d13/d23/d33/d48/ld1 dd/lda 0 2024-03-18T14:51:00.917 INFO:tasks.workunit.client.0.smithi012.stdout:1/584: link d8/d1e/d27/f3c d8/db/d10/d31/fba 0 2024-03-18T14:51:00.918 INFO:tasks.workunit.client.1.smithi100.stdout:5/243: truncate f1b 377598 0 2024-03-18T14:51:00.919 INFO:tasks.workunit.client.1.smithi100.stdout:6/258: mkdir d6/d7/df/d26/d4e 0 2024-03-18T14:51:00.921 INFO:tasks.workunit.client.0.smithi012.stdout:2/684: creat d0/d6b/dbf/d33/fd5 x:0 0 0 2024-03-18T14:51:00.922 INFO:tasks.workunit.client.1.smithi100.stdout:8/292: creat d4/dc/dd/d48/f5f x:0 0 0 2024-03-18T14:51:00.922 INFO:tasks.workunit.client.0.smithi012.stdout:3/595: getdents dc/d30/d3a 0 2024-03-18T14:51:00.923 INFO:tasks.workunit.client.1.smithi100.stdout:7/272: dwrite d5/d34/f54 [0,4194304] 0 2024-03-18T14:51:00.923 INFO:tasks.workunit.client.1.smithi100.stdout:7/273: readlink d5/d14/d16/d2e/l3d 0 2024-03-18T14:51:00.924 INFO:tasks.workunit.client.1.smithi100.stdout:3/285: creat d3/d16/d13/d1f/d3f/d4f/f68 x:0 0 0 2024-03-18T14:51:00.926 INFO:tasks.workunit.client.1.smithi100.stdout:0/339: rmdir dd/d39/d56/d62/d6e 39 2024-03-18T14:51:00.926 INFO:tasks.workunit.client.1.smithi100.stdout:0/340: write dd/f1b [259260,29184] 0 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.0.smithi012.stdout:9/701: unlink d3/d20/dbf/d77/d52/l55 0 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.0.smithi012.stdout:9/702: chown d3/d20/dbf/d77/f56 27163 1 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.0.smithi012.stdout:9/703: dread - d3/d20/dbf/d66/d21/f8e zero size 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.0.smithi012.stdout:9/704: creat d3/d20/dbf/d77/d52/d65/fe5 x:0 0 0 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.0.smithi012.stdout:9/705: write d3/d20/dbf/d66/d69/d7f/fe0 [64601,68258] 0 2024-03-18T14:51:00.929 INFO:tasks.workunit.client.1.smithi100.stdout:9/300: dwrite da/db/f36 [0,4194304] 0 2024-03-18T14:51:00.930 INFO:tasks.workunit.client.1.smithi100.stdout:9/301: stat da/d25 0 2024-03-18T14:51:00.930 INFO:tasks.workunit.client.1.smithi100.stdout:5/244: rmdir d29/d2d/d39 39 2024-03-18T14:51:00.930 INFO:tasks.workunit.client.0.smithi012.stdout:4/548: dwrite d6/da/fd [4194304,4194304] 0 2024-03-18T14:51:00.930 INFO:tasks.workunit.client.0.smithi012.stdout:0/641: rmdir d1/d20/d23/d82 39 2024-03-18T14:51:00.933 INFO:tasks.workunit.client.1.smithi100.stdout:6/259: mkdir d6/d10/d2b/d36/d4f 0 2024-03-18T14:51:00.934 INFO:tasks.workunit.client.0.smithi012.stdout:6/586: creat d17/d38/d51/d63/fb2 x:0 0 0 2024-03-18T14:51:00.936 INFO:tasks.workunit.client.1.smithi100.stdout:8/293: creat d4/f60 x:0 0 0 2024-03-18T14:51:00.936 INFO:tasks.workunit.client.1.smithi100.stdout:5/245: write d29/d2d/d39/f45 [2264208,23131] 0 2024-03-18T14:51:00.936 INFO:tasks.workunit.client.1.smithi100.stdout:5/246: chown d29/d2d 13173570 1 2024-03-18T14:51:00.938 INFO:tasks.workunit.client.0.smithi012.stdout:7/634: rename dd/d13/d23/d2f/d53/l94 to dd/d13/d23/ldb 0 2024-03-18T14:51:00.938 INFO:tasks.workunit.client.0.smithi012.stdout:7/635: creat dd/d13/d23/fdc x:0 0 0 2024-03-18T14:51:00.939 INFO:tasks.workunit.client.0.smithi012.stdout:7/636: dread dd/d13/d23/d76/fb2 [0,4194304] 0 2024-03-18T14:51:00.940 INFO:tasks.workunit.client.1.smithi100.stdout:4/274: sync 2024-03-18T14:51:00.942 INFO:tasks.workunit.client.0.smithi012.stdout:1/585: unlink d8/db/d10/d16/c22 0 2024-03-18T14:51:00.948 INFO:tasks.workunit.client.0.smithi012.stdout:3/596: symlink dc/d3c/d59/d86/d9a/d54/lc1 0 2024-03-18T14:51:00.949 INFO:tasks.workunit.client.1.smithi100.stdout:0/341: dwrite dd/f1f [0,4194304] 0 2024-03-18T14:51:00.952 INFO:tasks.workunit.client.0.smithi012.stdout:5/623: rmdir d3/d29/d36/d9b/d59/d8b 39 2024-03-18T14:51:00.952 INFO:tasks.workunit.client.0.smithi012.stdout:5/624: chown d3/d15/lc3 26 1 2024-03-18T14:51:00.952 INFO:tasks.workunit.client.0.smithi012.stdout:5/625: dread - d3/d15/d42/dc6/d65/da5/fae zero size 2024-03-18T14:51:00.953 INFO:tasks.workunit.client.1.smithi100.stdout:7/274: mkdir d5/d14/d16/d63 0 2024-03-18T14:51:00.956 INFO:tasks.workunit.client.1.smithi100.stdout:3/286: mknod d3/d16/d13/d1f/d3f/d4f/c69 0 2024-03-18T14:51:00.959 INFO:tasks.workunit.client.1.smithi100.stdout:3/287: dread d3/d16/d13/f39 [0,4194304] 0 2024-03-18T14:51:00.962 INFO:tasks.workunit.client.0.smithi012.stdout:4/549: mkdir d6/da/d89/da1/dbb 0 2024-03-18T14:51:00.962 INFO:tasks.workunit.client.0.smithi012.stdout:4/550: truncate d6/da/d58/f9e 800603 0 2024-03-18T14:51:00.963 INFO:tasks.workunit.client.1.smithi100.stdout:9/302: link da/l1b da/l57 0 2024-03-18T14:51:00.966 INFO:tasks.workunit.client.0.smithi012.stdout:1/586: dwrite d8/db/d10/d31/d81/fae [0,4194304] 0 2024-03-18T14:51:00.966 INFO:tasks.workunit.client.1.smithi100.stdout:4/275: dwrite d9/f10 [4194304,4194304] 0 2024-03-18T14:51:00.966 INFO:tasks.workunit.client.1.smithi100.stdout:4/276: creat d9/d27/d42/f50 x:0 0 0 2024-03-18T14:51:00.966 INFO:tasks.workunit.client.1.smithi100.stdout:4/277: read - d9/d27/f4b zero size 2024-03-18T14:51:00.967 INFO:tasks.workunit.client.0.smithi012.stdout:6/587: creat d17/d26/fb3 x:0 0 0 2024-03-18T14:51:00.967 INFO:tasks.workunit.client.0.smithi012.stdout:6/588: dread - d17/d19/d27/d58/f7c zero size 2024-03-18T14:51:00.968 INFO:tasks.workunit.client.1.smithi100.stdout:6/260: rename d6/d7/df/d26/l2e to d6/d7/df/d26/d4e/l50 0 2024-03-18T14:51:00.968 INFO:tasks.workunit.client.1.smithi100.stdout:6/261: fsync d6/d12/f3b 0 2024-03-18T14:51:00.968 INFO:tasks.workunit.client.1.smithi100.stdout:3/288: write d3/d16/fc [362700,42118] 0 2024-03-18T14:51:00.970 INFO:tasks.workunit.client.1.smithi100.stdout:3/289: dread d3/dd/f2a [0,4194304] 0 2024-03-18T14:51:00.970 INFO:tasks.workunit.client.0.smithi012.stdout:2/685: rename d0/da2/d59/f8c to d0/da2/fd6 0 2024-03-18T14:51:00.970 INFO:tasks.workunit.client.0.smithi012.stdout:2/686: dread - d0/d3c/d40/d47/d85/f94 zero size 2024-03-18T14:51:00.970 INFO:tasks.workunit.client.1.smithi100.stdout:2/236: sync 2024-03-18T14:51:00.971 INFO:tasks.workunit.client.1.smithi100.stdout:1/276: sync 2024-03-18T14:51:00.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/294: creat d4/dc/dd/de/d19/d37/f61 x:0 0 0 2024-03-18T14:51:00.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/295: fsync d4/dc/d31/f46 0 2024-03-18T14:51:00.974 INFO:tasks.workunit.client.1.smithi100.stdout:4/278: dread d9/d3a/f3c [0,4194304] 0 2024-03-18T14:51:00.975 INFO:tasks.workunit.client.0.smithi012.stdout:1/587: dread f5 [4194304,4194304] 0 2024-03-18T14:51:00.975 INFO:tasks.workunit.client.0.smithi012.stdout:1/588: truncate d8/d1e/d27/d44/f97 227605 0 2024-03-18T14:51:00.975 INFO:tasks.workunit.client.0.smithi012.stdout:7/637: mkdir dd/d13/d23/d76/d6f/dd9/ddd 0 2024-03-18T14:51:00.977 INFO:tasks.workunit.client.1.smithi100.stdout:5/247: mknod d29/d2d/d32/c49 0 2024-03-18T14:51:00.977 INFO:tasks.workunit.client.1.smithi100.stdout:5/248: write f2 [939805,9330] 0 2024-03-18T14:51:00.978 INFO:tasks.workunit.client.0.smithi012.stdout:5/626: truncate d3/d29/f7a 662004 0 2024-03-18T14:51:00.978 INFO:tasks.workunit.client.0.smithi012.stdout:5/627: truncate d3/d15/d5f/fc0 617861 0 2024-03-18T14:51:00.978 INFO:tasks.workunit.client.1.smithi100.stdout:6/262: dread d6/d12/f3a [0,4194304] 0 2024-03-18T14:51:00.983 INFO:tasks.workunit.client.1.smithi100.stdout:5/249: dread f10 [0,4194304] 0 2024-03-18T14:51:00.984 INFO:tasks.workunit.client.0.smithi012.stdout:4/551: creat d6/d1a/d92/fbc x:0 0 0 2024-03-18T14:51:00.985 INFO:tasks.workunit.client.1.smithi100.stdout:0/342: getdents dd/d39/d56/d62/d6e 0 2024-03-18T14:51:00.985 INFO:tasks.workunit.client.1.smithi100.stdout:0/343: chown dd/d20/d44 37 1 2024-03-18T14:51:00.988 INFO:tasks.workunit.client.1.smithi100.stdout:7/275: mknod d5/d14/c64 0 2024-03-18T14:51:00.990 INFO:tasks.workunit.client.1.smithi100.stdout:1/277: write d8/d1e/d11/f21 [2332060,87057] 0 2024-03-18T14:51:00.990 INFO:tasks.workunit.client.1.smithi100.stdout:1/278: fdatasync d8/d1e/d11/f1b 0 2024-03-18T14:51:00.994 INFO:tasks.workunit.client.0.smithi012.stdout:6/589: mknod d17/d19/d27/cb4 0 2024-03-18T14:51:00.997 INFO:tasks.workunit.client.1.smithi100.stdout:2/237: dwrite d3/d6/f44 [0,4194304] 0 2024-03-18T14:51:00.997 INFO:tasks.workunit.client.1.smithi100.stdout:2/238: creat d3/d6/d29/d36/d49/f5b x:0 0 0 2024-03-18T14:51:00.998 INFO:tasks.workunit.client.1.smithi100.stdout:7/276: dread d5/d30/f3c [0,4194304] 0 2024-03-18T14:51:00.998 INFO:tasks.workunit.client.0.smithi012.stdout:0/642: rename d1/d16/d19/c1e to d1/d20/d23/d3b/dc1/cd5 0 2024-03-18T14:51:00.998 INFO:tasks.workunit.client.0.smithi012.stdout:0/643: chown d1/d20/d23/d26/f90 93971458 1 2024-03-18T14:51:00.998 INFO:tasks.workunit.client.0.smithi012.stdout:0/644: chown d1/d20/d4e/d54/d67/dbc/cd4 107005 1 2024-03-18T14:51:01.000 INFO:tasks.workunit.client.1.smithi100.stdout:3/290: creat d3/dd/f6a x:0 0 0 2024-03-18T14:51:01.003 INFO:tasks.workunit.client.0.smithi012.stdout:2/687: creat d0/d6b/dbf/d57/fd7 x:0 0 0 2024-03-18T14:51:01.007 INFO:tasks.workunit.client.0.smithi012.stdout:1/589: creat d8/db/d10/d31/fbb x:0 0 0 2024-03-18T14:51:01.008 INFO:tasks.workunit.client.1.smithi100.stdout:0/344: dread dd/d20/f5f [0,4194304] 0 2024-03-18T14:51:01.010 INFO:tasks.workunit.client.1.smithi100.stdout:8/296: dwrite d4/dc/dd/de/d21/d23/d47/f5b [0,4194304] 0 2024-03-18T14:51:01.012 INFO:tasks.workunit.client.1.smithi100.stdout:6/263: link d6/d12/l4d d6/d10/d2b/l51 0 2024-03-18T14:51:01.017 INFO:tasks.workunit.client.1.smithi100.stdout:6/264: dread d6/d7/f40 [0,4194304] 0 2024-03-18T14:51:01.017 INFO:tasks.workunit.client.1.smithi100.stdout:6/265: write d6/d7/df/f2a [732804,32834] 0 2024-03-18T14:51:01.017 INFO:tasks.workunit.client.1.smithi100.stdout:6/266: creat d6/d10/f52 x:0 0 0 2024-03-18T14:51:01.021 INFO:tasks.workunit.client.0.smithi012.stdout:7/638: dwrite dd/d13/d23/d2f/d7a/dab/fbe [0,4194304] 0 2024-03-18T14:51:01.021 INFO:tasks.workunit.client.0.smithi012.stdout:7/639: creat dd/d13/d23/d76/d6f/db3/dc7/fde x:0 0 0 2024-03-18T14:51:01.022 INFO:tasks.workunit.client.1.smithi100.stdout:1/279: dwrite d8/f1a [4194304,4194304] 0 2024-03-18T14:51:01.022 INFO:tasks.workunit.client.1.smithi100.stdout:0/345: write fa [1326494,33266] 0 2024-03-18T14:51:01.022 INFO:tasks.workunit.client.1.smithi100.stdout:0/346: chown dd/f2e 655 1 2024-03-18T14:51:01.022 INFO:tasks.workunit.client.1.smithi100.stdout:5/250: dwrite d29/d2d/d32/f35 [4194304,4194304] 0 2024-03-18T14:51:01.023 INFO:tasks.workunit.client.1.smithi100.stdout:7/277: truncate d5/fa 2210713 0 2024-03-18T14:51:01.041 INFO:tasks.workunit.client.0.smithi012.stdout:1/590: dwrite d8/db/d10/f60 [0,4194304] 0 2024-03-18T14:51:01.041 INFO:tasks.workunit.client.0.smithi012.stdout:1/591: truncate d8/db/d10/d31/fba 770715 0 2024-03-18T14:51:01.041 INFO:tasks.workunit.client.0.smithi012.stdout:6/590: symlink d17/d26/d36/d4d/da3/lb5 0 2024-03-18T14:51:01.041 INFO:tasks.workunit.client.1.smithi100.stdout:3/291: link d3/dd/d49/f4a d3/d16/d13/d1f/d3f/d4f/d60/f6b 0 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:3/292: fsync d3/d16/d13/d17/f1a 0 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:4/279: rmdir d9/d13/d25 39 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:4/280: chown d9 9 1 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:2/239: dwrite d3/da/f30 [0,4194304] 0 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:2/240: truncate d3/d13/f38 1019139 0 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:8/297: creat d4/dc/dd/de/d21/d23/f62 x:0 0 0 2024-03-18T14:51:01.042 INFO:tasks.workunit.client.1.smithi100.stdout:8/298: chown d4/dc/dd/de/d19 71 1 2024-03-18T14:51:01.044 INFO:tasks.workunit.client.1.smithi100.stdout:8/299: write d4/f15 [1056839,24290] 0 2024-03-18T14:51:01.044 INFO:tasks.workunit.client.1.smithi100.stdout:8/300: write d4/dc/d31/f3d [700083,92388] 0 2024-03-18T14:51:01.045 INFO:tasks.workunit.client.1.smithi100.stdout:1/280: link d8/d1e/d11/d27/d3e/l4c d8/d2a/l4f 0 2024-03-18T14:51:01.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/645: rename d1/d20/d23/d6f/d93/ccb to d1/d16/d19/dd0/cd6 0 2024-03-18T14:51:01.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/646: chown d1/d20/d23/d26/l94 990443 1 2024-03-18T14:51:01.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/647: readlink d1/d20/d23/l27 0 2024-03-18T14:51:01.048 INFO:tasks.workunit.client.0.smithi012.stdout:0/648: fdatasync d1/d20/d23/d7f/fca 0 2024-03-18T14:51:01.049 INFO:tasks.workunit.client.0.smithi012.stdout:5/628: getdents d3/d29/d36/d9b/d59 0 2024-03-18T14:51:01.052 INFO:tasks.workunit.client.1.smithi100.stdout:4/281: dwrite d9/f1b [0,4194304] 0 2024-03-18T14:51:01.054 INFO:tasks.workunit.client.1.smithi100.stdout:2/241: dwrite d3/f22 [0,4194304] 0 2024-03-18T14:51:01.054 INFO:tasks.workunit.client.1.smithi100.stdout:2/242: truncate d3/da/f33 809299 0 2024-03-18T14:51:01.054 INFO:tasks.workunit.client.1.smithi100.stdout:4/282: dread d9/d13/f1e [0,4194304] 0 2024-03-18T14:51:01.054 INFO:tasks.workunit.client.1.smithi100.stdout:7/278: creat d5/d48/f65 x:0 0 0 2024-03-18T14:51:01.056 INFO:tasks.workunit.client.1.smithi100.stdout:3/293: creat d3/d16/d13/d1f/f6c x:0 0 0 2024-03-18T14:51:01.057 INFO:tasks.workunit.client.1.smithi100.stdout:8/301: dread d4/dc/dd/de/d21/d23/d47/f50 [0,4194304] 0 2024-03-18T14:51:01.057 INFO:tasks.workunit.client.1.smithi100.stdout:8/302: creat d4/dc/dd/de/f63 x:0 0 0 2024-03-18T14:51:01.057 INFO:tasks.workunit.client.1.smithi100.stdout:8/303: chown d4/dc/dd/de/d21/d23/d47 2 1 2024-03-18T14:51:01.127 INFO:tasks.workunit.client.0.smithi012.stdout:7/640: creat dd/d13/d23/d76/d6f/dd9/fdf x:0 0 0 2024-03-18T14:51:01.132 INFO:tasks.workunit.client.1.smithi100.stdout:1/281: mkdir d8/d2a/d37/d50 0 2024-03-18T14:51:01.134 INFO:tasks.workunit.client.0.smithi012.stdout:9/706: rename d3/d20 to d3/de6 0 2024-03-18T14:51:01.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/347: getdents dd/d61 0 2024-03-18T14:51:01.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/348: dread - dd/d20/f2c zero size 2024-03-18T14:51:01.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/349: stat f6 0 2024-03-18T14:51:01.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/350: chown dd/d39/l50 560387 1 2024-03-18T14:51:01.135 INFO:tasks.workunit.client.0.smithi012.stdout:6/591: truncate d17/f75 2125762 0 2024-03-18T14:51:01.136 INFO:tasks.workunit.client.0.smithi012.stdout:6/592: write d17/d19/d27/d58/f9f [392320,74010] 0 2024-03-18T14:51:01.136 INFO:tasks.workunit.client.1.smithi100.stdout:5/251: truncate d29/d2d/d32/f35 3083799 0 2024-03-18T14:51:01.140 INFO:tasks.workunit.client.0.smithi012.stdout:4/552: rename d6/d1a/d64/f9c to d6/da/d58/fbd 0 2024-03-18T14:51:01.140 INFO:tasks.workunit.client.0.smithi012.stdout:4/553: write d6/d27/f56 [1390342,18781] 0 2024-03-18T14:51:01.140 INFO:tasks.workunit.client.0.smithi012.stdout:4/554: fdatasync d6/d19/d6c/f7a 0 2024-03-18T14:51:01.146 INFO:tasks.workunit.client.0.smithi012.stdout:0/649: symlink d1/d20/d23/d26/ld7 0 2024-03-18T14:51:01.147 INFO:tasks.workunit.client.0.smithi012.stdout:0/650: dread d1/d16/d6e/fc7 [0,4194304] 0 2024-03-18T14:51:01.148 INFO:tasks.workunit.client.0.smithi012.stdout:5/629: mkdir d3/d29/d35/dcc 0 2024-03-18T14:51:01.156 INFO:tasks.workunit.client.1.smithi100.stdout:3/294: unlink d3/d16/d13/d17/f3c 0 2024-03-18T14:51:01.165 INFO:tasks.workunit.client.1.smithi100.stdout:8/304: getdents d4/dc/dd/de/d21 0 2024-03-18T14:51:01.245 INFO:tasks.workunit.client.1.smithi100.stdout:0/351: creat dd/d2b/f89 x:0 0 0 2024-03-18T14:51:01.250 INFO:tasks.workunit.client.1.smithi100.stdout:4/283: rmdir d9/d27/d42 39 2024-03-18T14:51:01.252 INFO:tasks.workunit.client.0.smithi012.stdout:1/592: creat d8/db/d10/d31/d43/d5e/fbc x:0 0 0 2024-03-18T14:51:01.252 INFO:tasks.workunit.client.0.smithi012.stdout:1/593: stat d8/db/d10/d31/d43/d5e/c64 0 2024-03-18T14:51:01.255 INFO:tasks.workunit.client.1.smithi100.stdout:3/295: unlink d3/d16/d13/d1f/l25 0 2024-03-18T14:51:01.260 INFO:tasks.workunit.client.0.smithi012.stdout:7/641: mknod dd/d13/d23/d76/d6b/ce0 0 2024-03-18T14:51:01.260 INFO:tasks.workunit.client.0.smithi012.stdout:7/642: fdatasync dd/f10 0 2024-03-18T14:51:01.261 INFO:tasks.workunit.client.1.smithi100.stdout:1/282: rmdir d8/d1e/d11/d27/d3f 39 2024-03-18T14:51:01.261 INFO:tasks.workunit.client.1.smithi100.stdout:1/283: fdatasync d8/d1e/d11/f1b 0 2024-03-18T14:51:01.262 INFO:tasks.workunit.client.1.smithi100.stdout:1/284: readlink d8/d1e/d11/d2d/l34 0 2024-03-18T14:51:01.262 INFO:tasks.workunit.client.1.smithi100.stdout:1/285: chown d8/d1e/f48 68 1 2024-03-18T14:51:01.262 INFO:tasks.workunit.client.1.smithi100.stdout:1/286: fdatasync d8/d1e/f28 0 2024-03-18T14:51:01.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/352: truncate dd/f1c 802756 0 2024-03-18T14:51:01.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/353: dread - dd/d20/f2d zero size 2024-03-18T14:51:01.265 INFO:tasks.workunit.client.1.smithi100.stdout:4/284: mkdir d9/d13/d25/d51 0 2024-03-18T14:51:01.265 INFO:tasks.workunit.client.1.smithi100.stdout:4/285: write d9/d13/f33 [2267782,94704] 0 2024-03-18T14:51:01.266 INFO:tasks.workunit.client.0.smithi012.stdout:4/555: getdents d6/d27 0 2024-03-18T14:51:01.266 INFO:tasks.workunit.client.0.smithi012.stdout:7/643: dread dd/d13/f72 [0,4194304] 0 2024-03-18T14:51:01.268 INFO:tasks.workunit.client.1.smithi100.stdout:3/296: mknod d3/dd/c6d 0 2024-03-18T14:51:01.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/644: dread dd/d13/d23/d60/f95 [0,4194304] 0 2024-03-18T14:51:01.269 INFO:tasks.workunit.client.0.smithi012.stdout:0/651: truncate d1/d20/d7b/fa1 1191212 0 2024-03-18T14:51:01.272 INFO:tasks.workunit.client.1.smithi100.stdout:1/287: unlink d8/d1e/d11/d27/l2e 0 2024-03-18T14:51:01.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/645: dread dd/f10 [0,4194304] 0 2024-03-18T14:51:01.273 INFO:tasks.workunit.client.0.smithi012.stdout:7/646: stat dd/d13/d23/lb4 0 2024-03-18T14:51:01.275 INFO:tasks.workunit.client.0.smithi012.stdout:5/630: creat d3/d15/d42/dc6/d73/d6b/fcd x:0 0 0 2024-03-18T14:51:01.277 INFO:tasks.workunit.client.1.smithi100.stdout:0/354: mknod dd/d39/d56/d59/d67/d79/d7d/c8a 0 2024-03-18T14:51:01.279 INFO:tasks.workunit.client.0.smithi012.stdout:1/594: symlink d8/db/d10/d16/d6b/lbd 0 2024-03-18T14:51:01.286 INFO:tasks.workunit.client.0.smithi012.stdout:4/556: truncate d6/d1a/d30/d46/d6f/d7c/f81 1396359 0 2024-03-18T14:51:01.286 INFO:tasks.workunit.client.0.smithi012.stdout:4/557: stat d6/d19 0 2024-03-18T14:51:01.286 INFO:tasks.workunit.client.1.smithi100.stdout:4/286: creat d9/d13/d39/d4c/f52 x:0 0 0 2024-03-18T14:51:01.286 INFO:tasks.workunit.client.1.smithi100.stdout:4/287: chown d9/d13/d25/d51 687083 1 2024-03-18T14:51:01.287 INFO:tasks.workunit.client.1.smithi100.stdout:5/252: truncate f7 3327567 0 2024-03-18T14:51:01.287 INFO:tasks.workunit.client.1.smithi100.stdout:9/303: sync 2024-03-18T14:51:01.288 INFO:tasks.workunit.client.0.smithi012.stdout:0/652: mknod d1/d16/d6e/dcc/cd8 0 2024-03-18T14:51:01.288 INFO:tasks.workunit.client.0.smithi012.stdout:0/653: truncate d1/d20/d23/d88/fd2 214735 0 2024-03-18T14:51:01.289 INFO:tasks.workunit.client.1.smithi100.stdout:3/297: mknod d3/dd/d49/c6e 0 2024-03-18T14:51:01.290 INFO:tasks.workunit.client.1.smithi100.stdout:0/355: mknod dd/d61/c8b 0 2024-03-18T14:51:01.291 INFO:tasks.workunit.client.1.smithi100.stdout:0/356: dread dd/f3a [0,4194304] 0 2024-03-18T14:51:01.293 INFO:tasks.workunit.client.0.smithi012.stdout:1/595: mkdir d8/db/dbe 0 2024-03-18T14:51:01.293 INFO:tasks.workunit.client.0.smithi012.stdout:1/596: write d8/d1e/d27/fb7 [174818,40560] 0 2024-03-18T14:51:01.293 INFO:tasks.workunit.client.0.smithi012.stdout:1/597: fsync d8/d1e/d27/d44/f97 0 2024-03-18T14:51:01.294 INFO:tasks.workunit.client.1.smithi100.stdout:4/288: creat d9/d13/d25/d41/f53 x:0 0 0 2024-03-18T14:51:01.295 INFO:tasks.workunit.client.0.smithi012.stdout:4/558: mkdir d6/da/d89/d94/dbe 0 2024-03-18T14:51:01.296 INFO:tasks.workunit.client.1.smithi100.stdout:5/253: mkdir d29/d4a 0 2024-03-18T14:51:01.298 INFO:tasks.workunit.client.0.smithi012.stdout:0/654: write d1/d20/d23/d26/f90 [1548924,58538] 0 2024-03-18T14:51:01.298 INFO:tasks.workunit.client.0.smithi012.stdout:0/655: write d1/d20/d4e/d54/d8a/fb9 [377431,94574] 0 2024-03-18T14:51:01.298 INFO:tasks.workunit.client.1.smithi100.stdout:1/288: dwrite d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:51:01.299 INFO:tasks.workunit.client.0.smithi012.stdout:1/598: mkdir d8/dbf 0 2024-03-18T14:51:01.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/559: rmdir d6/d1a/d98 39 2024-03-18T14:51:01.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/560: write d6/d1a/d92/fbc [422148,107936] 0 2024-03-18T14:51:01.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/561: fdatasync d6/d1a/d30/d46/f6b 0 2024-03-18T14:51:01.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/562: getdents d6/d1a/d98/daf/db5 0 2024-03-18T14:51:01.303 INFO:tasks.workunit.client.1.smithi100.stdout:3/298: symlink d3/d16/d13/d1f/d3f/l6f 0 2024-03-18T14:51:01.303 INFO:tasks.workunit.client.1.smithi100.stdout:3/299: write d3/d16/d13/d1f/d37/f54 [455225,88295] 0 2024-03-18T14:51:01.304 INFO:tasks.workunit.client.0.smithi012.stdout:0/656: symlink d1/d20/d23/d82/ld9 0 2024-03-18T14:51:01.305 INFO:tasks.workunit.client.1.smithi100.stdout:0/357: symlink dd/d20/l8c 0 2024-03-18T14:51:01.307 INFO:tasks.workunit.client.0.smithi012.stdout:1/599: mkdir d8/db/d10/d31/d43/d5e/d65/d84/dc0 0 2024-03-18T14:51:01.307 INFO:tasks.workunit.client.0.smithi012.stdout:1/600: write d8/db/d10/d16/da1/f23 [940315,67779] 0 2024-03-18T14:51:01.309 INFO:tasks.workunit.client.1.smithi100.stdout:4/289: mknod d9/d3a/c54 0 2024-03-18T14:51:01.309 INFO:tasks.workunit.client.1.smithi100.stdout:4/290: chown d9/d3a 1 1 2024-03-18T14:51:01.309 INFO:tasks.workunit.client.1.smithi100.stdout:4/291: readlink d9/d17/l4f 0 2024-03-18T14:51:01.309 INFO:tasks.workunit.client.1.smithi100.stdout:4/292: dread - d9/d27/f3d zero size 2024-03-18T14:51:01.312 INFO:tasks.workunit.client.0.smithi012.stdout:4/563: stat d6/d1a/d30/d46/d6f/d8b/c9b 0 2024-03-18T14:51:01.316 INFO:tasks.workunit.client.1.smithi100.stdout:0/358: getdents dd/d39/d56/d62 0 2024-03-18T14:51:01.318 INFO:tasks.workunit.client.0.smithi012.stdout:1/601: mkdir d8/db/d10/d31/d43/d5e/d65/d84/dc1 0 2024-03-18T14:51:01.318 INFO:tasks.workunit.client.1.smithi100.stdout:4/293: dread d9/f2b [0,4194304] 0 2024-03-18T14:51:01.319 INFO:tasks.workunit.client.0.smithi012.stdout:0/657: dread d1/d16/d19/f7d [0,4194304] 0 2024-03-18T14:51:01.321 INFO:tasks.workunit.client.1.smithi100.stdout:3/300: dwrite d3/d16/f62 [0,4194304] 0 2024-03-18T14:51:01.322 INFO:tasks.workunit.client.1.smithi100.stdout:5/254: dwrite d29/d2d/d39/f45 [0,4194304] 0 2024-03-18T14:51:01.326 INFO:tasks.workunit.client.0.smithi012.stdout:1/602: dread d8/d1e/d27/fb7 [0,4194304] 0 2024-03-18T14:51:01.329 INFO:tasks.workunit.client.1.smithi100.stdout:5/255: unlink f1b 0 2024-03-18T14:51:01.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/603: mkdir d8/d1e/d27/d44/dc2 0 2024-03-18T14:51:01.334 INFO:tasks.workunit.client.0.smithi012.stdout:1/604: unlink d8/db/d10/d16/d6b/d7e/c86 0 2024-03-18T14:51:01.335 INFO:tasks.workunit.client.0.smithi012.stdout:1/605: dread - d8/db/d10/d56/f8f zero size 2024-03-18T14:51:01.336 INFO:tasks.workunit.client.1.smithi100.stdout:4/294: dwrite d9/f3b [0,4194304] 0 2024-03-18T14:51:01.337 INFO:tasks.workunit.client.0.smithi012.stdout:1/606: mkdir d8/db/d10/d16/d6b/dc3 0 2024-03-18T14:51:01.339 INFO:tasks.workunit.client.1.smithi100.stdout:5/256: dread f15 [0,4194304] 0 2024-03-18T14:51:01.343 INFO:tasks.workunit.client.1.smithi100.stdout:5/257: mknod d29/d2d/d32/c4b 0 2024-03-18T14:51:01.345 INFO:tasks.workunit.client.0.smithi012.stdout:0/658: dwrite d1/f10 [0,4194304] 0 2024-03-18T14:51:01.347 INFO:tasks.workunit.client.0.smithi012.stdout:1/607: write d8/f40 [2120695,59571] 0 2024-03-18T14:51:01.348 INFO:tasks.workunit.client.1.smithi100.stdout:5/258: dread f23 [0,4194304] 0 2024-03-18T14:51:01.350 INFO:tasks.workunit.client.0.smithi012.stdout:1/608: symlink d8/db/d10/d31/d43/lc4 0 2024-03-18T14:51:01.351 INFO:tasks.workunit.client.0.smithi012.stdout:1/609: dread d8/d1e/d27/d44/f97 [0,4194304] 0 2024-03-18T14:51:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:1/610: mkdir d8/db/d10/d31/d81/dc5 0 2024-03-18T14:51:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:1/611: fsync d8/f1a 0 2024-03-18T14:51:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:1/612: read - d8/db/d10/d16/d6b/d7e/fad zero size 2024-03-18T14:51:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:1/613: fdatasync d8/d1e/d27/f28 0 2024-03-18T14:51:01.354 INFO:tasks.workunit.client.0.smithi012.stdout:1/614: chown d8/db/d10/d31/d45/f6d 18125 1 2024-03-18T14:51:01.355 INFO:tasks.workunit.client.0.smithi012.stdout:0/659: dread d1/d16/d19/f21 [0,4194304] 0 2024-03-18T14:51:01.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/295: dwrite d9/f11 [0,4194304] 0 2024-03-18T14:51:01.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/296: readlink d9/d17/l49 0 2024-03-18T14:51:01.355 INFO:tasks.workunit.client.1.smithi100.stdout:4/297: dread - d9/d13/d25/f4d zero size 2024-03-18T14:51:01.361 INFO:tasks.workunit.client.0.smithi012.stdout:7/647: mkdir dd/d13/d23/d2f/de1 0 2024-03-18T14:51:01.361 INFO:tasks.workunit.client.0.smithi012.stdout:7/648: readlink dd/l14 0 2024-03-18T14:51:01.361 INFO:tasks.workunit.client.1.smithi100.stdout:4/298: symlink d9/d3a/l55 0 2024-03-18T14:51:01.370 INFO:tasks.workunit.client.0.smithi012.stdout:7/649: truncate dd/d13/d23/f99 45893 0 2024-03-18T14:51:01.383 INFO:tasks.workunit.client.1.smithi100.stdout:9/304: unlink da/dd/f54 0 2024-03-18T14:51:01.383 INFO:tasks.workunit.client.1.smithi100.stdout:2/243: rmdir d3/d1a 39 2024-03-18T14:51:01.383 INFO:tasks.workunit.client.1.smithi100.stdout:2/244: chown d3/da/de/d20/f24 522104621 1 2024-03-18T14:51:01.385 INFO:tasks.workunit.client.1.smithi100.stdout:2/245: symlink d3/da/de/d48/l5c 0 2024-03-18T14:51:01.385 INFO:tasks.workunit.client.1.smithi100.stdout:2/246: fsync d3/d6/d29/f35 0 2024-03-18T14:51:01.386 INFO:tasks.workunit.client.1.smithi100.stdout:2/247: creat d3/da/de/d48/f5d x:0 0 0 2024-03-18T14:51:01.386 INFO:tasks.workunit.client.1.smithi100.stdout:2/248: creat d3/d6/d29/d36/f5e x:0 0 0 2024-03-18T14:51:01.388 INFO:tasks.workunit.client.1.smithi100.stdout:2/249: unlink d3/f1b 0 2024-03-18T14:51:01.389 INFO:tasks.workunit.client.1.smithi100.stdout:2/250: rmdir d3/d13 39 2024-03-18T14:51:01.390 INFO:tasks.workunit.client.1.smithi100.stdout:4/299: dwrite d9/d13/d25/f48 [0,4194304] 0 2024-03-18T14:51:01.390 INFO:tasks.workunit.client.1.smithi100.stdout:4/300: stat d9/d13/d39/d4c/f52 0 2024-03-18T14:51:01.390 INFO:tasks.workunit.client.1.smithi100.stdout:4/301: fdatasync d9/d17/f1d 0 2024-03-18T14:51:01.391 INFO:tasks.workunit.client.0.smithi012.stdout:1/615: dwrite d8/db/d10/fb0 [4194304,4194304] 0 2024-03-18T14:51:01.391 INFO:tasks.workunit.client.0.smithi012.stdout:1/616: write d8/db/d10/d16/fab [378612,126913] 0 2024-03-18T14:51:01.391 INFO:tasks.workunit.client.1.smithi100.stdout:2/251: truncate d3/d1a/f2d 2174021 0 2024-03-18T14:51:01.393 INFO:tasks.workunit.client.0.smithi012.stdout:1/617: truncate d8/db/f2d 241707 0 2024-03-18T14:51:01.394 INFO:tasks.workunit.client.0.smithi012.stdout:8/795: sync 2024-03-18T14:51:01.394 INFO:tasks.workunit.client.0.smithi012.stdout:3/597: sync 2024-03-18T14:51:01.395 INFO:tasks.workunit.client.1.smithi100.stdout:2/252: mkdir d3/d6/d5f 0 2024-03-18T14:51:01.396 INFO:tasks.workunit.client.1.smithi100.stdout:2/253: dread - d3/d6/d29/d36/d49/f5b zero size 2024-03-18T14:51:01.397 INFO:tasks.workunit.client.0.smithi012.stdout:8/796: mknod d0/d5a/d5e/dd7/d84/d2a/d3e/cf9 0 2024-03-18T14:51:01.398 INFO:tasks.workunit.client.1.smithi100.stdout:9/305: dread da/db/f13 [0,4194304] 0 2024-03-18T14:51:01.398 INFO:tasks.workunit.client.1.smithi100.stdout:2/254: creat d3/d1a/f60 x:0 0 0 2024-03-18T14:51:01.401 INFO:tasks.workunit.client.1.smithi100.stdout:2/255: mknod d3/d6/d29/d36/d49/c61 0 2024-03-18T14:51:01.401 INFO:tasks.workunit.client.1.smithi100.stdout:9/306: getdents da/d46/d4d 0 2024-03-18T14:51:01.404 INFO:tasks.workunit.client.1.smithi100.stdout:9/307: creat da/dd/d3c/d4b/f58 x:0 0 0 2024-03-18T14:51:01.406 INFO:tasks.workunit.client.1.smithi100.stdout:9/308: link da/dd/f10 da/dd/d3c/f59 0 2024-03-18T14:51:01.409 INFO:tasks.workunit.client.0.smithi012.stdout:8/797: write d0/d5a/d19/d57/f68 [945515,64003] 0 2024-03-18T14:51:01.411 INFO:tasks.workunit.client.0.smithi012.stdout:8/798: creat d0/d15/d4e/ffa x:0 0 0 2024-03-18T14:51:01.413 INFO:tasks.workunit.client.0.smithi012.stdout:8/799: mkdir d0/d5a/d5e/dd7/d84/d54/d9f/dfb 0 2024-03-18T14:51:01.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/618: dwrite d8/db/d10/d31/d45/f77 [0,4194304] 0 2024-03-18T14:51:01.423 INFO:tasks.workunit.client.1.smithi100.stdout:2/256: dwrite d3/da/de/d48/f5d [0,4194304] 0 2024-03-18T14:51:01.426 INFO:tasks.workunit.client.1.smithi100.stdout:2/257: mknod d3/d13/d4f/c62 0 2024-03-18T14:51:01.431 INFO:tasks.workunit.client.0.smithi012.stdout:2/688: rename d0/d10/l3b to d0/ld8 0 2024-03-18T14:51:01.435 INFO:tasks.workunit.client.0.smithi012.stdout:2/689: getdents d0/d6b/dbf/d33 0 2024-03-18T14:51:01.435 INFO:tasks.workunit.client.0.smithi012.stdout:2/690: creat d0/d3c/fd9 x:0 0 0 2024-03-18T14:51:01.437 INFO:tasks.workunit.client.0.smithi012.stdout:2/691: write d0/d6b/dbf/d4f/f6f [192033,71316] 0 2024-03-18T14:51:01.438 INFO:tasks.workunit.client.0.smithi012.stdout:9/707: rename d3/de6/dbf/d66/l4b to d3/de6/dbf/d77/da8/le7 0 2024-03-18T14:51:01.439 INFO:tasks.workunit.client.0.smithi012.stdout:2/692: symlink d0/d6b/dbf/d33/d80/lda 0 2024-03-18T14:51:01.441 INFO:tasks.workunit.client.0.smithi012.stdout:6/593: rename d17/d26/d36/l49 to d17/d33/lb6 0 2024-03-18T14:51:01.442 INFO:tasks.workunit.client.0.smithi012.stdout:9/708: link d3/de6/dbf/d66/d21/d35/d48/l91 d3/de6/dbf/d77/d52/d65/le8 0 2024-03-18T14:51:01.443 INFO:tasks.workunit.client.0.smithi012.stdout:2/693: creat d0/d3c/d40/d47/d85/fdb x:0 0 0 2024-03-18T14:51:01.444 INFO:tasks.workunit.client.1.smithi100.stdout:2/258: dwrite d3/da/de/d48/f5d [0,4194304] 0 2024-03-18T14:51:01.444 INFO:tasks.workunit.client.0.smithi012.stdout:1/619: mknod d8/db/d10/d16/cc6 0 2024-03-18T14:51:01.445 INFO:tasks.workunit.client.0.smithi012.stdout:5/631: rename d3/d84/c96 to d3/d15/d42/dc6/d86/cce 0 2024-03-18T14:51:01.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/594: rmdir d17/d19/d27/d6e 39 2024-03-18T14:51:01.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/595: write d17/d18/f9e [4662680,120171] 0 2024-03-18T14:51:01.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/596: fdatasync f16 0 2024-03-18T14:51:01.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/597: write d17/d26/d36/d4d/d82/f8b [1115924,23687] 0 2024-03-18T14:51:01.449 INFO:tasks.workunit.client.0.smithi012.stdout:9/709: symlink d3/de6/dbf/d77/d52/d78/le9 0 2024-03-18T14:51:01.450 INFO:tasks.workunit.client.0.smithi012.stdout:9/710: write d3/f12 [894796,9828] 0 2024-03-18T14:51:01.450 INFO:tasks.workunit.client.1.smithi100.stdout:2/259: truncate d3/d6/f2f 408140 0 2024-03-18T14:51:01.452 INFO:tasks.workunit.client.0.smithi012.stdout:2/694: creat d0/d6c/fdc x:0 0 0 2024-03-18T14:51:01.455 INFO:tasks.workunit.client.0.smithi012.stdout:1/620: mkdir d8/db/d10/d31/d43/d5e/d65/d70/dc7 0 2024-03-18T14:51:01.458 INFO:tasks.workunit.client.0.smithi012.stdout:7/650: rename dd/d13/d23/d33/l63 to dd/d13/d23/d76/d6f/le2 0 2024-03-18T14:51:01.458 INFO:tasks.workunit.client.0.smithi012.stdout:7/651: write dd/d13/f73 [2438344,110375] 0 2024-03-18T14:51:01.459 INFO:tasks.workunit.client.1.smithi100.stdout:7/279: rename d5/d34 to d5/d14/d16/d2e/d62/d66 0 2024-03-18T14:51:01.459 INFO:tasks.workunit.client.1.smithi100.stdout:7/280: chown l2 4112193 1 2024-03-18T14:51:01.460 INFO:tasks.workunit.client.0.smithi012.stdout:5/632: mknod d3/d15/ccf 0 2024-03-18T14:51:01.460 INFO:tasks.workunit.client.0.smithi012.stdout:5/633: stat d3/d15/l4c 0 2024-03-18T14:51:01.460 INFO:tasks.workunit.client.0.smithi012.stdout:5/634: write d3/d1e/d33/f74 [1183775,92092] 0 2024-03-18T14:51:01.460 INFO:tasks.workunit.client.0.smithi012.stdout:5/635: chown d3/d15/d42/dc6/d73 3 1 2024-03-18T14:51:01.461 INFO:tasks.workunit.client.0.smithi012.stdout:5/636: stat d3/d15/l75 0 2024-03-18T14:51:01.463 INFO:tasks.workunit.client.0.smithi012.stdout:6/598: dwrite d17/d26/f28 [0,4194304] 0 2024-03-18T14:51:01.464 INFO:tasks.workunit.client.1.smithi100.stdout:7/281: unlink d5/d22/d24/c5e 0 2024-03-18T14:51:01.467 INFO:tasks.workunit.client.0.smithi012.stdout:1/621: mknod d8/db/d10/d31/d43/d5e/cc8 0 2024-03-18T14:51:01.471 INFO:tasks.workunit.client.0.smithi012.stdout:3/598: rename dc/d30/d35/ca6 to dc/d3c/d59/d86/d9a/d54/cc2 0 2024-03-18T14:51:01.474 INFO:tasks.workunit.client.0.smithi012.stdout:7/652: creat dd/d13/d23/d33/dcc/fe3 x:0 0 0 2024-03-18T14:51:01.474 INFO:tasks.workunit.client.0.smithi012.stdout:7/653: truncate dd/d13/d23/fc8 158147 0 2024-03-18T14:51:01.479 INFO:tasks.workunit.client.0.smithi012.stdout:1/622: rmdir d8/db/d10/d31/d43/d5e/dac 0 2024-03-18T14:51:01.481 INFO:tasks.workunit.client.0.smithi012.stdout:8/800: rename d0/d5a/d5e/dd7/d84/d54/d9f/dfb to d0/d5a/d5e/ddf/dfc 0 2024-03-18T14:51:01.481 INFO:tasks.workunit.client.0.smithi012.stdout:8/801: chown d0/d47/f4f 455725 1 2024-03-18T14:51:01.481 INFO:tasks.workunit.client.0.smithi012.stdout:8/802: chown d0/d5a/d19/d57/f68 3 1 2024-03-18T14:51:01.482 INFO:tasks.workunit.client.0.smithi012.stdout:3/599: symlink dc/d3c/d4c/d57/lc3 0 2024-03-18T14:51:01.482 INFO:tasks.workunit.client.0.smithi012.stdout:3/600: creat dc/d30/fc4 x:0 0 0 2024-03-18T14:51:01.482 INFO:tasks.workunit.client.0.smithi012.stdout:3/601: write dc/d30/f3f [353095,16132] 0 2024-03-18T14:51:01.482 INFO:tasks.workunit.client.0.smithi012.stdout:3/602: readlink dc/d3c/d59/d86/d9a/d54/lbe 0 2024-03-18T14:51:01.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/654: symlink dd/d13/d23/d2f/d53/dc6/le4 0 2024-03-18T14:51:01.483 INFO:tasks.workunit.client.0.smithi012.stdout:7/655: chown dd/d13/d23/d76/d6f/db3 44435 1 2024-03-18T14:51:01.484 INFO:tasks.workunit.client.0.smithi012.stdout:4/564: symlink d6/d1a/d30/d46/lbf 0 2024-03-18T14:51:01.484 INFO:tasks.workunit.client.0.smithi012.stdout:4/565: write d6/f21 [761198,37763] 0 2024-03-18T14:51:01.486 INFO:tasks.workunit.client.0.smithi012.stdout:2/695: rename d0/d3c/f41 to d0/d6b/dbf/d33/d80/fdd 0 2024-03-18T14:51:01.486 INFO:tasks.workunit.client.0.smithi012.stdout:8/803: truncate d0/d47/d59/d67/fda 64619 0 2024-03-18T14:51:01.488 INFO:tasks.workunit.client.0.smithi012.stdout:5/637: dwrite d3/d29/d36/f6d [0,4194304] 0 2024-03-18T14:51:01.490 INFO:tasks.workunit.client.1.smithi100.stdout:1/289: rename d8/d1e/f48 to d8/d1e/d11/d27/d3f/f51 0 2024-03-18T14:51:01.490 INFO:tasks.workunit.client.0.smithi012.stdout:4/566: mkdir d6/da/dc0 0 2024-03-18T14:51:01.492 INFO:tasks.workunit.client.0.smithi012.stdout:4/567: dread d6/d1a/d30/d46/d6f/f71 [0,4194304] 0 2024-03-18T14:51:01.494 INFO:tasks.workunit.client.0.smithi012.stdout:1/623: rename d8/db/d10/d31/d43/lc4 to d8/db/d10/d31/d43/d5e/d65/d70/lc9 0 2024-03-18T14:51:01.496 INFO:tasks.workunit.client.1.smithi100.stdout:0/359: rename dd/c3c to dd/d31/c8d 0 2024-03-18T14:51:01.496 INFO:tasks.workunit.client.1.smithi100.stdout:7/282: rename d5/d22/d3b to d5/d22/d3b/d47/d67 22 2024-03-18T14:51:01.499 INFO:tasks.workunit.client.0.smithi012.stdout:2/696: symlink d0/d3c/d40/d47/d85/lde 0 2024-03-18T14:51:01.501 INFO:tasks.workunit.client.0.smithi012.stdout:7/656: dread f5 [0,4194304] 0 2024-03-18T14:51:01.504 INFO:tasks.workunit.client.1.smithi100.stdout:0/360: creat dd/d61/f8e x:0 0 0 2024-03-18T14:51:01.505 INFO:tasks.workunit.client.0.smithi012.stdout:1/624: dread d8/db/d10/d16/d54/f89 [0,4194304] 0 2024-03-18T14:51:01.506 INFO:tasks.workunit.client.1.smithi100.stdout:2/260: write d3/d1a/f2d [202741,100675] 0 2024-03-18T14:51:01.509 INFO:tasks.workunit.client.0.smithi012.stdout:3/603: dwrite dc/f22 [0,4194304] 0 2024-03-18T14:51:01.509 INFO:tasks.workunit.client.0.smithi012.stdout:3/604: dread - dc/fa4 zero size 2024-03-18T14:51:01.511 INFO:tasks.workunit.client.1.smithi100.stdout:7/283: rename d5/d14/d16/l53 to d5/d14/d16/l68 0 2024-03-18T14:51:01.513 INFO:tasks.workunit.client.0.smithi012.stdout:5/638: unlink d3/d29/d36/d9b/d59/d8b/fa0 0 2024-03-18T14:51:01.513 INFO:tasks.workunit.client.0.smithi012.stdout:5/639: fdatasync d3/d29/d36/f45 0 2024-03-18T14:51:01.515 INFO:tasks.workunit.client.1.smithi100.stdout:0/361: creat dd/f8f x:0 0 0 2024-03-18T14:51:01.516 INFO:tasks.workunit.client.0.smithi012.stdout:4/568: symlink d6/d19/d6c/lc1 0 2024-03-18T14:51:01.518 INFO:tasks.workunit.client.0.smithi012.stdout:1/625: dread d8/db/f41 [0,4194304] 0 2024-03-18T14:51:01.519 INFO:tasks.workunit.client.1.smithi100.stdout:2/261: mkdir d3/d6/d5f/d63 0 2024-03-18T14:51:01.522 INFO:tasks.workunit.client.0.smithi012.stdout:3/605: dread dc/f44 [0,4194304] 0 2024-03-18T14:51:01.524 INFO:tasks.workunit.client.1.smithi100.stdout:7/284: creat d5/d22/d24/f69 x:0 0 0 2024-03-18T14:51:01.527 INFO:tasks.workunit.client.1.smithi100.stdout:2/262: mknod d3/da/de/c64 0 2024-03-18T14:51:01.529 INFO:tasks.workunit.client.0.smithi012.stdout:5/640: link d3/d29/d36/cbf d3/d15/d42/cd0 0 2024-03-18T14:51:01.530 INFO:tasks.workunit.client.0.smithi012.stdout:4/569: dread d6/da/d89/d95/fa4 [0,4194304] 0 2024-03-18T14:51:01.530 INFO:tasks.workunit.client.0.smithi012.stdout:4/570: truncate d6/d19/d6c/f7a 103906 0 2024-03-18T14:51:01.531 INFO:tasks.workunit.client.0.smithi012.stdout:1/626: link d8/db/l42 d8/db/d10/lca 0 2024-03-18T14:51:01.533 INFO:tasks.workunit.client.0.smithi012.stdout:1/627: read d8/db/d10/d31/f4d [54413,116854] 0 2024-03-18T14:51:01.533 INFO:tasks.workunit.client.0.smithi012.stdout:1/628: truncate d8/d1e/d27/d44/f59 5422278 0 2024-03-18T14:51:01.533 INFO:tasks.workunit.client.0.smithi012.stdout:1/629: stat d8/db/d10/d31/d43/d5e/da5 0 2024-03-18T14:51:01.533 INFO:tasks.workunit.client.1.smithi100.stdout:2/263: rename d3/d6/d29/f47 to d3/d13/d40/d53/f65 0 2024-03-18T14:51:01.536 INFO:tasks.workunit.client.0.smithi012.stdout:3/606: mkdir dc/d30/d3a/da0/dc5 0 2024-03-18T14:51:01.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/290: dwrite d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:51:01.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/291: chown d8/f1a 5315544 1 2024-03-18T14:51:01.537 INFO:tasks.workunit.client.0.smithi012.stdout:5/641: mknod d3/d29/d36/d9b/cd1 0 2024-03-18T14:51:01.538 INFO:tasks.workunit.client.0.smithi012.stdout:8/804: dwrite d0/fdd [0,4194304] 0 2024-03-18T14:51:01.538 INFO:tasks.workunit.client.0.smithi012.stdout:8/805: readlink d0/d5a/d19/d63/lcf 0 2024-03-18T14:51:01.539 INFO:tasks.workunit.client.0.smithi012.stdout:8/806: dread d0/d5a/f9a [0,4194304] 0 2024-03-18T14:51:01.540 INFO:tasks.workunit.client.0.smithi012.stdout:8/807: write d0/f7 [1257982,24579] 0 2024-03-18T14:51:01.540 INFO:tasks.workunit.client.1.smithi100.stdout:0/362: dwrite dd/d61/f8e [0,4194304] 0 2024-03-18T14:51:01.540 INFO:tasks.workunit.client.1.smithi100.stdout:0/363: write dd/d2b/f89 [194290,61623] 0 2024-03-18T14:51:01.540 INFO:tasks.workunit.client.1.smithi100.stdout:0/364: write dd/f3a [1534671,1327] 0 2024-03-18T14:51:01.543 INFO:tasks.workunit.client.0.smithi012.stdout:4/571: unlink d6/da/d58/c8f 0 2024-03-18T14:51:01.545 INFO:tasks.workunit.client.0.smithi012.stdout:3/607: creat dc/d3c/d4c/fc6 x:0 0 0 2024-03-18T14:51:01.545 INFO:tasks.workunit.client.0.smithi012.stdout:3/608: chown dc/d30/d35/c53 3028619 1 2024-03-18T14:51:01.546 INFO:tasks.workunit.client.0.smithi012.stdout:7/657: dwrite dd/f5a [0,4194304] 0 2024-03-18T14:51:01.546 INFO:tasks.workunit.client.0.smithi012.stdout:7/658: write dd/d13/d23/d2f/d53/fa5 [568119,62182] 0 2024-03-18T14:51:01.546 INFO:tasks.workunit.client.0.smithi012.stdout:7/659: write dd/d13/d23/d33/f51 [2790043,81159] 0 2024-03-18T14:51:01.546 INFO:tasks.workunit.client.0.smithi012.stdout:7/660: stat dd/d13/d23/d2f/d7a 0 2024-03-18T14:51:01.548 INFO:tasks.workunit.client.0.smithi012.stdout:5/642: creat d3/d15/d5f/fd2 x:0 0 0 2024-03-18T14:51:01.553 INFO:tasks.workunit.client.0.smithi012.stdout:5/643: dread d3/d1e/d33/f74 [0,4194304] 0 2024-03-18T14:51:01.559 INFO:tasks.workunit.client.1.smithi100.stdout:1/292: rename d8/f1a to d8/f52 0 2024-03-18T14:51:01.564 INFO:tasks.workunit.client.0.smithi012.stdout:8/808: getdents d0/d5a/d5e/dd7/d84/d54/d62 0 2024-03-18T14:51:01.567 INFO:tasks.workunit.client.0.smithi012.stdout:5/644: mkdir d3/d15/d42/dc6/d73/dd3 0 2024-03-18T14:51:01.567 INFO:tasks.workunit.client.0.smithi012.stdout:5/645: write d3/db6/fb7 [1358644,11811] 0 2024-03-18T14:51:01.568 INFO:tasks.workunit.client.1.smithi100.stdout:7/285: dwrite d5/d14/d16/d2e/d62/d66/f54 [0,4194304] 0 2024-03-18T14:51:01.574 INFO:tasks.workunit.client.0.smithi012.stdout:8/809: link d0/d5a/d5e/dd7/d84/d54/d62/dd0/f92 d0/d5a/d19/ffd 0 2024-03-18T14:51:01.574 INFO:tasks.workunit.client.0.smithi012.stdout:8/810: fsync d0/d47/d59/f8e 0 2024-03-18T14:51:01.577 INFO:tasks.workunit.client.0.smithi012.stdout:1/630: dwrite d8/d1e/d27/f48 [0,4194304] 0 2024-03-18T14:51:01.577 INFO:tasks.workunit.client.1.smithi100.stdout:0/365: dwrite dd/d20/f2c [0,4194304] 0 2024-03-18T14:51:01.578 INFO:tasks.workunit.client.1.smithi100.stdout:2/264: dwrite d3/d6/d29/d36/f5e [0,4194304] 0 2024-03-18T14:51:01.579 INFO:tasks.workunit.client.1.smithi100.stdout:2/265: truncate d3/d13/d40/d53/f65 377886 0 2024-03-18T14:51:01.579 INFO:tasks.workunit.client.1.smithi100.stdout:2/266: fdatasync d3/d6/d29/d36/d49/f5b 0 2024-03-18T14:51:01.579 INFO:tasks.workunit.client.1.smithi100.stdout:1/293: rename d8/d2a/d37/c3b to d8/d1e/c53 0 2024-03-18T14:51:01.579 INFO:tasks.workunit.client.0.smithi012.stdout:3/609: dwrite dc/d3c/d59/f5a [0,4194304] 0 2024-03-18T14:51:01.579 INFO:tasks.workunit.client.0.smithi012.stdout:3/610: readlink dc/d3c/l55 0 2024-03-18T14:51:01.580 INFO:tasks.workunit.client.0.smithi012.stdout:3/611: truncate dc/d30/f3f 750318 0 2024-03-18T14:51:01.580 INFO:tasks.workunit.client.0.smithi012.stdout:4/572: dwrite d6/d19/d39/f41 [4194304,4194304] 0 2024-03-18T14:51:01.582 INFO:tasks.workunit.client.0.smithi012.stdout:5/646: rename d3/d29/f95 to d3/d29/d35/fd4 0 2024-03-18T14:51:01.587 INFO:tasks.workunit.client.0.smithi012.stdout:8/811: creat d0/d47/d59/dc9/ffe x:0 0 0 2024-03-18T14:51:01.587 INFO:tasks.workunit.client.0.smithi012.stdout:8/812: read - d0/d5a/d5e/ddf/f97 zero size 2024-03-18T14:51:01.588 INFO:tasks.workunit.client.1.smithi100.stdout:8/305: sync 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:3/301: sync 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:6/267: sync 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:8/306: dread - d4/dc/d1f/f32 zero size 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:3/302: write d3/f29 [685020,88946] 0 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:7/286: dwrite d5/d14/d16/d2e/d62/d66/f54 [4194304,4194304] 0 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:7/287: dread - d5/d48/f65 zero size 2024-03-18T14:51:01.589 INFO:tasks.workunit.client.1.smithi100.stdout:7/288: fsync d5/f35 0 2024-03-18T14:51:01.595 INFO:tasks.workunit.client.1.smithi100.stdout:0/366: read dd/fe [2607071,83366] 0 2024-03-18T14:51:01.597 INFO:tasks.workunit.client.1.smithi100.stdout:1/294: rename d8/d1e/f31 to d8/d30/f54 0 2024-03-18T14:51:01.599 INFO:tasks.workunit.client.1.smithi100.stdout:6/268: getdents d6 0 2024-03-18T14:51:01.601 INFO:tasks.workunit.client.1.smithi100.stdout:8/307: creat d4/dc/d1f/d59/f64 x:0 0 0 2024-03-18T14:51:01.601 INFO:tasks.workunit.client.1.smithi100.stdout:8/308: chown d4/dc/dd/de/d21/d23/d25 373 1 2024-03-18T14:51:01.601 INFO:tasks.workunit.client.1.smithi100.stdout:8/309: creat d4/dc/dd/de/d19/d37/f65 x:0 0 0 2024-03-18T14:51:01.603 INFO:tasks.workunit.client.0.smithi012.stdout:4/573: rename d6/da/f54 to d6/db6/fc2 0 2024-03-18T14:51:01.605 INFO:tasks.workunit.client.1.smithi100.stdout:3/303: creat d3/dd/d49/f70 x:0 0 0 2024-03-18T14:51:01.605 INFO:tasks.workunit.client.1.smithi100.stdout:3/304: write d3/d16/d13/d1f/d3f/d4f/d60/f6b [86007,19540] 0 2024-03-18T14:51:01.605 INFO:tasks.workunit.client.1.smithi100.stdout:3/305: dread - d3/d16/f58 zero size 2024-03-18T14:51:01.605 INFO:tasks.workunit.client.1.smithi100.stdout:3/306: chown d3/d16/d13/d1f/d3f/d4f/f51 4226 1 2024-03-18T14:51:01.605 INFO:tasks.workunit.client.1.smithi100.stdout:3/307: truncate d3/d16/d13/d1f/d3f/d4f/f68 924352 0 2024-03-18T14:51:01.606 INFO:tasks.workunit.client.1.smithi100.stdout:2/267: dwrite d3/f17 [0,4194304] 0 2024-03-18T14:51:01.606 INFO:tasks.workunit.client.0.smithi012.stdout:5/647: symlink d3/d29/d35/d5b/ld5 0 2024-03-18T14:51:01.606 INFO:tasks.workunit.client.0.smithi012.stdout:5/648: chown d3/d15/d42/dc6/f6e 908 1 2024-03-18T14:51:01.607 INFO:tasks.workunit.client.1.smithi100.stdout:7/289: creat d5/d30/d4d/d5a/f6a x:0 0 0 2024-03-18T14:51:01.609 INFO:tasks.workunit.client.0.smithi012.stdout:8/813: creat d0/d5a/d5e/dd7/fff x:0 0 0 2024-03-18T14:51:01.610 INFO:tasks.workunit.client.1.smithi100.stdout:0/367: creat dd/d39/d56/d71/f90 x:0 0 0 2024-03-18T14:51:01.610 INFO:tasks.workunit.client.1.smithi100.stdout:0/368: chown f3 64055651 1 2024-03-18T14:51:01.610 INFO:tasks.workunit.client.1.smithi100.stdout:0/369: chown dd/d39/d56/l58 98845 1 2024-03-18T14:51:01.610 INFO:tasks.workunit.client.1.smithi100.stdout:2/268: dread d3/fc [0,4194304] 0 2024-03-18T14:51:01.610 INFO:tasks.workunit.client.1.smithi100.stdout:2/269: fdatasync d3/d6/d29/f35 0 2024-03-18T14:51:01.612 INFO:tasks.workunit.client.1.smithi100.stdout:1/295: getdents d8/d1e/d11 0 2024-03-18T14:51:01.614 INFO:tasks.workunit.client.1.smithi100.stdout:6/269: unlink d6/d10/l3c 0 2024-03-18T14:51:01.614 INFO:tasks.workunit.client.1.smithi100.stdout:1/296: read d8/f29 [254327,4947] 0 2024-03-18T14:51:01.614 INFO:tasks.workunit.client.1.smithi100.stdout:1/297: fsync d8/d1e/d11/f1b 0 2024-03-18T14:51:01.616 INFO:tasks.workunit.client.0.smithi012.stdout:4/574: rename d6/d1a/f2d to d6/da/d89/da1/dbb/fc3 0 2024-03-18T14:51:01.616 INFO:tasks.workunit.client.0.smithi012.stdout:4/575: readlink d6/d19/d6c/lc1 0 2024-03-18T14:51:01.618 INFO:tasks.workunit.client.0.smithi012.stdout:5/649: symlink d3/ld6 0 2024-03-18T14:51:01.618 INFO:tasks.workunit.client.0.smithi012.stdout:5/650: chown d3/d1e/c9 704363009 1 2024-03-18T14:51:01.618 INFO:tasks.workunit.client.0.smithi012.stdout:5/651: chown d3/d1e/d33/faa 4982 1 2024-03-18T14:51:01.619 INFO:tasks.workunit.client.1.smithi100.stdout:8/310: mkdir d4/dc/d31/d66 0 2024-03-18T14:51:01.620 INFO:tasks.workunit.client.0.smithi012.stdout:8/814: creat d0/d5a/d19/dad/f100 x:0 0 0 2024-03-18T14:51:01.622 INFO:tasks.workunit.client.1.smithi100.stdout:3/308: mkdir d3/d16/d13/d71 0 2024-03-18T14:51:01.622 INFO:tasks.workunit.client.1.smithi100.stdout:3/309: truncate d3/d16/f58 49750 0 2024-03-18T14:51:01.622 INFO:tasks.workunit.client.1.smithi100.stdout:3/310: write d3/d16/f36 [522248,81740] 0 2024-03-18T14:51:01.622 INFO:tasks.workunit.client.1.smithi100.stdout:3/311: creat d3/dd/d49/f72 x:0 0 0 2024-03-18T14:51:01.624 INFO:tasks.workunit.client.1.smithi100.stdout:7/290: symlink d5/d14/l6b 0 2024-03-18T14:51:01.626 INFO:tasks.workunit.client.0.smithi012.stdout:4/576: creat d6/d19/d39/fc4 x:0 0 0 2024-03-18T14:51:01.626 INFO:tasks.workunit.client.1.smithi100.stdout:2/270: rmdir d3/d13/d4f 39 2024-03-18T14:51:01.627 INFO:tasks.workunit.client.1.smithi100.stdout:2/271: truncate d3/d13/d40/d53/f65 1377757 0 2024-03-18T14:51:01.627 INFO:tasks.workunit.client.0.smithi012.stdout:5/652: rmdir d3/d29 39 2024-03-18T14:51:01.628 INFO:tasks.workunit.client.1.smithi100.stdout:1/298: read d8/f52 [4251075,60514] 0 2024-03-18T14:51:01.629 INFO:tasks.workunit.client.1.smithi100.stdout:1/299: stat d8/d30/f40 0 2024-03-18T14:51:01.630 INFO:tasks.workunit.client.1.smithi100.stdout:6/270: truncate d6/d10/f22 2324498 0 2024-03-18T14:51:01.631 INFO:tasks.workunit.client.0.smithi012.stdout:5/653: dread d3/d1e/f1a [0,4194304] 0 2024-03-18T14:51:01.631 INFO:tasks.workunit.client.0.smithi012.stdout:5/654: chown d3/d29/d35 247 1 2024-03-18T14:51:01.631 INFO:tasks.workunit.client.0.smithi012.stdout:5/655: chown d3/f9e 29 1 2024-03-18T14:51:01.633 INFO:tasks.workunit.client.1.smithi100.stdout:7/291: dread d5/d14/d16/d2c/f43 [0,4194304] 0 2024-03-18T14:51:01.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/311: rename d4/dc/dd/de/d19/f40 to d4/dc/dd/de/d21/d23/d47/d54/f67 0 2024-03-18T14:51:01.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/312: fdatasync d4/dc/dd/de/d21/d23/d47/d54/f67 0 2024-03-18T14:51:01.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/313: dread - d4/dc/d1f/d59/f64 zero size 2024-03-18T14:51:01.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/314: write d4/dc/dd/de/d21/d23/f62 [579906,81415] 0 2024-03-18T14:51:01.637 INFO:tasks.workunit.client.1.smithi100.stdout:8/315: chown d4/dc/dd/de/d21/d23/d47/d54/f67 35393320 1 2024-03-18T14:51:01.638 INFO:tasks.workunit.client.1.smithi100.stdout:0/370: dwrite dd/f5d [0,4194304] 0 2024-03-18T14:51:01.639 INFO:tasks.workunit.client.0.smithi012.stdout:4/577: dread d6/d19/f85 [0,4194304] 0 2024-03-18T14:51:01.641 INFO:tasks.workunit.client.1.smithi100.stdout:8/316: dread d4/dc/dd/de/d19/f2f [0,4194304] 0 2024-03-18T14:51:01.642 INFO:tasks.workunit.client.1.smithi100.stdout:1/300: creat d8/d1e/d11/f55 x:0 0 0 2024-03-18T14:51:01.642 INFO:tasks.workunit.client.0.smithi012.stdout:8/815: dwrite d0/d5a/d19/dad/f4b [4194304,4194304] 0 2024-03-18T14:51:01.642 INFO:tasks.workunit.client.0.smithi012.stdout:8/816: chown d0/d15/c22 1021677473 1 2024-03-18T14:51:01.644 INFO:tasks.workunit.client.1.smithi100.stdout:3/312: dread d3/d16/f36 [0,4194304] 0 2024-03-18T14:51:01.644 INFO:tasks.workunit.client.1.smithi100.stdout:2/272: dread d3/da/de/ff [0,4194304] 0 2024-03-18T14:51:01.644 INFO:tasks.workunit.client.1.smithi100.stdout:7/292: symlink d5/d30/d4d/d5a/l6c 0 2024-03-18T14:51:01.652 INFO:tasks.workunit.client.1.smithi100.stdout:0/371: rmdir dd/d39/d7b 39 2024-03-18T14:51:01.653 INFO:tasks.workunit.client.1.smithi100.stdout:5/259: sync 2024-03-18T14:51:01.653 INFO:tasks.workunit.client.1.smithi100.stdout:5/260: write f23 [303926,10564] 0 2024-03-18T14:51:01.653 INFO:tasks.workunit.client.1.smithi100.stdout:5/261: chown f26 944948195 1 2024-03-18T14:51:01.653 INFO:tasks.workunit.client.1.smithi100.stdout:8/317: mknod d4/dc/dd/de/d21/d23/c68 0 2024-03-18T14:51:01.655 INFO:tasks.workunit.client.1.smithi100.stdout:7/293: dread d5/d22/d24/d39/f45 [0,4194304] 0 2024-03-18T14:51:01.655 INFO:tasks.workunit.client.1.smithi100.stdout:7/294: write d5/f10 [2081673,14040] 0 2024-03-18T14:51:01.656 INFO:tasks.workunit.client.1.smithi100.stdout:9/309: sync 2024-03-18T14:51:01.656 INFO:tasks.workunit.client.1.smithi100.stdout:4/302: sync 2024-03-18T14:51:01.656 INFO:tasks.workunit.client.1.smithi100.stdout:5/262: dread f26 [0,4194304] 0 2024-03-18T14:51:01.657 INFO:tasks.workunit.client.1.smithi100.stdout:4/303: write d9/d27/d42/f4a [768676,9502] 0 2024-03-18T14:51:01.658 INFO:tasks.workunit.client.1.smithi100.stdout:1/301: creat d8/dd/f56 x:0 0 0 2024-03-18T14:51:01.661 INFO:tasks.workunit.client.0.smithi012.stdout:5/656: rmdir d3/d84 39 2024-03-18T14:51:01.662 INFO:tasks.workunit.client.1.smithi100.stdout:0/372: dread dd/d39/f66 [0,4194304] 0 2024-03-18T14:51:01.664 INFO:tasks.workunit.client.1.smithi100.stdout:2/273: unlink d3/d6/d29/d36/l3e 0 2024-03-18T14:51:01.664 INFO:tasks.workunit.client.1.smithi100.stdout:2/274: fdatasync d3/d1a/f60 0 2024-03-18T14:51:01.666 INFO:tasks.workunit.client.1.smithi100.stdout:9/310: write da/d2c/d2f/f31 [9926,45698] 0 2024-03-18T14:51:01.667 INFO:tasks.workunit.client.0.smithi012.stdout:4/578: mkdir d6/da/d89/d97/dc5 0 2024-03-18T14:51:01.673 INFO:tasks.workunit.client.0.smithi012.stdout:8/817: link d0/d5a/d5e/dd7/d84/f30 d0/d5a/d19/f101 0 2024-03-18T14:51:01.675 INFO:tasks.workunit.client.0.smithi012.stdout:4/579: creat d6/da/d89/d94/da2/fc6 x:0 0 0 2024-03-18T14:51:01.676 INFO:tasks.workunit.client.0.smithi012.stdout:4/580: chown d6/d1a/d30/d46/d6f/d7c/faa 1759029 1 2024-03-18T14:51:01.676 INFO:tasks.workunit.client.0.smithi012.stdout:4/581: truncate d6/da/f91 393136 0 2024-03-18T14:51:01.676 INFO:tasks.workunit.client.1.smithi100.stdout:0/373: dread dd/f1c [0,4194304] 0 2024-03-18T14:51:01.676 INFO:tasks.workunit.client.1.smithi100.stdout:0/374: truncate dd/d2b/f89 1208850 0 2024-03-18T14:51:01.686 INFO:tasks.workunit.client.0.smithi012.stdout:8/818: dread d0/d5a/d19/fe5 [0,4194304] 0 2024-03-18T14:51:01.688 INFO:tasks.workunit.client.1.smithi100.stdout:8/318: dwrite d4/f15 [0,4194304] 0 2024-03-18T14:51:01.688 INFO:tasks.workunit.client.1.smithi100.stdout:8/319: dread - d4/dc/d1f/f32 zero size 2024-03-18T14:51:01.688 INFO:tasks.workunit.client.1.smithi100.stdout:8/320: creat d4/dc/d31/f69 x:0 0 0 2024-03-18T14:51:01.688 INFO:tasks.workunit.client.1.smithi100.stdout:7/295: truncate d5/d22/d24/d39/f45 3046541 0 2024-03-18T14:51:01.689 INFO:tasks.workunit.client.0.smithi012.stdout:8/819: symlink d0/d5a/d19/dad/l102 0 2024-03-18T14:51:01.690 INFO:tasks.workunit.client.1.smithi100.stdout:5/263: creat d29/d2d/d39/f4c x:0 0 0 2024-03-18T14:51:01.690 INFO:tasks.workunit.client.1.smithi100.stdout:5/264: dread - d29/f37 zero size 2024-03-18T14:51:01.691 INFO:tasks.workunit.client.1.smithi100.stdout:2/275: creat d3/d13/d40/d53/f66 x:0 0 0 2024-03-18T14:51:01.692 INFO:tasks.workunit.client.1.smithi100.stdout:2/276: fdatasync d3/da/f3a 0 2024-03-18T14:51:01.693 INFO:tasks.workunit.client.1.smithi100.stdout:1/302: dwrite f7 [0,4194304] 0 2024-03-18T14:51:01.693 INFO:tasks.workunit.client.1.smithi100.stdout:1/303: chown d8/d1e 71430315 1 2024-03-18T14:51:01.693 INFO:tasks.workunit.client.1.smithi100.stdout:4/304: dwrite d9/fb [0,4194304] 0 2024-03-18T14:51:01.694 INFO:tasks.workunit.client.0.smithi012.stdout:8/820: symlink d0/d5a/d5e/dd7/d84/d54/d9f/l103 0 2024-03-18T14:51:01.694 INFO:tasks.workunit.client.1.smithi100.stdout:0/375: dwrite dd/d39/f3f [4194304,4194304] 0 2024-03-18T14:51:01.694 INFO:tasks.workunit.client.1.smithi100.stdout:0/376: dread - dd/d39/d56/d71/f90 zero size 2024-03-18T14:51:01.698 INFO:tasks.workunit.client.1.smithi100.stdout:0/377: dread dd/f1a [0,4194304] 0 2024-03-18T14:51:01.698 INFO:tasks.workunit.client.0.smithi012.stdout:8/821: truncate d0/d5a/d5e/dd7/d84/d54/ff6 675103 0 2024-03-18T14:51:01.698 INFO:tasks.workunit.client.0.smithi012.stdout:8/822: write d0/d5a/d5e/dd7/d84/d54/fb2 [915791,112375] 0 2024-03-18T14:51:01.699 INFO:tasks.workunit.client.1.smithi100.stdout:8/321: mknod d4/dc/d31/c6a 0 2024-03-18T14:51:01.701 INFO:tasks.workunit.client.0.smithi012.stdout:8/823: creat d0/d5a/d5e/dd7/d84/d54/d62/dd0/f104 x:0 0 0 2024-03-18T14:51:01.701 INFO:tasks.workunit.client.0.smithi012.stdout:8/824: write d0/d5a/d5e/dd7/d84/d2a/fa1 [1474040,128976] 0 2024-03-18T14:51:01.702 INFO:tasks.workunit.client.1.smithi100.stdout:7/296: mknod d5/d22/d3b/c6d 0 2024-03-18T14:51:01.707 INFO:tasks.workunit.client.1.smithi100.stdout:1/304: symlink d8/d1e/l57 0 2024-03-18T14:51:01.710 INFO:tasks.workunit.client.1.smithi100.stdout:4/305: rename d9/cd to d9/c56 0 2024-03-18T14:51:01.710 INFO:tasks.workunit.client.1.smithi100.stdout:4/306: chown d9/d17/l26 59 1 2024-03-18T14:51:01.716 INFO:tasks.workunit.client.1.smithi100.stdout:5/265: dwrite f10 [0,4194304] 0 2024-03-18T14:51:01.716 INFO:tasks.workunit.client.1.smithi100.stdout:8/322: chown d4/l8 454927 1 2024-03-18T14:51:01.717 INFO:tasks.workunit.client.1.smithi100.stdout:1/305: write f7 [640317,23749] 0 2024-03-18T14:51:01.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/297: mkdir d5/d22/d24/d6e 0 2024-03-18T14:51:01.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/298: dread - d5/d30/f5d zero size 2024-03-18T14:51:01.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/299: creat d5/d30/d4d/d5a/f6f x:0 0 0 2024-03-18T14:51:01.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/300: fdatasync d5/d14/f40 0 2024-03-18T14:51:01.719 INFO:tasks.workunit.client.0.smithi012.stdout:8/825: dwrite d0/d5a/d55/fe3 [0,4194304] 0 2024-03-18T14:51:01.719 INFO:tasks.workunit.client.0.smithi012.stdout:8/826: truncate d0/d5a/d19/d57/f66 1542096 0 2024-03-18T14:51:01.719 INFO:tasks.workunit.client.0.smithi012.stdout:8/827: creat d0/d5a/d55/f105 x:0 0 0 2024-03-18T14:51:01.724 INFO:tasks.workunit.client.1.smithi100.stdout:4/307: write f8 [15709,92053] 0 2024-03-18T14:51:01.725 INFO:tasks.workunit.client.1.smithi100.stdout:5/266: dread d29/f2c [0,4194304] 0 2024-03-18T14:51:01.731 INFO:tasks.workunit.client.0.smithi012.stdout:8/828: symlink d0/d47/d59/dc9/dc4/l106 0 2024-03-18T14:51:01.734 INFO:tasks.workunit.client.0.smithi012.stdout:8/829: mknod d0/d5a/d5e/dd7/d84/d54/d9f/c107 0 2024-03-18T14:51:01.734 INFO:tasks.workunit.client.0.smithi012.stdout:8/830: chown d0/d15/l88 6 1 2024-03-18T14:51:01.734 INFO:tasks.workunit.client.0.smithi012.stdout:8/831: chown d0/d5a/d19/l26 6784371 1 2024-03-18T14:51:01.734 INFO:tasks.workunit.client.0.smithi012.stdout:8/832: chown d0/d5a/d5e/dd7 25 1 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/306: dwrite d8/d1e/d11/f21 [0,4194304] 0 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/307: chown d8/d1e/d11/d27/d3f 7130 1 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/308: write d8/d30/f35 [610769,48123] 0 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/309: write d8/d1e/f28 [984104,46579] 0 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/310: truncate d8/d1e/d11/f55 128034 0 2024-03-18T14:51:01.735 INFO:tasks.workunit.client.1.smithi100.stdout:7/301: rename d5/d14/d16/d44/l4b to d5/d14/d16/d44/l70 0 2024-03-18T14:51:01.737 INFO:tasks.workunit.client.1.smithi100.stdout:4/308: symlink d9/d13/d25/l57 0 2024-03-18T14:51:01.739 INFO:tasks.workunit.client.0.smithi012.stdout:8/833: link d0/d5a/d5e/dd7/d84/d2a/d3e/f5f d0/d5a/d5e/dd7/d84/d54/d62/daf/de8/f108 0 2024-03-18T14:51:01.741 INFO:tasks.workunit.client.1.smithi100.stdout:5/267: dwrite f17 [0,4194304] 0 2024-03-18T14:51:01.741 INFO:tasks.workunit.client.1.smithi100.stdout:5/268: write d29/d2d/f41 [5182839,122814] 0 2024-03-18T14:51:01.741 INFO:tasks.workunit.client.1.smithi100.stdout:5/269: chown d29/d2d/d32/l40 101570 1 2024-03-18T14:51:01.741 INFO:tasks.workunit.client.1.smithi100.stdout:5/270: truncate d29/f2b 4200983 0 2024-03-18T14:51:01.742 INFO:tasks.workunit.client.1.smithi100.stdout:0/378: rename dd/d39 to dd/d31/d91 0 2024-03-18T14:51:01.742 INFO:tasks.workunit.client.1.smithi100.stdout:0/379: creat dd/d20/f92 x:0 0 0 2024-03-18T14:51:01.744 INFO:tasks.workunit.client.1.smithi100.stdout:8/323: rmdir d4/d3c 39 2024-03-18T14:51:01.747 INFO:tasks.workunit.client.1.smithi100.stdout:4/309: link d9/l31 d9/d13/d39/l58 0 2024-03-18T14:51:01.749 INFO:tasks.workunit.client.1.smithi100.stdout:5/271: mknod d29/d4a/c4d 0 2024-03-18T14:51:01.749 INFO:tasks.workunit.client.1.smithi100.stdout:5/272: creat d29/d2d/d39/f4e x:0 0 0 2024-03-18T14:51:01.750 INFO:tasks.workunit.client.1.smithi100.stdout:7/302: dwrite d5/d22/f57 [0,4194304] 0 2024-03-18T14:51:01.750 INFO:tasks.workunit.client.1.smithi100.stdout:7/303: write d5/f46 [365304,77629] 0 2024-03-18T14:51:01.751 INFO:tasks.workunit.client.1.smithi100.stdout:0/380: rmdir dd/d31/d91/d56 39 2024-03-18T14:51:01.751 INFO:tasks.workunit.client.1.smithi100.stdout:0/381: truncate dd/f8f 403404 0 2024-03-18T14:51:01.754 INFO:tasks.workunit.client.1.smithi100.stdout:8/324: creat d4/dc/dd/de/d21/d23/d47/d54/d55/f6b x:0 0 0 2024-03-18T14:51:01.755 INFO:tasks.workunit.client.1.smithi100.stdout:4/310: mkdir d9/d13/d39/d59 0 2024-03-18T14:51:01.758 INFO:tasks.workunit.client.1.smithi100.stdout:5/273: rename d29/d2d/d32/c49 to d29/d2d/d32/d38/c4f 0 2024-03-18T14:51:01.759 INFO:tasks.workunit.client.1.smithi100.stdout:0/382: rename dd/d31/d91/d56/d59/d67 to dd/d31/d91/d56/d59/d67/d79/d93 22 2024-03-18T14:51:01.770 INFO:tasks.workunit.client.1.smithi100.stdout:7/304: symlink d5/d14/d16/d63/l71 0 2024-03-18T14:51:01.773 INFO:tasks.workunit.client.1.smithi100.stdout:8/325: mkdir d4/dc/d1f/d59/d6c 0 2024-03-18T14:51:01.776 INFO:tasks.workunit.client.1.smithi100.stdout:5/274: creat d29/f50 x:0 0 0 2024-03-18T14:51:01.779 INFO:tasks.workunit.client.1.smithi100.stdout:0/383: rename cc to dd/d31/d91/d7b/c94 0 2024-03-18T14:51:01.779 INFO:tasks.workunit.client.0.smithi012.stdout:4/582: dwrite d6/f2e [0,4194304] 0 2024-03-18T14:51:01.779 INFO:tasks.workunit.client.0.smithi012.stdout:4/583: stat d6/da/d89/da1/dbb 0 2024-03-18T14:51:01.780 INFO:tasks.workunit.client.1.smithi100.stdout:4/311: dwrite d9/f1a [8388608,4194304] 0 2024-03-18T14:51:01.781 INFO:tasks.workunit.client.1.smithi100.stdout:7/305: getdents d5/d22/d3b 0 2024-03-18T14:51:01.783 INFO:tasks.workunit.client.1.smithi100.stdout:8/326: mknod d4/dc/d1f/d59/c6d 0 2024-03-18T14:51:01.786 INFO:tasks.workunit.client.1.smithi100.stdout:5/275: symlink d29/d2d/d39/l51 0 2024-03-18T14:51:01.786 INFO:tasks.workunit.client.1.smithi100.stdout:5/276: creat d29/d2d/d32/d38/f52 x:0 0 0 2024-03-18T14:51:01.791 INFO:tasks.workunit.client.1.smithi100.stdout:0/384: rename dd/d31/d91/d56/d59/d67 to dd/d2b/d95 0 2024-03-18T14:51:01.791 INFO:tasks.workunit.client.1.smithi100.stdout:0/385: fdatasync dd/f43 0 2024-03-18T14:51:01.795 INFO:tasks.workunit.client.0.smithi012.stdout:4/584: truncate d6/d1a/d30/d46/f48 248795 0 2024-03-18T14:51:01.801 INFO:tasks.workunit.client.1.smithi100.stdout:4/312: mkdir d9/d13/d5a 0 2024-03-18T14:51:01.807 INFO:tasks.workunit.client.1.smithi100.stdout:6/271: sync 2024-03-18T14:51:01.812 INFO:tasks.workunit.client.1.smithi100.stdout:4/313: write d9/d17/f1c [1516099,81876] 0 2024-03-18T14:51:01.813 INFO:tasks.workunit.client.1.smithi100.stdout:6/272: dread d6/d10/d2b/f2d [0,4194304] 0 2024-03-18T14:51:01.823 INFO:tasks.workunit.client.1.smithi100.stdout:7/306: unlink d5/d14/d16/f41 0 2024-03-18T14:51:01.825 INFO:tasks.workunit.client.0.smithi012.stdout:4/585: creat d6/d19/d6c/d96/fc7 x:0 0 0 2024-03-18T14:51:01.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/327: getdents d4/dc/dd/de/d21/d23/d25 0 2024-03-18T14:51:01.833 INFO:tasks.workunit.client.1.smithi100.stdout:0/386: mknod dd/d31/d91/d56/d59/c96 0 2024-03-18T14:51:01.833 INFO:tasks.workunit.client.1.smithi100.stdout:0/387: creat dd/d20/f97 x:0 0 0 2024-03-18T14:51:01.836 INFO:tasks.workunit.client.0.smithi012.stdout:4/586: rmdir d6/d1a/d30/d46/d6f/d7c 39 2024-03-18T14:51:01.836 INFO:tasks.workunit.client.0.smithi012.stdout:4/587: stat d6/d1a/d30/d46/d6f/la6 0 2024-03-18T14:51:01.839 INFO:tasks.workunit.client.1.smithi100.stdout:8/328: dwrite d4/dc/dd/de/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:01.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/588: creat d6/d1a/d30/fc8 x:0 0 0 2024-03-18T14:51:01.849 INFO:tasks.workunit.client.1.smithi100.stdout:6/273: mkdir d6/d7/df/d26/d4e/d53 0 2024-03-18T14:51:01.850 INFO:tasks.workunit.client.0.smithi012.stdout:4/589: write d6/d19/f2f [2140659,112824] 0 2024-03-18T14:51:01.851 INFO:tasks.workunit.client.1.smithi100.stdout:7/307: creat d5/f72 x:0 0 0 2024-03-18T14:51:01.852 INFO:tasks.workunit.client.1.smithi100.stdout:7/308: chown d5/d14/c27 601 1 2024-03-18T14:51:01.852 INFO:tasks.workunit.client.1.smithi100.stdout:7/309: dread - d5/d22/d24/f69 zero size 2024-03-18T14:51:01.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/277: fsync d29/f50 0 2024-03-18T14:51:01.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/278: truncate d29/f50 666995 0 2024-03-18T14:51:01.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/279: creat d29/d2d/d32/d38/f53 x:0 0 0 2024-03-18T14:51:01.858 INFO:tasks.workunit.client.1.smithi100.stdout:0/388: mknod dd/d31/d91/d56/d62/d6b/c98 0 2024-03-18T14:51:01.863 INFO:tasks.workunit.client.0.smithi012.stdout:4/590: read d6/d1a/f29 [7299856,121242] 0 2024-03-18T14:51:01.868 INFO:tasks.workunit.client.1.smithi100.stdout:0/389: dread dd/d20/f2c [0,4194304] 0 2024-03-18T14:51:01.871 INFO:tasks.workunit.client.1.smithi100.stdout:4/314: rmdir d9/d13 39 2024-03-18T14:51:01.871 INFO:tasks.workunit.client.1.smithi100.stdout:4/315: readlink d9/d13/d39/l58 0 2024-03-18T14:51:01.872 INFO:tasks.workunit.client.1.smithi100.stdout:6/274: creat d6/d10/d2b/d36/f54 x:0 0 0 2024-03-18T14:51:01.872 INFO:tasks.workunit.client.1.smithi100.stdout:6/275: write d6/d10/f13 [2536333,96977] 0 2024-03-18T14:51:01.872 INFO:tasks.workunit.client.1.smithi100.stdout:6/276: chown d6/d7/df/d26/d4e 15121 1 2024-03-18T14:51:01.875 INFO:tasks.workunit.client.1.smithi100.stdout:4/316: dread d9/d27/f36 [0,4194304] 0 2024-03-18T14:51:01.876 INFO:tasks.workunit.client.1.smithi100.stdout:4/317: dread - d9/d27/d42/f50 zero size 2024-03-18T14:51:01.876 INFO:tasks.workunit.client.1.smithi100.stdout:6/277: dread d6/d7/df/f2a [0,4194304] 0 2024-03-18T14:51:01.876 INFO:tasks.workunit.client.0.smithi012.stdout:4/591: dread d6/d1a/d30/f47 [0,4194304] 0 2024-03-18T14:51:01.877 INFO:tasks.workunit.client.1.smithi100.stdout:5/280: dwrite d29/f2c [0,4194304] 0 2024-03-18T14:51:01.878 INFO:tasks.workunit.client.1.smithi100.stdout:8/329: rename d4/dc/d1f/l39 to d4/l6e 0 2024-03-18T14:51:01.878 INFO:tasks.workunit.client.1.smithi100.stdout:8/330: chown d4/l8 5749 1 2024-03-18T14:51:01.878 INFO:tasks.workunit.client.1.smithi100.stdout:7/310: dwrite d5/d14/d16/d2c/f51 [0,4194304] 0 2024-03-18T14:51:01.878 INFO:tasks.workunit.client.1.smithi100.stdout:5/281: write d29/f37 [96777,13323] 0 2024-03-18T14:51:01.878 INFO:tasks.workunit.client.1.smithi100.stdout:7/311: creat d5/d48/f73 x:0 0 0 2024-03-18T14:51:01.879 INFO:tasks.workunit.client.0.smithi012.stdout:4/592: mknod d6/da/d89/d94/cc9 0 2024-03-18T14:51:01.879 INFO:tasks.workunit.client.0.smithi012.stdout:4/593: readlink d6/d1a/d30/d46/lbf 0 2024-03-18T14:51:01.881 INFO:tasks.workunit.client.1.smithi100.stdout:0/390: truncate dd/f1f 1202827 0 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:3/313: sync 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:9/311: sync 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:1/311: sync 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:2/277: sync 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:2/278: chown d3/d6/f2f 0 1 2024-03-18T14:51:01.883 INFO:tasks.workunit.client.1.smithi100.stdout:2/279: chown d3/d1a/f2d 138006721 1 2024-03-18T14:51:01.887 INFO:tasks.workunit.client.1.smithi100.stdout:6/278: creat d6/d7/df/f55 x:0 0 0 2024-03-18T14:51:01.887 INFO:tasks.workunit.client.1.smithi100.stdout:6/279: stat f4 0 2024-03-18T14:51:01.887 INFO:tasks.workunit.client.1.smithi100.stdout:6/280: creat d6/d7/df/d26/f56 x:0 0 0 2024-03-18T14:51:01.889 INFO:tasks.workunit.client.0.smithi012.stdout:0/660: sync 2024-03-18T14:51:01.889 INFO:tasks.workunit.client.0.smithi012.stdout:0/661: fdatasync d1/d16/d79/f8f 0 2024-03-18T14:51:01.890 INFO:tasks.workunit.client.1.smithi100.stdout:8/331: dread f1 [0,4194304] 0 2024-03-18T14:51:01.890 INFO:tasks.workunit.client.1.smithi100.stdout:8/332: chown d4/dc/dd/de 1445378 1 2024-03-18T14:51:01.890 INFO:tasks.workunit.client.1.smithi100.stdout:8/333: dread - d4/dc/dd/d48/f5f zero size 2024-03-18T14:51:01.890 INFO:tasks.workunit.client.1.smithi100.stdout:4/318: dwrite d9/f32 [4194304,4194304] 0 2024-03-18T14:51:01.891 INFO:tasks.workunit.client.1.smithi100.stdout:5/282: creat d29/d4a/f54 x:0 0 0 2024-03-18T14:51:01.892 INFO:tasks.workunit.client.0.smithi012.stdout:0/662: truncate d1/d20/f2e 3044354 0 2024-03-18T14:51:01.898 INFO:tasks.workunit.client.1.smithi100.stdout:0/391: mknod dd/d20/d44/c99 0 2024-03-18T14:51:01.899 INFO:tasks.workunit.client.1.smithi100.stdout:0/392: dread dd/f5b [0,4194304] 0 2024-03-18T14:51:01.899 INFO:tasks.workunit.client.1.smithi100.stdout:0/393: chown dd/d61/f72 0 1 2024-03-18T14:51:01.900 INFO:tasks.workunit.client.1.smithi100.stdout:0/394: fdatasync dd/d31/d91/d56/d71/f90 0 2024-03-18T14:51:01.905 INFO:tasks.workunit.client.1.smithi100.stdout:1/312: mknod d8/d1e/c58 0 2024-03-18T14:51:01.907 INFO:tasks.workunit.client.1.smithi100.stdout:3/314: link d3/d16/d13/d1f/d3f/l6f d3/dd/d21/l73 0 2024-03-18T14:51:01.908 INFO:tasks.workunit.client.1.smithi100.stdout:2/280: rmdir d3/d13/d4f 39 2024-03-18T14:51:01.908 INFO:tasks.workunit.client.1.smithi100.stdout:2/281: creat d3/d6/f67 x:0 0 0 2024-03-18T14:51:01.908 INFO:tasks.workunit.client.0.smithi012.stdout:0/663: write d1/d20/d4e/d54/d67/f72 [1325158,60628] 0 2024-03-18T14:51:01.909 INFO:tasks.workunit.client.0.smithi012.stdout:0/664: chown d1/d20/d4e/d54/d67/ca0 272429855 1 2024-03-18T14:51:01.909 INFO:tasks.workunit.client.1.smithi100.stdout:3/315: dread d3/d16/d13/d1f/d3f/d4f/d60/f6b [0,4194304] 0 2024-03-18T14:51:01.910 INFO:tasks.workunit.client.1.smithi100.stdout:8/334: mknod d4/dc/dd/de/c6f 0 2024-03-18T14:51:01.911 INFO:tasks.workunit.client.0.smithi012.stdout:0/665: mknod d1/d20/d23/d6f/cda 0 2024-03-18T14:51:01.912 INFO:tasks.workunit.client.0.smithi012.stdout:0/666: stat d1/d20/d23/d88/fd2 0 2024-03-18T14:51:01.912 INFO:tasks.workunit.client.0.smithi012.stdout:0/667: write d1/d20/d23/d7f/fca [633373,49866] 0 2024-03-18T14:51:01.913 INFO:tasks.workunit.client.1.smithi100.stdout:4/319: rename d9/d13/d25/d41 to d9/d13/d5a/d5b 0 2024-03-18T14:51:01.914 INFO:tasks.workunit.client.1.smithi100.stdout:0/395: truncate dd/d61/f8e 3216617 0 2024-03-18T14:51:01.914 INFO:tasks.workunit.client.1.smithi100.stdout:0/396: fsync dd/d20/f5f 0 2024-03-18T14:51:01.914 INFO:tasks.workunit.client.1.smithi100.stdout:0/397: readlink dd/d2b/l36 0 2024-03-18T14:51:01.914 INFO:tasks.workunit.client.1.smithi100.stdout:0/398: chown f3 166002913 1 2024-03-18T14:51:01.915 INFO:tasks.workunit.client.1.smithi100.stdout:1/313: mkdir d8/d1e/d11/d27/d3c/d59 0 2024-03-18T14:51:01.916 INFO:tasks.workunit.client.1.smithi100.stdout:0/399: dread dd/d20/f21 [0,4194304] 0 2024-03-18T14:51:01.917 INFO:tasks.workunit.client.1.smithi100.stdout:6/281: dwrite d6/d7/df/d26/d32/d48/f4a [0,4194304] 0 2024-03-18T14:51:01.917 INFO:tasks.workunit.client.1.smithi100.stdout:2/282: creat d3/d6/d5f/f68 x:0 0 0 2024-03-18T14:51:01.918 INFO:tasks.workunit.client.1.smithi100.stdout:5/283: dwrite d29/d2d/d39/f4c [0,4194304] 0 2024-03-18T14:51:01.918 INFO:tasks.workunit.client.1.smithi100.stdout:9/312: dwrite da/dd/f10 [4194304,4194304] 0 2024-03-18T14:51:01.919 INFO:tasks.workunit.client.1.smithi100.stdout:0/400: dread dd/f1d [0,4194304] 0 2024-03-18T14:51:01.920 INFO:tasks.workunit.client.1.smithi100.stdout:4/320: rename d9/c28 to d9/d13/d5a/c5c 0 2024-03-18T14:51:01.921 INFO:tasks.workunit.client.1.smithi100.stdout:3/316: dread d3/dd/d21/f22 [0,4194304] 0 2024-03-18T14:51:01.921 INFO:tasks.workunit.client.1.smithi100.stdout:3/317: fsync d3/d16/d13/d1f/d3f/d4f/d60/f6b 0 2024-03-18T14:51:01.921 INFO:tasks.workunit.client.1.smithi100.stdout:3/318: dread - d3/dd/d21/f55 zero size 2024-03-18T14:51:01.921 INFO:tasks.workunit.client.1.smithi100.stdout:3/319: fsync d3/d16/d13/f47 0 2024-03-18T14:51:01.926 INFO:tasks.workunit.client.1.smithi100.stdout:1/314: dread d8/d1e/d11/d2d/f25 [0,4194304] 0 2024-03-18T14:51:01.926 INFO:tasks.workunit.client.1.smithi100.stdout:2/283: mkdir d3/d1a/d69 0 2024-03-18T14:51:01.927 INFO:tasks.workunit.client.1.smithi100.stdout:5/284: mknod d29/d2d/c55 0 2024-03-18T14:51:01.927 INFO:tasks.workunit.client.1.smithi100.stdout:5/285: creat d29/d2d/d39/f56 x:0 0 0 2024-03-18T14:51:01.927 INFO:tasks.workunit.client.1.smithi100.stdout:5/286: readlink d29/d2d/d39/l51 0 2024-03-18T14:51:01.927 INFO:tasks.workunit.client.1.smithi100.stdout:5/287: chown d29/d2d/d32/c3c 75213689 1 2024-03-18T14:51:01.929 INFO:tasks.workunit.client.1.smithi100.stdout:6/282: dread d6/d7/df/d26/d32/d48/f4a [0,4194304] 0 2024-03-18T14:51:01.929 INFO:tasks.workunit.client.1.smithi100.stdout:9/313: truncate da/dd/f27 1389853 0 2024-03-18T14:51:01.929 INFO:tasks.workunit.client.1.smithi100.stdout:9/314: creat da/dd/d29/f5a x:0 0 0 2024-03-18T14:51:01.930 INFO:tasks.workunit.client.1.smithi100.stdout:4/321: creat d9/d13/d25/d51/f5d x:0 0 0 2024-03-18T14:51:01.932 INFO:tasks.workunit.client.1.smithi100.stdout:3/320: mkdir d3/d16/d13/d71/d74 0 2024-03-18T14:51:01.933 INFO:tasks.workunit.client.1.smithi100.stdout:3/321: read - d3/d16/d13/d1f/d3f/d4f/f51 zero size 2024-03-18T14:51:01.936 INFO:tasks.workunit.client.1.smithi100.stdout:1/315: link d8/d1e/d11/d2d/l2c d8/l5a 0 2024-03-18T14:51:01.939 INFO:tasks.workunit.client.1.smithi100.stdout:3/322: dread d3/ff [0,4194304] 0 2024-03-18T14:51:01.943 INFO:tasks.workunit.client.1.smithi100.stdout:5/288: link d29/f2a d29/f57 0 2024-03-18T14:51:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:0/668: dwrite d1/d20/d23/d3b/d33/f8b [0,4194304] 0 2024-03-18T14:51:01.949 INFO:tasks.workunit.client.1.smithi100.stdout:2/284: rmdir d3/da/de/d48 39 2024-03-18T14:51:01.949 INFO:tasks.workunit.client.1.smithi100.stdout:2/285: read f0 [3637342,4415] 0 2024-03-18T14:51:01.951 INFO:tasks.workunit.client.1.smithi100.stdout:6/283: getdents d6/d10 0 2024-03-18T14:51:01.951 INFO:tasks.workunit.client.1.smithi100.stdout:6/284: chown d6/d7/df/c37 215865207 1 2024-03-18T14:51:01.953 INFO:tasks.workunit.client.1.smithi100.stdout:6/285: read d6/d7/df/f1e [7761,95274] 0 2024-03-18T14:51:01.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/286: mkdir d3/d6/d5f/d6a 0 2024-03-18T14:51:01.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/287: write d3/d1a/f2d [426298,84277] 0 2024-03-18T14:51:01.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/288: chown d3/d13/d40/d53/f65 92916241 1 2024-03-18T14:51:01.955 INFO:tasks.workunit.client.0.smithi012.stdout:0/669: dread d1/d20/d23/d3b/d33/d64/f74 [0,4194304] 0 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:0/401: dwrite dd/f5b [0,4194304] 0 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:0/402: chown dd/d2b/d54 213 1 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:0/403: dread - dd/d2b/f40 zero size 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:8/335: dwrite d4/dc/dd/de/d19/f1e [4194304,4194304] 0 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:8/336: chown d4/d3c/l35 16 1 2024-03-18T14:51:01.959 INFO:tasks.workunit.client.1.smithi100.stdout:8/337: write d4/dc/dd/de/d21/d23/d47/d54/d55/f6b [146537,77688] 0 2024-03-18T14:51:01.960 INFO:tasks.workunit.client.1.smithi100.stdout:6/286: dread d6/d7/df/f1e [0,4194304] 0 2024-03-18T14:51:01.964 INFO:tasks.workunit.client.1.smithi100.stdout:0/404: truncate dd/d31/d91/d56/d59/f7c 192795 0 2024-03-18T14:51:01.972 INFO:tasks.workunit.client.1.smithi100.stdout:9/315: truncate da/f4f 7077459 0 2024-03-18T14:51:01.972 INFO:tasks.workunit.client.1.smithi100.stdout:9/316: readlink da/d46/l53 0 2024-03-18T14:51:01.972 INFO:tasks.workunit.client.1.smithi100.stdout:9/317: readlink l8 0 2024-03-18T14:51:01.972 INFO:tasks.workunit.client.1.smithi100.stdout:9/318: creat da/d2c/d2f/f5b x:0 0 0 2024-03-18T14:51:01.972 INFO:tasks.workunit.client.0.smithi012.stdout:0/670: dread d1/d20/d23/d3b/d33/d51/fa8 [0,4194304] 0 2024-03-18T14:51:01.973 INFO:tasks.workunit.client.1.smithi100.stdout:8/338: dread d4/f15 [0,4194304] 0 2024-03-18T14:51:01.975 INFO:tasks.workunit.client.1.smithi100.stdout:4/322: dwrite d9/d27/f2e [0,4194304] 0 2024-03-18T14:51:01.975 INFO:tasks.workunit.client.1.smithi100.stdout:9/319: creat da/d51/d55/f5c x:0 0 0 2024-03-18T14:51:01.975 INFO:tasks.workunit.client.1.smithi100.stdout:9/320: write da/dd/d29/f34 [4845730,73529] 0 2024-03-18T14:51:01.976 INFO:tasks.workunit.client.0.smithi012.stdout:0/671: mknod d1/d20/d4e/cdb 0 2024-03-18T14:51:01.977 INFO:tasks.workunit.client.1.smithi100.stdout:1/316: dwrite d8/d1e/d11/d2d/f18 [0,4194304] 0 2024-03-18T14:51:01.977 INFO:tasks.workunit.client.1.smithi100.stdout:1/317: chown d8/d1e/d11/d27/f4d 15 1 2024-03-18T14:51:01.978 INFO:tasks.workunit.client.1.smithi100.stdout:8/339: mknod d4/dc/dd/de/d21/d23/d25/c70 0 2024-03-18T14:51:01.979 INFO:tasks.workunit.client.0.smithi012.stdout:0/672: getdents d1/d20/d23/d3b 0 2024-03-18T14:51:01.981 INFO:tasks.workunit.client.0.smithi012.stdout:0/673: symlink d1/d20/d23/d3b/ldc 0 2024-03-18T14:51:01.982 INFO:tasks.workunit.client.1.smithi100.stdout:0/405: write dd/f55 [794009,47289] 0 2024-03-18T14:51:01.982 INFO:tasks.workunit.client.1.smithi100.stdout:0/406: write dd/d31/d91/d56/d62/f84 [443814,123416] 0 2024-03-18T14:51:01.984 INFO:tasks.workunit.client.1.smithi100.stdout:3/323: dwrite d3/d16/d13/f47 [0,4194304] 0 2024-03-18T14:51:01.984 INFO:tasks.workunit.client.1.smithi100.stdout:3/324: chown d3/f57 290622 1 2024-03-18T14:51:01.984 INFO:tasks.workunit.client.1.smithi100.stdout:5/289: dwrite f12 [0,4194304] 0 2024-03-18T14:51:01.984 INFO:tasks.workunit.client.1.smithi100.stdout:5/290: write d29/d2d/d39/f56 [262922,104772] 0 2024-03-18T14:51:01.984 INFO:tasks.workunit.client.1.smithi100.stdout:2/289: dwrite d3/d6/d29/f3d [0,4194304] 0 2024-03-18T14:51:01.985 INFO:tasks.workunit.client.1.smithi100.stdout:6/287: dwrite d6/fe [0,4194304] 0 2024-03-18T14:51:01.985 INFO:tasks.workunit.client.1.smithi100.stdout:3/325: dread d3/d16/f36 [0,4194304] 0 2024-03-18T14:51:01.985 INFO:tasks.workunit.client.1.smithi100.stdout:6/288: truncate d6/d7/df/d26/f1c 4504939 0 2024-03-18T14:51:01.986 INFO:tasks.workunit.client.0.smithi012.stdout:0/674: creat d1/d20/d23/d3b/dc1/fdd x:0 0 0 2024-03-18T14:51:01.986 INFO:tasks.workunit.client.0.smithi012.stdout:0/675: chown d1/faa 708813051 1 2024-03-18T14:51:01.989 INFO:tasks.workunit.client.1.smithi100.stdout:4/323: creat d9/d13/d39/d4c/f5e x:0 0 0 2024-03-18T14:51:01.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/340: rename d4/dc/dd/de/d21/d23/d47/d54/f67 to d4/dc/dd/de/d21/d23/d25/f71 0 2024-03-18T14:51:01.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/341: write d4/dc/d1f/d59/f64 [1027049,60717] 0 2024-03-18T14:51:01.992 INFO:tasks.workunit.client.0.smithi012.stdout:0/676: dread d1/d16/d19/f68 [0,4194304] 0 2024-03-18T14:51:01.992 INFO:tasks.workunit.client.0.smithi012.stdout:0/677: read d1/d20/d23/d26/f36 [1536187,892] 0 2024-03-18T14:51:01.993 INFO:tasks.workunit.client.1.smithi100.stdout:0/407: getdents dd/d31/d91/d56/d62/d6b 0 2024-03-18T14:51:01.994 INFO:tasks.workunit.client.0.smithi012.stdout:0/678: mknod d1/d20/d23/d6f/cde 0 2024-03-18T14:51:01.995 INFO:tasks.workunit.client.1.smithi100.stdout:1/318: dwrite d8/d1e/f28 [0,4194304] 0 2024-03-18T14:51:01.996 INFO:tasks.workunit.client.1.smithi100.stdout:5/291: write d29/d2d/d39/f4c [1052498,44097] 0 2024-03-18T14:51:01.996 INFO:tasks.workunit.client.1.smithi100.stdout:5/292: chown f17 44674412 1 2024-03-18T14:51:01.996 INFO:tasks.workunit.client.1.smithi100.stdout:5/293: fdatasync d29/f57 0 2024-03-18T14:51:01.997 INFO:tasks.workunit.client.0.smithi012.stdout:0/679: chown d1/d20/d23/d3b/d33/f8b 316125059 1 2024-03-18T14:51:02.006 INFO:tasks.workunit.client.0.smithi012.stdout:0/680: mknod d1/d20/d4e/d54/d67/dbc/cdf 0 2024-03-18T14:51:02.009 INFO:tasks.workunit.client.1.smithi100.stdout:2/290: mknod d3/d6/d5f/c6b 0 2024-03-18T14:51:02.009 INFO:tasks.workunit.client.1.smithi100.stdout:2/291: fdatasync d3/d6/d5f/f68 0 2024-03-18T14:51:02.010 INFO:tasks.workunit.client.0.smithi012.stdout:0/681: symlink d1/d20/d23/d7f/le0 0 2024-03-18T14:51:02.011 INFO:tasks.workunit.client.1.smithi100.stdout:5/294: dread f10 [0,4194304] 0 2024-03-18T14:51:02.014 INFO:tasks.workunit.client.1.smithi100.stdout:3/326: link d3/dd/f28 d3/dd/d21/d3e/f75 0 2024-03-18T14:51:02.014 INFO:tasks.workunit.client.1.smithi100.stdout:3/327: rename d3/d16/d13 to d3/d16/d13/d17/d76 22 2024-03-18T14:51:02.016 INFO:tasks.workunit.client.0.smithi012.stdout:0/682: creat d1/d20/d23/d6f/d93/dba/fe1 x:0 0 0 2024-03-18T14:51:02.016 INFO:tasks.workunit.client.0.smithi012.stdout:0/683: fsync d1/d20/d4e/d54/d67/fbf 0 2024-03-18T14:51:02.016 INFO:tasks.workunit.client.1.smithi100.stdout:4/324: dwrite f8 [0,4194304] 0 2024-03-18T14:51:02.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/289: link d6/d10/d2b/l51 d6/d7/df/l57 0 2024-03-18T14:51:02.020 INFO:tasks.workunit.client.1.smithi100.stdout:0/408: link dd/d2b/l4f dd/d31/d91/l9a 0 2024-03-18T14:51:02.021 INFO:tasks.workunit.client.1.smithi100.stdout:0/409: truncate dd/d2b/f3d 1474884 0 2024-03-18T14:51:02.021 INFO:tasks.workunit.client.1.smithi100.stdout:0/410: fsync dd/f5b 0 2024-03-18T14:51:02.021 INFO:tasks.workunit.client.0.smithi012.stdout:9/711: sync 2024-03-18T14:51:02.021 INFO:tasks.workunit.client.0.smithi012.stdout:6/599: sync 2024-03-18T14:51:02.022 INFO:tasks.workunit.client.0.smithi012.stdout:6/600: stat d17/d38/fa2 0 2024-03-18T14:51:02.022 INFO:tasks.workunit.client.0.smithi012.stdout:9/712: chown d3/f3b 3 1 2024-03-18T14:51:02.024 INFO:tasks.workunit.client.1.smithi100.stdout:9/321: dread da/dd/f10 [4194304,4194304] 0 2024-03-18T14:51:02.024 INFO:tasks.workunit.client.1.smithi100.stdout:9/322: readlink da/l1e 0 2024-03-18T14:51:02.024 INFO:tasks.workunit.client.1.smithi100.stdout:1/319: mknod d8/c5b 0 2024-03-18T14:51:02.024 INFO:tasks.workunit.client.1.smithi100.stdout:1/320: fdatasync d8/d1e/d11/d2d/f18 0 2024-03-18T14:51:02.025 INFO:tasks.workunit.client.0.smithi012.stdout:9/713: rmdir d3/de6/dbf 39 2024-03-18T14:51:02.026 INFO:tasks.workunit.client.1.smithi100.stdout:7/312: sync 2024-03-18T14:51:02.027 INFO:tasks.workunit.client.1.smithi100.stdout:3/328: symlink d3/d16/d13/d1f/l77 0 2024-03-18T14:51:02.027 INFO:tasks.workunit.client.1.smithi100.stdout:3/329: truncate d3/dd/f6a 286333 0 2024-03-18T14:51:02.027 INFO:tasks.workunit.client.1.smithi100.stdout:3/330: creat d3/d16/d13/d1f/f78 x:0 0 0 2024-03-18T14:51:02.029 INFO:tasks.workunit.client.1.smithi100.stdout:4/325: creat d9/d13/d5a/f5f x:0 0 0 2024-03-18T14:51:02.029 INFO:tasks.workunit.client.1.smithi100.stdout:4/326: chown d9/d13/f1e 0 1 2024-03-18T14:51:02.029 INFO:tasks.workunit.client.1.smithi100.stdout:4/327: creat d9/d13/d25/d51/f60 x:0 0 0 2024-03-18T14:51:02.030 INFO:tasks.workunit.client.1.smithi100.stdout:6/290: creat d6/d10/d2b/d36/d4f/f58 x:0 0 0 2024-03-18T14:51:02.031 INFO:tasks.workunit.client.1.smithi100.stdout:9/323: unlink da/db/c19 0 2024-03-18T14:51:02.031 INFO:tasks.workunit.client.1.smithi100.stdout:9/324: truncate da/d2c/d2f/f30 2309878 0 2024-03-18T14:51:02.031 INFO:tasks.workunit.client.1.smithi100.stdout:9/325: chown f2 23 1 2024-03-18T14:51:02.032 INFO:tasks.workunit.client.0.smithi012.stdout:6/601: read d17/d19/f2b [1328713,79954] 0 2024-03-18T14:51:02.032 INFO:tasks.workunit.client.1.smithi100.stdout:1/321: link d8/dd/c1c d8/dd/c5c 0 2024-03-18T14:51:02.034 INFO:tasks.workunit.client.1.smithi100.stdout:1/322: write d8/d1e/d11/d2d/f25 [4679048,130205] 0 2024-03-18T14:51:02.035 INFO:tasks.workunit.client.1.smithi100.stdout:5/295: getdents d29 0 2024-03-18T14:51:02.039 INFO:tasks.workunit.client.0.smithi012.stdout:6/602: unlink d17/d19/d27/d6e/d84/l9d 0 2024-03-18T14:51:02.039 INFO:tasks.workunit.client.0.smithi012.stdout:6/603: getdents d17/d26/d36/d4d/d82/d97 0 2024-03-18T14:51:02.039 INFO:tasks.workunit.client.0.smithi012.stdout:6/604: chown d17/d33/c42 15 1 2024-03-18T14:51:02.039 INFO:tasks.workunit.client.0.smithi012.stdout:6/605: write d17/d38/fa2 [1000647,34553] 0 2024-03-18T14:51:02.040 INFO:tasks.workunit.client.1.smithi100.stdout:7/313: creat d5/d14/d16/d2c/f74 x:0 0 0 2024-03-18T14:51:02.041 INFO:tasks.workunit.client.1.smithi100.stdout:4/328: dread d9/d17/f23 [0,4194304] 0 2024-03-18T14:51:02.044 INFO:tasks.workunit.client.0.smithi012.stdout:6/606: rename d17/d38/d3f/l70 to d17/d38/d51/d63/lb7 0 2024-03-18T14:51:02.044 INFO:tasks.workunit.client.0.smithi012.stdout:6/607: truncate d17/d38/d51/d66/faf 611640 0 2024-03-18T14:51:02.046 INFO:tasks.workunit.client.1.smithi100.stdout:3/331: link d3/f29 d3/dd/d21/d3e/f79 0 2024-03-18T14:51:02.050 INFO:tasks.workunit.client.1.smithi100.stdout:9/326: write da/fc [8056295,63011] 0 2024-03-18T14:51:02.051 INFO:tasks.workunit.client.1.smithi100.stdout:6/291: link d6/d7/df/d26/d32/d48/f4a d6/d7/f59 0 2024-03-18T14:51:02.054 INFO:tasks.workunit.client.1.smithi100.stdout:1/323: truncate f5 1273909 0 2024-03-18T14:51:02.054 INFO:tasks.workunit.client.1.smithi100.stdout:1/324: chown d8/d1e/d11/d27/d3c/d59 6 1 2024-03-18T14:51:02.055 INFO:tasks.workunit.client.1.smithi100.stdout:1/325: dread d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:02.056 INFO:tasks.workunit.client.1.smithi100.stdout:7/314: dread d5/d14/d16/f1d [0,4194304] 0 2024-03-18T14:51:02.056 INFO:tasks.workunit.client.1.smithi100.stdout:7/315: creat d5/d14/d16/d21/f75 x:0 0 0 2024-03-18T14:51:02.057 INFO:tasks.workunit.client.1.smithi100.stdout:4/329: rename d9/d17/f1c to d9/d13/d25/d51/f61 0 2024-03-18T14:51:02.057 INFO:tasks.workunit.client.1.smithi100.stdout:3/332: rmdir d3/d16/d13/d1f/d37 39 2024-03-18T14:51:02.058 INFO:tasks.workunit.client.1.smithi100.stdout:3/333: dread - d3/d16/d13/d1f/f59 zero size 2024-03-18T14:51:02.060 INFO:tasks.workunit.client.1.smithi100.stdout:6/292: link d6/d10/d27/c25 d6/d10/d2b/d36/d4f/c5a 0 2024-03-18T14:51:02.060 INFO:tasks.workunit.client.1.smithi100.stdout:6/293: readlink d6/d7/df/d26/l4b 0 2024-03-18T14:51:02.061 INFO:tasks.workunit.client.1.smithi100.stdout:5/296: getdents d29/d4a 0 2024-03-18T14:51:02.061 INFO:tasks.workunit.client.1.smithi100.stdout:2/292: dwrite d3/d6/f44 [0,4194304] 0 2024-03-18T14:51:02.061 INFO:tasks.workunit.client.1.smithi100.stdout:8/342: dwrite d4/f6 [0,4194304] 0 2024-03-18T14:51:02.062 INFO:tasks.workunit.client.0.smithi012.stdout:0/684: dwrite d1/f10 [0,4194304] 0 2024-03-18T14:51:02.062 INFO:tasks.workunit.client.0.smithi012.stdout:0/685: dread - d1/d20/d23/d3b/d33/fbd zero size 2024-03-18T14:51:02.062 INFO:tasks.workunit.client.1.smithi100.stdout:0/411: dwrite dd/f1d [0,4194304] 0 2024-03-18T14:51:02.062 INFO:tasks.workunit.client.0.smithi012.stdout:9/714: dwrite d3/de6/dbf/f6a [0,4194304] 0 2024-03-18T14:51:02.064 INFO:tasks.workunit.client.1.smithi100.stdout:7/316: link d5/d30/d4d/d5a/f6f d5/d14/d16/d44/f76 0 2024-03-18T14:51:02.064 INFO:tasks.workunit.client.1.smithi100.stdout:7/317: chown d5/d30/d4d 35 1 2024-03-18T14:51:02.066 INFO:tasks.workunit.client.1.smithi100.stdout:4/330: mkdir d9/d27/d62 0 2024-03-18T14:51:02.066 INFO:tasks.workunit.client.1.smithi100.stdout:1/326: dread d8/d1e/d11/d27/f4d [0,4194304] 0 2024-03-18T14:51:02.068 INFO:tasks.workunit.client.0.smithi012.stdout:0/686: dread d1/d20/d4e/d54/f57 [4194304,4194304] 0 2024-03-18T14:51:02.068 INFO:tasks.workunit.client.1.smithi100.stdout:3/334: creat d3/d16/d13/d17/f7a x:0 0 0 2024-03-18T14:51:02.068 INFO:tasks.workunit.client.1.smithi100.stdout:3/335: write d3/d16/d13/d1f/f6c [1022821,23810] 0 2024-03-18T14:51:02.068 INFO:tasks.workunit.client.1.smithi100.stdout:3/336: chown d3/d16/d13/f47 2335 1 2024-03-18T14:51:02.069 INFO:tasks.workunit.client.0.smithi012.stdout:9/715: creat d3/fea x:0 0 0 2024-03-18T14:51:02.071 INFO:tasks.workunit.client.1.smithi100.stdout:9/327: getdents da/dd/d29 0 2024-03-18T14:51:02.071 INFO:tasks.workunit.client.1.smithi100.stdout:9/328: stat da/d51/f52 0 2024-03-18T14:51:02.071 INFO:tasks.workunit.client.1.smithi100.stdout:9/329: fsync f9 0 2024-03-18T14:51:02.073 INFO:tasks.workunit.client.1.smithi100.stdout:6/294: creat d6/d10/f5b x:0 0 0 2024-03-18T14:51:02.073 INFO:tasks.workunit.client.1.smithi100.stdout:6/295: write d6/d12/f1f [3374722,28434] 0 2024-03-18T14:51:02.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/297: rename d29/d2d/d32/d38/c4f to d29/d2d/d39/c58 0 2024-03-18T14:51:02.075 INFO:tasks.workunit.client.1.smithi100.stdout:9/330: rename da/d2c to da/d2c/d5d 22 2024-03-18T14:51:02.076 INFO:tasks.workunit.client.1.smithi100.stdout:9/331: read da/f1d [264792,87332] 0 2024-03-18T14:51:02.077 INFO:tasks.workunit.client.1.smithi100.stdout:1/327: dwrite d8/d1e/d11/f55 [0,4194304] 0 2024-03-18T14:51:02.078 INFO:tasks.workunit.client.0.smithi012.stdout:0/687: rmdir d1/d2d 39 2024-03-18T14:51:02.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/293: truncate d3/d6/d29/f35 4150939 0 2024-03-18T14:51:02.080 INFO:tasks.workunit.client.0.smithi012.stdout:9/716: dread - d3/de6/dbf/d77/da8/fb4 zero size 2024-03-18T14:51:02.083 INFO:tasks.workunit.client.0.smithi012.stdout:0/688: link d1/d16/d19/cb4 d1/d16/d6e/ce2 0 2024-03-18T14:51:02.085 INFO:tasks.workunit.client.0.smithi012.stdout:9/717: truncate d3/d9/f29 4526276 0 2024-03-18T14:51:02.090 INFO:tasks.workunit.client.1.smithi100.stdout:6/296: dwrite d6/f2f [0,4194304] 0 2024-03-18T14:51:02.090 INFO:tasks.workunit.client.0.smithi012.stdout:6/608: rename d17/d38/d3f to d17/d18/db8 0 2024-03-18T14:51:02.092 INFO:tasks.workunit.client.1.smithi100.stdout:0/412: getdents dd/d2b/d95/d79 0 2024-03-18T14:51:02.094 INFO:tasks.workunit.client.1.smithi100.stdout:7/318: creat d5/d14/f77 x:0 0 0 2024-03-18T14:51:02.094 INFO:tasks.workunit.client.1.smithi100.stdout:7/319: fsync d5/d22/f57 0 2024-03-18T14:51:02.094 INFO:tasks.workunit.client.1.smithi100.stdout:7/320: creat d5/d30/d4d/d5a/f78 x:0 0 0 2024-03-18T14:51:02.094 INFO:tasks.workunit.client.1.smithi100.stdout:7/321: chown d5/d30/f3c 391 1 2024-03-18T14:51:02.094 INFO:tasks.workunit.client.1.smithi100.stdout:7/322: chown d5/d14/d16/c19 971 1 2024-03-18T14:51:02.097 INFO:tasks.workunit.client.0.smithi012.stdout:0/689: rename d1/d16/d19/f68 to d1/d20/d7b/fe3 0 2024-03-18T14:51:02.099 INFO:tasks.workunit.client.1.smithi100.stdout:4/331: mknod d9/c63 0 2024-03-18T14:51:02.101 INFO:tasks.workunit.client.1.smithi100.stdout:3/337: symlink d3/d16/d13/d1f/d3f/d4f/d65/l7b 0 2024-03-18T14:51:02.103 INFO:tasks.workunit.client.0.smithi012.stdout:0/690: symlink d1/d20/d4e/le4 0 2024-03-18T14:51:02.103 INFO:tasks.workunit.client.0.smithi012.stdout:0/691: fdatasync d1/d20/d4e/d54/d8a/fb9 0 2024-03-18T14:51:02.103 INFO:tasks.workunit.client.0.smithi012.stdout:6/609: dread d17/d19/f6d [4194304,4194304] 0 2024-03-18T14:51:02.105 INFO:tasks.workunit.client.1.smithi100.stdout:5/298: chown l3 19 1 2024-03-18T14:51:02.107 INFO:tasks.workunit.client.1.smithi100.stdout:9/332: unlink da/d2c/d2f/c43 0 2024-03-18T14:51:02.107 INFO:tasks.workunit.client.1.smithi100.stdout:9/333: readlink l8 0 2024-03-18T14:51:02.108 INFO:tasks.workunit.client.0.smithi012.stdout:0/692: getdents d1/d20 0 2024-03-18T14:51:02.108 INFO:tasks.workunit.client.1.smithi100.stdout:6/297: dread f4 [0,4194304] 0 2024-03-18T14:51:02.114 INFO:tasks.workunit.client.0.smithi012.stdout:6/610: mkdir d17/d38/d51/d66/db9 0 2024-03-18T14:51:02.117 INFO:tasks.workunit.client.1.smithi100.stdout:9/334: write da/dd/f27 [946714,11193] 0 2024-03-18T14:51:02.117 INFO:tasks.workunit.client.1.smithi100.stdout:9/335: chown da/dd/c39 6 1 2024-03-18T14:51:02.117 INFO:tasks.workunit.client.1.smithi100.stdout:9/336: chown f2 9930 1 2024-03-18T14:51:02.119 INFO:tasks.workunit.client.1.smithi100.stdout:1/328: creat d8/d2a/d37/f5d x:0 0 0 2024-03-18T14:51:02.121 INFO:tasks.workunit.client.1.smithi100.stdout:6/298: write d6/fe [1979587,37674] 0 2024-03-18T14:51:02.123 INFO:tasks.workunit.client.0.smithi012.stdout:0/693: unlink d1/fa 0 2024-03-18T14:51:02.124 INFO:tasks.workunit.client.0.smithi012.stdout:0/694: read - d1/d20/d4e/d54/fc6 zero size 2024-03-18T14:51:02.125 INFO:tasks.workunit.client.1.smithi100.stdout:2/294: mkdir d3/d1a/d6c 0 2024-03-18T14:51:02.127 INFO:tasks.workunit.client.1.smithi100.stdout:8/343: getdents d4/dc/dd/de/d19/d37 0 2024-03-18T14:51:02.128 INFO:tasks.workunit.client.1.smithi100.stdout:7/323: dwrite d5/f52 [4194304,4194304] 0 2024-03-18T14:51:02.129 INFO:tasks.workunit.client.0.smithi012.stdout:0/695: symlink d1/d16/d19/le5 0 2024-03-18T14:51:02.134 INFO:tasks.workunit.client.0.smithi012.stdout:0/696: creat d1/d20/d4e/d54/fe6 x:0 0 0 2024-03-18T14:51:02.134 INFO:tasks.workunit.client.0.smithi012.stdout:0/697: write d1/d16/d31/f35 [4291105,21396] 0 2024-03-18T14:51:02.136 INFO:tasks.workunit.client.1.smithi100.stdout:5/299: rmdir d29/d4a 39 2024-03-18T14:51:02.136 INFO:tasks.workunit.client.1.smithi100.stdout:5/300: fsync f23 0 2024-03-18T14:51:02.136 INFO:tasks.workunit.client.1.smithi100.stdout:5/301: chown l1c 3527 1 2024-03-18T14:51:02.141 INFO:tasks.workunit.client.1.smithi100.stdout:0/413: dwrite dd/d2b/f40 [0,4194304] 0 2024-03-18T14:51:02.141 INFO:tasks.workunit.client.1.smithi100.stdout:0/414: chown dd/d2b/d54/c87 1 1 2024-03-18T14:51:02.141 INFO:tasks.workunit.client.1.smithi100.stdout:0/415: chown dd/d20/c51 0 1 2024-03-18T14:51:02.142 INFO:tasks.workunit.client.0.smithi012.stdout:0/698: unlink d1/d20/d23/d6f/fa5 0 2024-03-18T14:51:02.142 INFO:tasks.workunit.client.0.smithi012.stdout:0/699: write d1/d16/d19/f7d [652948,105221] 0 2024-03-18T14:51:02.142 INFO:tasks.workunit.client.1.smithi100.stdout:2/295: dread d3/f22 [0,4194304] 0 2024-03-18T14:51:02.143 INFO:tasks.workunit.client.1.smithi100.stdout:2/296: dread - d3/d6/d29/d36/d49/f5b zero size 2024-03-18T14:51:02.143 INFO:tasks.workunit.client.0.smithi012.stdout:0/700: dread d1/d16/d19/d8d/fad [0,4194304] 0 2024-03-18T14:51:02.145 INFO:tasks.workunit.client.1.smithi100.stdout:1/329: unlink d8/dd/c22 0 2024-03-18T14:51:02.145 INFO:tasks.workunit.client.0.smithi012.stdout:6/611: dwrite d17/d26/f79 [0,4194304] 0 2024-03-18T14:51:02.146 INFO:tasks.workunit.client.1.smithi100.stdout:4/332: dwrite d9/f10 [4194304,4194304] 0 2024-03-18T14:51:02.146 INFO:tasks.workunit.client.1.smithi100.stdout:4/333: creat d9/d3a/f64 x:0 0 0 2024-03-18T14:51:02.147 INFO:tasks.workunit.client.1.smithi100.stdout:3/338: dwrite d3/f10 [4194304,4194304] 0 2024-03-18T14:51:02.147 INFO:tasks.workunit.client.1.smithi100.stdout:3/339: chown d3/dd/d49 4297 1 2024-03-18T14:51:02.147 INFO:tasks.workunit.client.1.smithi100.stdout:8/344: creat d4/dc/dd/de/d21/f72 x:0 0 0 2024-03-18T14:51:02.149 INFO:tasks.workunit.client.1.smithi100.stdout:7/324: rename d5/d22/d24/f69 to d5/d22/d24/f79 0 2024-03-18T14:51:02.152 INFO:tasks.workunit.client.1.smithi100.stdout:5/302: symlink d29/d2d/d32/d38/l59 0 2024-03-18T14:51:02.157 INFO:tasks.workunit.client.1.smithi100.stdout:9/337: link da/db/l14 da/dd/d3b/l5e 0 2024-03-18T14:51:02.158 INFO:tasks.workunit.client.1.smithi100.stdout:3/340: dread d3/dd/d49/f4a [0,4194304] 0 2024-03-18T14:51:02.159 INFO:tasks.workunit.client.1.smithi100.stdout:6/299: dread d6/d12/f1f [0,4194304] 0 2024-03-18T14:51:02.160 INFO:tasks.workunit.client.1.smithi100.stdout:1/330: dread d8/f52 [0,4194304] 0 2024-03-18T14:51:02.162 INFO:tasks.workunit.client.0.smithi012.stdout:0/701: dread d1/d16/d31/f1d [0,4194304] 0 2024-03-18T14:51:02.165 INFO:tasks.workunit.client.0.smithi012.stdout:6/612: creat d17/d26/d36/d4d/fba x:0 0 0 2024-03-18T14:51:02.167 INFO:tasks.workunit.client.1.smithi100.stdout:0/416: dwrite dd/d20/f5c [0,4194304] 0 2024-03-18T14:51:02.169 INFO:tasks.workunit.client.1.smithi100.stdout:6/300: fdatasync d6/fe 0 2024-03-18T14:51:02.169 INFO:tasks.workunit.client.1.smithi100.stdout:2/297: dwrite d3/da/f30 [0,4194304] 0 2024-03-18T14:51:02.170 INFO:tasks.workunit.client.1.smithi100.stdout:2/298: creat d3/d6/d5f/f6d x:0 0 0 2024-03-18T14:51:02.170 INFO:tasks.workunit.client.0.smithi012.stdout:0/702: rename d1/d20/d23/d6f/cda to d1/d20/d4e/d54/d8a/ce7 0 2024-03-18T14:51:02.170 INFO:tasks.workunit.client.0.smithi012.stdout:0/703: write d1/d20/d23/d3b/d33/fbd [928220,68715] 0 2024-03-18T14:51:02.172 INFO:tasks.workunit.client.1.smithi100.stdout:4/334: unlink d9/c2a 0 2024-03-18T14:51:02.172 INFO:tasks.workunit.client.0.smithi012.stdout:6/613: mknod d17/d26/d36/d4d/d82/cbb 0 2024-03-18T14:51:02.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/345: symlink d4/dc/dd/de/d19/d37/l73 0 2024-03-18T14:51:02.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/346: chown d4/dc/d1f/f5d 55197917 1 2024-03-18T14:51:02.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/347: chown d4/dc/dd/de/d19/f2f 167 1 2024-03-18T14:51:02.178 INFO:tasks.workunit.client.1.smithi100.stdout:8/348: chown d4/l8 58748 1 2024-03-18T14:51:02.181 INFO:tasks.workunit.client.1.smithi100.stdout:9/338: dwrite da/d2c/d2f/f4a [0,4194304] 0 2024-03-18T14:51:02.183 INFO:tasks.workunit.client.1.smithi100.stdout:7/325: mkdir d5/d22/d24/d6e/d7a 0 2024-03-18T14:51:02.184 INFO:tasks.workunit.client.0.smithi012.stdout:6/614: truncate d17/d33/fa5 544731 0 2024-03-18T14:51:02.185 INFO:tasks.workunit.client.0.smithi012.stdout:2/697: sync 2024-03-18T14:51:02.185 INFO:tasks.workunit.client.0.smithi012.stdout:2/698: write d0/d3c/d40/d47/d85/f94 [948155,102963] 0 2024-03-18T14:51:02.185 INFO:tasks.workunit.client.1.smithi100.stdout:5/303: truncate d29/f34 359905 0 2024-03-18T14:51:02.187 INFO:tasks.workunit.client.1.smithi100.stdout:3/341: unlink d3/d16/d13/d17/f1a 0 2024-03-18T14:51:02.187 INFO:tasks.workunit.client.1.smithi100.stdout:0/417: dwrite dd/f11 [0,4194304] 0 2024-03-18T14:51:02.188 INFO:tasks.workunit.client.0.smithi012.stdout:6/615: dread d17/d26/d36/f4b [0,4194304] 0 2024-03-18T14:51:02.189 INFO:tasks.workunit.client.0.smithi012.stdout:2/699: symlink d0/d6b/dbf/d33/dac/dd0/ldf 0 2024-03-18T14:51:02.190 INFO:tasks.workunit.client.1.smithi100.stdout:4/335: dwrite f0 [4194304,4194304] 0 2024-03-18T14:51:02.190 INFO:tasks.workunit.client.1.smithi100.stdout:4/336: read d9/fe [763160,4285] 0 2024-03-18T14:51:02.191 INFO:tasks.workunit.client.1.smithi100.stdout:1/331: link d8/d1e/d11/d2d/f25 d8/d2a/d37/d50/f5e 0 2024-03-18T14:51:02.191 INFO:tasks.workunit.client.1.smithi100.stdout:1/332: dread - d8/d1e/d11/d27/d3f/f49 zero size 2024-03-18T14:51:02.192 INFO:tasks.workunit.client.0.smithi012.stdout:6/616: truncate d17/d19/f6d 5546427 0 2024-03-18T14:51:02.196 INFO:tasks.workunit.client.0.smithi012.stdout:2/700: dread d0/d3c/f46 [0,4194304] 0 2024-03-18T14:51:02.196 INFO:tasks.workunit.client.0.smithi012.stdout:2/701: chown d0/d6b/dbf/d33 122 1 2024-03-18T14:51:02.196 INFO:tasks.workunit.client.0.smithi012.stdout:6/617: rename d17/d19/d27/d2f/f43 to d17/d26/fbc 0 2024-03-18T14:51:02.198 INFO:tasks.workunit.client.0.smithi012.stdout:0/704: getdents d1/d20/d23/d6f 0 2024-03-18T14:51:02.198 INFO:tasks.workunit.client.0.smithi012.stdout:0/705: write d1/d20/d23/d82/fc0 [595788,100066] 0 2024-03-18T14:51:02.198 INFO:tasks.workunit.client.0.smithi012.stdout:0/706: write d1/d20/d23/d7f/faf [1584156,30969] 0 2024-03-18T14:51:02.199 INFO:tasks.workunit.client.0.smithi012.stdout:2/702: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:51:02.199 INFO:tasks.workunit.client.0.smithi012.stdout:2/703: truncate d0/da2/f5c 1089023 0 2024-03-18T14:51:02.201 INFO:tasks.workunit.client.0.smithi012.stdout:0/707: dread d1/d16/d19/f7d [0,4194304] 0 2024-03-18T14:51:02.203 INFO:tasks.workunit.client.1.smithi100.stdout:6/301: creat d6/d12/f5c x:0 0 0 2024-03-18T14:51:02.204 INFO:tasks.workunit.client.1.smithi100.stdout:1/333: dread d8/d1e/d11/f21 [0,4194304] 0 2024-03-18T14:51:02.204 INFO:tasks.workunit.client.1.smithi100.stdout:1/334: creat d8/d2a/d37/f5f x:0 0 0 2024-03-18T14:51:02.205 INFO:tasks.workunit.client.1.smithi100.stdout:1/335: write d8/d1e/d11/d27/d3f/f51 [894347,64467] 0 2024-03-18T14:51:02.205 INFO:tasks.workunit.client.1.smithi100.stdout:1/336: write d8/d2a/f47 [966978,103464] 0 2024-03-18T14:51:02.205 INFO:tasks.workunit.client.1.smithi100.stdout:1/337: fsync d8/d1e/f28 0 2024-03-18T14:51:02.212 INFO:tasks.workunit.client.1.smithi100.stdout:2/299: mkdir d3/da/de/d6e 0 2024-03-18T14:51:02.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/618: dwrite d17/d38/d51/f85 [0,4194304] 0 2024-03-18T14:51:02.215 INFO:tasks.workunit.client.1.smithi100.stdout:2/300: dread d3/d6/f2f [0,4194304] 0 2024-03-18T14:51:02.215 INFO:tasks.workunit.client.1.smithi100.stdout:8/349: link d4/l3f d4/dc/d31/l74 0 2024-03-18T14:51:02.220 INFO:tasks.workunit.client.0.smithi012.stdout:5/657: sync 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:1/631: sync 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:7/661: sync 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:3/612: sync 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:7/662: read - dd/d13/d23/d2f/d53/fd2 zero size 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:8/834: sync 2024-03-18T14:51:02.221 INFO:tasks.workunit.client.0.smithi012.stdout:4/594: sync 2024-03-18T14:51:02.222 INFO:tasks.workunit.client.1.smithi100.stdout:9/339: mknod da/dd/d3c/d4b/c5f 0 2024-03-18T14:51:02.222 INFO:tasks.workunit.client.1.smithi100.stdout:9/340: chown l3 1738 1 2024-03-18T14:51:02.222 INFO:tasks.workunit.client.1.smithi100.stdout:9/341: fsync da/dd/d29/f2e 0 2024-03-18T14:51:02.223 INFO:tasks.workunit.client.0.smithi012.stdout:6/619: write d17/d18/f9e [987923,97565] 0 2024-03-18T14:51:02.224 INFO:tasks.workunit.client.0.smithi012.stdout:2/704: symlink d0/d6b/dbf/d33/d71/d90/le0 0 2024-03-18T14:51:02.224 INFO:tasks.workunit.client.0.smithi012.stdout:2/705: read - d0/d6b/dbf/d57/fc4 zero size 2024-03-18T14:51:02.224 INFO:tasks.workunit.client.0.smithi012.stdout:2/706: chown d0/d6b/dbf/d4f/fb1 2 1 2024-03-18T14:51:02.224 INFO:tasks.workunit.client.0.smithi012.stdout:2/707: readlink d0/d3c/d40/d47/d85/lde 0 2024-03-18T14:51:02.226 INFO:tasks.workunit.client.1.smithi100.stdout:5/304: rename d29/d2d/d39/f45 to d29/f5a 0 2024-03-18T14:51:02.228 INFO:tasks.workunit.client.0.smithi012.stdout:0/708: symlink d1/d16/d19/dd0/le8 0 2024-03-18T14:51:02.230 INFO:tasks.workunit.client.1.smithi100.stdout:3/342: mkdir d3/d16/d13/d1f/d7c 0 2024-03-18T14:51:02.231 INFO:tasks.workunit.client.0.smithi012.stdout:0/709: dread d1/d16/d19/f7d [0,4194304] 0 2024-03-18T14:51:02.234 INFO:tasks.workunit.client.1.smithi100.stdout:0/418: mknod dd/d2b/d95/c9b 0 2024-03-18T14:51:02.234 INFO:tasks.workunit.client.1.smithi100.stdout:0/419: stat dd/d20/l70 0 2024-03-18T14:51:02.234 INFO:tasks.workunit.client.1.smithi100.stdout:0/420: creat dd/f9c x:0 0 0 2024-03-18T14:51:02.239 INFO:tasks.workunit.client.0.smithi012.stdout:3/613: dwrite f6 [0,4194304] 0 2024-03-18T14:51:02.239 INFO:tasks.workunit.client.0.smithi012.stdout:3/614: dread - dc/fa5 zero size 2024-03-18T14:51:02.244 INFO:tasks.workunit.client.0.smithi012.stdout:5/658: creat d3/d29/d9d/dba/fd7 x:0 0 0 2024-03-18T14:51:02.244 INFO:tasks.workunit.client.0.smithi012.stdout:5/659: write d3/d15/d42/dc6/f7f [1355879,73039] 0 2024-03-18T14:51:02.244 INFO:tasks.workunit.client.0.smithi012.stdout:5/660: stat d3/d1e/ccb 0 2024-03-18T14:51:02.246 INFO:tasks.workunit.client.0.smithi012.stdout:1/632: creat d8/db/d10/d31/d43/d5e/d65/d70/fcb x:0 0 0 2024-03-18T14:51:02.248 INFO:tasks.workunit.client.0.smithi012.stdout:7/663: link dd/d13/d23/d2f/d53/c87 dd/d13/d23/d33/d48/ce5 0 2024-03-18T14:51:02.249 INFO:tasks.workunit.client.0.smithi012.stdout:9/718: sync 2024-03-18T14:51:02.250 INFO:tasks.workunit.client.1.smithi100.stdout:6/302: rmdir d6/d10/d27 39 2024-03-18T14:51:02.250 INFO:tasks.workunit.client.1.smithi100.stdout:6/303: fsync d6/d10/f22 0 2024-03-18T14:51:02.250 INFO:tasks.workunit.client.1.smithi100.stdout:6/304: write d6/d7/df/d26/f45 [112461,17990] 0 2024-03-18T14:51:02.250 INFO:tasks.workunit.client.1.smithi100.stdout:6/305: fsync d6/d7/f8 0 2024-03-18T14:51:02.252 INFO:tasks.workunit.client.0.smithi012.stdout:8/835: rename d0/d5a/d5e/dd7/lf3 to d0/d5a/d5e/ddf/l109 0 2024-03-18T14:51:02.256 INFO:tasks.workunit.client.1.smithi100.stdout:1/338: rmdir d8/d1e/d11/d2d 39 2024-03-18T14:51:02.256 INFO:tasks.workunit.client.1.smithi100.stdout:1/339: fsync d8/d1e/d11/d27/d3f/f51 0 2024-03-18T14:51:02.256 INFO:tasks.workunit.client.1.smithi100.stdout:1/340: chown d8 1616700831 1 2024-03-18T14:51:02.256 INFO:tasks.workunit.client.0.smithi012.stdout:4/595: dread d6/d1a/f31 [0,4194304] 0 2024-03-18T14:51:02.257 INFO:tasks.workunit.client.0.smithi012.stdout:6/620: rename d17/d38/d51/d66/caa to d17/d38/d51/d66/cbd 0 2024-03-18T14:51:02.257 INFO:tasks.workunit.client.0.smithi012.stdout:6/621: write f4 [4760711,46914] 0 2024-03-18T14:51:02.258 INFO:tasks.workunit.client.0.smithi012.stdout:4/596: dread d6/d1a/f2a [0,4194304] 0 2024-03-18T14:51:02.258 INFO:tasks.workunit.client.1.smithi100.stdout:1/341: dread d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:02.259 INFO:tasks.workunit.client.1.smithi100.stdout:1/342: creat d8/d1e/d11/f60 x:0 0 0 2024-03-18T14:51:02.259 INFO:tasks.workunit.client.0.smithi012.stdout:2/708: mkdir d0/d6b/dbf/d33/d80/de1 0 2024-03-18T14:51:02.265 INFO:tasks.workunit.client.0.smithi012.stdout:3/615: unlink dc/d30/d3a/f5e 0 2024-03-18T14:51:02.265 INFO:tasks.workunit.client.0.smithi012.stdout:3/616: write dc/d30/d35/f75 [924183,39814] 0 2024-03-18T14:51:02.278 INFO:tasks.workunit.client.0.smithi012.stdout:1/633: unlink d8/db/d10/f94 0 2024-03-18T14:51:02.283 INFO:tasks.workunit.client.0.smithi012.stdout:9/719: mkdir d3/de6/dbf/d77/d52/d95/deb 0 2024-03-18T14:51:02.283 INFO:tasks.workunit.client.0.smithi012.stdout:9/720: write d3/de6/dbf/d66/f51 [1768153,102084] 0 2024-03-18T14:51:02.285 INFO:tasks.workunit.client.0.smithi012.stdout:8/836: mkdir d0/d47/d59/dc9/dc4/d10a 0 2024-03-18T14:51:02.289 INFO:tasks.workunit.client.0.smithi012.stdout:6/622: rename d17/d26/c7b to d17/d38/d51/da8/cbe 0 2024-03-18T14:51:02.292 INFO:tasks.workunit.client.0.smithi012.stdout:3/617: mkdir dc/d30/d3a/da0/dc5/dc7 0 2024-03-18T14:51:02.297 INFO:tasks.workunit.client.0.smithi012.stdout:9/721: creat d3/de6/dbf/d66/d69/d7f/d8c/dac/fec x:0 0 0 2024-03-18T14:51:02.299 INFO:tasks.workunit.client.1.smithi100.stdout:1/343: dwrite d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:8/837: symlink d0/d5a/d5e/dd7/l10b 0 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:8/838: chown d0/d5a/d19/l26 312 1 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:2/709: dwrite d0/d6b/dbf/f3a [0,4194304] 0 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:2/710: read d0/da2/fd6 [2557532,24065] 0 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:2/711: chown d0/d3c/fd9 11991 1 2024-03-18T14:51:02.300 INFO:tasks.workunit.client.0.smithi012.stdout:9/722: dread d3/de6/dbf/f4d [0,4194304] 0 2024-03-18T14:51:02.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/597: dwrite d6/d19/d39/fc4 [0,4194304] 0 2024-03-18T14:51:02.301 INFO:tasks.workunit.client.0.smithi012.stdout:4/598: fdatasync d6/d19/d6c/d96/fc7 0 2024-03-18T14:51:02.301 INFO:tasks.workunit.client.0.smithi012.stdout:5/661: dwrite d3/d15/d42/dc6/d73/f99 [0,4194304] 0 2024-03-18T14:51:02.303 INFO:tasks.workunit.client.0.smithi012.stdout:6/623: getdents d17/d18 0 2024-03-18T14:51:02.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/344: dread d8/d1e/d11/d2d/f18 [0,4194304] 0 2024-03-18T14:51:02.310 INFO:tasks.workunit.client.0.smithi012.stdout:3/618: getdents dc/d30/dab 0 2024-03-18T14:51:02.310 INFO:tasks.workunit.client.1.smithi100.stdout:9/342: creat da/dd/f60 x:0 0 0 2024-03-18T14:51:02.311 INFO:tasks.workunit.client.0.smithi012.stdout:1/634: mkdir d8/db/d10/d56/dcc 0 2024-03-18T14:51:02.311 INFO:tasks.workunit.client.1.smithi100.stdout:7/326: rmdir d5/d14/d16/d21 39 2024-03-18T14:51:02.313 INFO:tasks.workunit.client.0.smithi012.stdout:8/839: mknod d0/d5a/d19/dad/c10c 0 2024-03-18T14:51:02.317 INFO:tasks.workunit.client.1.smithi100.stdout:2/301: rename d3/f22 to d3/d13/d40/d53/f6f 0 2024-03-18T14:51:02.319 INFO:tasks.workunit.client.0.smithi012.stdout:2/712: rmdir d0/db4 39 2024-03-18T14:51:02.323 INFO:tasks.workunit.client.1.smithi100.stdout:5/305: mknod d29/d2d/c5b 0 2024-03-18T14:51:02.325 INFO:tasks.workunit.client.0.smithi012.stdout:9/723: mknod d3/de6/dbf/ced 0 2024-03-18T14:51:02.325 INFO:tasks.workunit.client.0.smithi012.stdout:9/724: readlink d3/de6/dbf/d77/d52/d65/lde 0 2024-03-18T14:51:02.325 INFO:tasks.workunit.client.0.smithi012.stdout:9/725: fdatasync d3/de6/dbf/fe3 0 2024-03-18T14:51:02.325 INFO:tasks.workunit.client.0.smithi012.stdout:9/726: truncate d3/de6/dbf/d66/d69/d7f/d8c/dac/fec 516661 0 2024-03-18T14:51:02.326 INFO:tasks.workunit.client.0.smithi012.stdout:8/840: write d0/d5a/d19/dad/f46 [3288958,43704] 0 2024-03-18T14:51:02.326 INFO:tasks.workunit.client.1.smithi100.stdout:2/302: dread d3/f4 [0,4194304] 0 2024-03-18T14:51:02.326 INFO:tasks.workunit.client.1.smithi100.stdout:2/303: truncate d3/da/de/d48/f5d 4283105 0 2024-03-18T14:51:02.329 INFO:tasks.workunit.client.1.smithi100.stdout:3/343: rmdir d3/d16/d13/d1f/d37 39 2024-03-18T14:51:02.329 INFO:tasks.workunit.client.1.smithi100.stdout:3/344: chown d3/d16/d13/d1f/f59 16378 1 2024-03-18T14:51:02.335 INFO:tasks.workunit.client.1.smithi100.stdout:0/421: creat dd/d2b/d95/d79/d7d/f9d x:0 0 0 2024-03-18T14:51:02.335 INFO:tasks.workunit.client.1.smithi100.stdout:0/422: stat dd/d31/d91/c86 0 2024-03-18T14:51:02.337 INFO:tasks.workunit.client.0.smithi012.stdout:4/599: link d6/c10 d6/d19/d6c/cca 0 2024-03-18T14:51:02.337 INFO:tasks.workunit.client.0.smithi012.stdout:4/600: fdatasync d6/d1a/d92/f79 0 2024-03-18T14:51:02.342 INFO:tasks.workunit.client.1.smithi100.stdout:2/304: dread d3/d6/d29/f3d [0,4194304] 0 2024-03-18T14:51:02.345 INFO:tasks.workunit.client.1.smithi100.stdout:7/327: dwrite d5/d22/d3b/d47/f49 [0,4194304] 0 2024-03-18T14:51:02.349 INFO:tasks.workunit.client.0.smithi012.stdout:2/713: dwrite d0/d10/f32 [0,4194304] 0 2024-03-18T14:51:02.358 INFO:tasks.workunit.client.1.smithi100.stdout:1/345: mkdir d8/d30/d61 0 2024-03-18T14:51:02.358 INFO:tasks.workunit.client.0.smithi012.stdout:9/727: dwrite d3/de6/dbf/d77/da8/f76 [0,4194304] 0 2024-03-18T14:51:02.361 INFO:tasks.workunit.client.0.smithi012.stdout:5/662: truncate d3/f3a 127726 0 2024-03-18T14:51:02.361 INFO:tasks.workunit.client.0.smithi012.stdout:5/663: dread - d3/d29/d36/d9b/fbd zero size 2024-03-18T14:51:02.361 INFO:tasks.workunit.client.1.smithi100.stdout:9/343: rmdir da/d46/d4d 39 2024-03-18T14:51:02.366 INFO:tasks.workunit.client.1.smithi100.stdout:5/306: creat d29/d2d/d32/f5c x:0 0 0 2024-03-18T14:51:02.368 INFO:tasks.workunit.client.1.smithi100.stdout:5/307: write f2 [402784,14556] 0 2024-03-18T14:51:02.368 INFO:tasks.workunit.client.1.smithi100.stdout:5/308: creat d29/d2d/d32/d38/f5d x:0 0 0 2024-03-18T14:51:02.369 INFO:tasks.workunit.client.1.smithi100.stdout:5/309: dread - d29/d2d/d32/d38/f5d zero size 2024-03-18T14:51:02.369 INFO:tasks.workunit.client.1.smithi100.stdout:5/310: creat d29/d2d/f5e x:0 0 0 2024-03-18T14:51:02.370 INFO:tasks.workunit.client.1.smithi100.stdout:0/423: creat dd/d20/d44/f9e x:0 0 0 2024-03-18T14:51:02.372 INFO:tasks.workunit.client.1.smithi100.stdout:5/311: dread d29/d2d/d39/f4c [0,4194304] 0 2024-03-18T14:51:02.372 INFO:tasks.workunit.client.1.smithi100.stdout:5/312: fdatasync d29/d2d/d32/f5c 0 2024-03-18T14:51:02.373 INFO:tasks.workunit.client.0.smithi012.stdout:2/714: write d0/d10/f32 [2528092,95831] 0 2024-03-18T14:51:02.379 INFO:tasks.workunit.client.0.smithi012.stdout:9/728: dread d3/d9/f13 [0,4194304] 0 2024-03-18T14:51:02.380 INFO:tasks.workunit.client.0.smithi012.stdout:6/624: creat d17/d38/d51/fbf x:0 0 0 2024-03-18T14:51:02.380 INFO:tasks.workunit.client.0.smithi012.stdout:6/625: readlink d17/d18/db8/la1 0 2024-03-18T14:51:02.383 INFO:tasks.workunit.client.1.smithi100.stdout:2/305: creat d3/d1a/d69/f70 x:0 0 0 2024-03-18T14:51:02.383 INFO:tasks.workunit.client.1.smithi100.stdout:2/306: write f0 [3802481,26408] 0 2024-03-18T14:51:02.385 INFO:tasks.workunit.client.0.smithi012.stdout:3/619: creat dc/d3c/d59/fc8 x:0 0 0 2024-03-18T14:51:02.389 INFO:tasks.workunit.client.1.smithi100.stdout:3/345: dwrite d3/dd/f11 [4194304,4194304] 0 2024-03-18T14:51:02.389 INFO:tasks.workunit.client.1.smithi100.stdout:3/346: chown d3/d16/f36 1957 1 2024-03-18T14:51:02.390 INFO:tasks.workunit.client.0.smithi012.stdout:1/635: rmdir d8/d1e/d27/d44 39 2024-03-18T14:51:02.390 INFO:tasks.workunit.client.0.smithi012.stdout:1/636: stat d8/db/c95 0 2024-03-18T14:51:02.393 INFO:tasks.workunit.client.1.smithi100.stdout:4/337: sync 2024-03-18T14:51:02.395 INFO:tasks.workunit.client.0.smithi012.stdout:8/841: unlink d0/d47/d59/dc9/d87/cb4 0 2024-03-18T14:51:02.396 INFO:tasks.workunit.client.1.smithi100.stdout:2/307: write d3/da/f25 [1942797,81680] 0 2024-03-18T14:51:02.396 INFO:tasks.workunit.client.1.smithi100.stdout:2/308: write d3/d6/d29/d36/d49/f54 [334350,128011] 0 2024-03-18T14:51:02.398 INFO:tasks.workunit.client.1.smithi100.stdout:9/344: symlink da/d51/l61 0 2024-03-18T14:51:02.400 INFO:tasks.workunit.client.0.smithi012.stdout:4/601: mknod d6/d1a/d30/d46/ccb 0 2024-03-18T14:51:02.400 INFO:tasks.workunit.client.0.smithi012.stdout:4/602: stat d6/d76 0 2024-03-18T14:51:02.404 INFO:tasks.workunit.client.0.smithi012.stdout:2/715: dwrite d0/f6e [0,4194304] 0 2024-03-18T14:51:02.405 INFO:tasks.workunit.client.1.smithi100.stdout:5/313: dwrite f7 [0,4194304] 0 2024-03-18T14:51:02.405 INFO:tasks.workunit.client.1.smithi100.stdout:5/314: chown d29/d2d/d32/d38/f52 0 1 2024-03-18T14:51:02.406 INFO:tasks.workunit.client.1.smithi100.stdout:1/346: dwrite d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:02.406 INFO:tasks.workunit.client.1.smithi100.stdout:1/347: read - d8/d1e/d11/d27/d3e/f4b zero size 2024-03-18T14:51:02.406 INFO:tasks.workunit.client.1.smithi100.stdout:8/350: sync 2024-03-18T14:51:02.410 INFO:tasks.workunit.client.0.smithi012.stdout:5/664: mknod d3/d29/cd8 0 2024-03-18T14:51:02.410 INFO:tasks.workunit.client.0.smithi012.stdout:5/665: readlink d3/d29/d35/l81 0 2024-03-18T14:51:02.410 INFO:tasks.workunit.client.0.smithi012.stdout:5/666: dread - d3/d29/f9a zero size 2024-03-18T14:51:02.410 INFO:tasks.workunit.client.0.smithi012.stdout:5/667: chown d3/d29/d36/d9b/d59/f7c 30162 1 2024-03-18T14:51:02.412 INFO:tasks.workunit.client.0.smithi012.stdout:3/620: mknod dc/d3c/d4c/d57/cc9 0 2024-03-18T14:51:02.413 INFO:tasks.workunit.client.0.smithi012.stdout:8/842: creat d0/d47/f10d x:0 0 0 2024-03-18T14:51:02.414 INFO:tasks.workunit.client.1.smithi100.stdout:3/347: mkdir d3/d16/d13/d17/d7d 0 2024-03-18T14:51:02.415 INFO:tasks.workunit.client.1.smithi100.stdout:4/338: chown d9/d13/d5a/d5b 1279002 1 2024-03-18T14:51:02.416 INFO:tasks.workunit.client.1.smithi100.stdout:4/339: creat d9/d13/d5a/f65 x:0 0 0 2024-03-18T14:51:02.421 INFO:tasks.workunit.client.1.smithi100.stdout:2/309: symlink d3/da/l71 0 2024-03-18T14:51:02.422 INFO:tasks.workunit.client.1.smithi100.stdout:9/345: mkdir da/d46/d4d/d62 0 2024-03-18T14:51:02.428 INFO:tasks.workunit.client.0.smithi012.stdout:9/729: dread d3/de6/dbf/d77/da8/f96 [0,4194304] 0 2024-03-18T14:51:02.428 INFO:tasks.workunit.client.1.smithi100.stdout:1/348: symlink d8/dd/l62 0 2024-03-18T14:51:02.430 INFO:tasks.workunit.client.1.smithi100.stdout:8/351: mkdir d4/dc/d1f/d59/d6c/d75 0 2024-03-18T14:51:02.430 INFO:tasks.workunit.client.1.smithi100.stdout:8/352: readlink d4/dc/l58 0 2024-03-18T14:51:02.432 INFO:tasks.workunit.client.1.smithi100.stdout:1/349: write d8/d1e/d11/f33 [258226,10509] 0 2024-03-18T14:51:02.433 INFO:tasks.workunit.client.1.smithi100.stdout:3/348: creat d3/dd/d21/d3e/f7e x:0 0 0 2024-03-18T14:51:02.433 INFO:tasks.workunit.client.1.smithi100.stdout:3/349: creat d3/d16/d13/d17/f7f x:0 0 0 2024-03-18T14:51:02.433 INFO:tasks.workunit.client.1.smithi100.stdout:3/350: creat d3/dd/d21/d3e/f80 x:0 0 0 2024-03-18T14:51:02.434 INFO:tasks.workunit.client.1.smithi100.stdout:3/351: dread - d3/d16/d13/d17/f61 zero size 2024-03-18T14:51:02.434 INFO:tasks.workunit.client.1.smithi100.stdout:3/352: creat d3/d16/d13/d1f/d3f/d4f/f81 x:0 0 0 2024-03-18T14:51:02.434 INFO:tasks.workunit.client.0.smithi012.stdout:2/716: rmdir d0/d3c/d40/d47/d85 39 2024-03-18T14:51:02.434 INFO:tasks.workunit.client.0.smithi012.stdout:2/717: dread - d0/d6b/dbf/d33/d80/fc6 zero size 2024-03-18T14:51:02.435 INFO:tasks.workunit.client.1.smithi100.stdout:5/315: dwrite f20 [0,4194304] 0 2024-03-18T14:51:02.435 INFO:tasks.workunit.client.1.smithi100.stdout:5/316: creat d29/d2d/d32/d38/f5f x:0 0 0 2024-03-18T14:51:02.435 INFO:tasks.workunit.client.1.smithi100.stdout:5/317: fdatasync d29/d2d/d39/f4e 0 2024-03-18T14:51:02.436 INFO:tasks.workunit.client.0.smithi012.stdout:4/603: dwrite d6/d1a/d30/d46/d6f/d7c/f87 [4194304,4194304] 0 2024-03-18T14:51:02.436 INFO:tasks.workunit.client.0.smithi012.stdout:4/604: chown d6/d1a/d30/d46/c63 19307 1 2024-03-18T14:51:02.437 INFO:tasks.workunit.client.0.smithi012.stdout:3/621: getdents dc/d3c/d72 0 2024-03-18T14:51:02.438 INFO:tasks.workunit.client.1.smithi100.stdout:4/340: unlink d9/f1a 0 2024-03-18T14:51:02.438 INFO:tasks.workunit.client.1.smithi100.stdout:4/341: chown d9/d17/f1d 24 1 2024-03-18T14:51:02.441 INFO:tasks.workunit.client.1.smithi100.stdout:7/328: rename d5/d48 to d5/d14/d16/d2e/d7b 0 2024-03-18T14:51:02.441 INFO:tasks.workunit.client.1.smithi100.stdout:7/329: fdatasync d5/d22/d24/f79 0 2024-03-18T14:51:02.442 INFO:tasks.workunit.client.0.smithi012.stdout:4/605: mknod d6/da/d89/d97/dc5/ccc 0 2024-03-18T14:51:02.443 INFO:tasks.workunit.client.1.smithi100.stdout:9/346: truncate da/d25/f3a 289403 0 2024-03-18T14:51:02.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/730: dread d3/de6/dbf/f6a [0,4194304] 0 2024-03-18T14:51:02.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/731: write d3/d9/f29 [1513929,19028] 0 2024-03-18T14:51:02.445 INFO:tasks.workunit.client.1.smithi100.stdout:8/353: getdents d4/dc/dd/de/d19/d37 0 2024-03-18T14:51:02.446 INFO:tasks.workunit.client.0.smithi012.stdout:5/668: dwrite d3/d84/f85 [0,4194304] 0 2024-03-18T14:51:02.446 INFO:tasks.workunit.client.0.smithi012.stdout:3/622: rename dc/d3c/d4c/d5f/c90 to dc/d3c/d59/d86/d9a/d54/cca 0 2024-03-18T14:51:02.448 INFO:tasks.workunit.client.0.smithi012.stdout:2/718: write d0/d6b/dbf/f3a [3314314,15846] 0 2024-03-18T14:51:02.448 INFO:tasks.workunit.client.0.smithi012.stdout:8/843: dwrite d0/d5a/d19/dad/f2d [0,4194304] 0 2024-03-18T14:51:02.448 INFO:tasks.workunit.client.0.smithi012.stdout:2/719: write d0/d6b/dbf/d33/d99/fce [893930,77613] 0 2024-03-18T14:51:02.449 INFO:tasks.workunit.client.0.smithi012.stdout:4/606: unlink d6/d1a/d92/f5e 0 2024-03-18T14:51:02.451 INFO:tasks.workunit.client.0.smithi012.stdout:2/720: dread d0/d3c/d40/d47/d85/f94 [0,4194304] 0 2024-03-18T14:51:02.454 INFO:tasks.workunit.client.0.smithi012.stdout:0/710: sync 2024-03-18T14:51:02.454 INFO:tasks.workunit.client.0.smithi012.stdout:7/664: sync 2024-03-18T14:51:02.454 INFO:tasks.workunit.client.0.smithi012.stdout:7/665: fsync dd/d13/d23/d2f/d53/fd2 0 2024-03-18T14:51:02.454 INFO:tasks.workunit.client.0.smithi012.stdout:7/666: creat dd/d13/d23/d76/d6f/db3/dc7/fe6 x:0 0 0 2024-03-18T14:51:02.454 INFO:tasks.workunit.client.0.smithi012.stdout:2/721: fdatasync d0/d6b/dbf/f3a 0 2024-03-18T14:51:02.455 INFO:tasks.workunit.client.0.smithi012.stdout:3/623: dread dc/d30/d3a/f4f [0,4194304] 0 2024-03-18T14:51:02.456 INFO:tasks.workunit.client.0.smithi012.stdout:2/722: dread d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:51:02.457 INFO:tasks.workunit.client.0.smithi012.stdout:2/723: truncate d0/d6c/fdc 234378 0 2024-03-18T14:51:02.457 INFO:tasks.workunit.client.1.smithi100.stdout:3/353: creat d3/d16/d13/d17/d7d/f82 x:0 0 0 2024-03-18T14:51:02.458 INFO:tasks.workunit.client.0.smithi012.stdout:5/669: creat d3/d29/d36/d9b/fd9 x:0 0 0 2024-03-18T14:51:02.461 INFO:tasks.workunit.client.0.smithi012.stdout:8/844: unlink d0/d15/f43 0 2024-03-18T14:51:02.462 INFO:tasks.workunit.client.0.smithi012.stdout:8/845: chown d0/d5a/d19/d63/dbe/ce7 248808933 1 2024-03-18T14:51:02.462 INFO:tasks.workunit.client.1.smithi100.stdout:6/306: sync 2024-03-18T14:51:02.464 INFO:tasks.workunit.client.1.smithi100.stdout:6/307: dread d6/d7/f40 [0,4194304] 0 2024-03-18T14:51:02.465 INFO:tasks.workunit.client.1.smithi100.stdout:0/424: rename dd/c2a to dd/d31/d91/d7b/c9f 0 2024-03-18T14:51:02.465 INFO:tasks.workunit.client.1.smithi100.stdout:0/425: creat dd/d20/fa0 x:0 0 0 2024-03-18T14:51:02.467 INFO:tasks.workunit.client.1.smithi100.stdout:7/330: mknod d5/d14/d16/d2e/d62/d66/d4f/c7c 0 2024-03-18T14:51:02.471 INFO:tasks.workunit.client.1.smithi100.stdout:8/354: unlink d4/l18 0 2024-03-18T14:51:02.473 INFO:tasks.workunit.client.1.smithi100.stdout:0/426: dread dd/f16 [0,4194304] 0 2024-03-18T14:51:02.473 INFO:tasks.workunit.client.1.smithi100.stdout:3/354: dwrite d3/d16/d13/d1f/d3f/d4f/f51 [0,4194304] 0 2024-03-18T14:51:02.474 INFO:tasks.workunit.client.1.smithi100.stdout:3/355: truncate d3/dd/f38 1011404 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/308: creat d6/d10/d27/f5d x:0 0 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/309: write d6/d12/f3a [4811726,20216] 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/310: write d6/d10/d27/f5d [448259,99970] 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/311: fsync d6/d12/d2c/f4c 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/312: fdatasync d6/d7/df/d26/f56 0 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/313: dread - d6/d10/f5b zero size 2024-03-18T14:51:02.477 INFO:tasks.workunit.client.1.smithi100.stdout:6/314: write d6/d12/d2c/f4c [783013,104241] 0 2024-03-18T14:51:02.478 INFO:tasks.workunit.client.1.smithi100.stdout:2/310: rename d3/d13/f37 to d3/d6/d5f/d63/f72 0 2024-03-18T14:51:02.479 INFO:tasks.workunit.client.1.smithi100.stdout:2/311: read - d3/d6/d5f/f6d zero size 2024-03-18T14:51:02.479 INFO:tasks.workunit.client.0.smithi012.stdout:3/624: creat dc/fcb x:0 0 0 2024-03-18T14:51:02.481 INFO:tasks.workunit.client.0.smithi012.stdout:2/724: mkdir d0/d6c/de2 0 2024-03-18T14:51:02.484 INFO:tasks.workunit.client.0.smithi012.stdout:8/846: symlink d0/d5a/d5e/dd7/d84/d54/d9f/l10e 0 2024-03-18T14:51:02.486 INFO:tasks.workunit.client.0.smithi012.stdout:4/607: rename d6/d1a/d30/d46/d6f/d7c/f87 to d6/d76/fcd 0 2024-03-18T14:51:02.487 INFO:tasks.workunit.client.0.smithi012.stdout:0/711: write d1/d16/d31/f1d [2288626,45473] 0 2024-03-18T14:51:02.487 INFO:tasks.workunit.client.0.smithi012.stdout:0/712: write d1/d20/d4e/d54/d67/f72 [2630736,61904] 0 2024-03-18T14:51:02.488 INFO:tasks.workunit.client.1.smithi100.stdout:9/347: rmdir da/d2c 39 2024-03-18T14:51:02.489 INFO:tasks.workunit.client.0.smithi012.stdout:2/725: mknod d0/da2/ce3 0 2024-03-18T14:51:02.493 INFO:tasks.workunit.client.0.smithi012.stdout:7/667: rename dd/d13/d23/lbd to dd/d13/d23/d76/d6f/db3/dc7/le7 0 2024-03-18T14:51:02.493 INFO:tasks.workunit.client.0.smithi012.stdout:7/668: write dd/d13/d23/d2f/f9f [1166987,19372] 0 2024-03-18T14:51:02.494 INFO:tasks.workunit.client.0.smithi012.stdout:4/608: creat d6/da/d89/fce x:0 0 0 2024-03-18T14:51:02.494 INFO:tasks.workunit.client.0.smithi012.stdout:4/609: fdatasync d6/d1a/f31 0 2024-03-18T14:51:02.496 INFO:tasks.workunit.client.1.smithi100.stdout:0/427: mknod dd/d20/ca1 0 2024-03-18T14:51:02.496 INFO:tasks.workunit.client.1.smithi100.stdout:0/428: write dd/d20/f2d [1019558,110920] 0 2024-03-18T14:51:02.496 INFO:tasks.workunit.client.1.smithi100.stdout:0/429: stat dd/d2b/d95 0 2024-03-18T14:51:02.497 INFO:tasks.workunit.client.1.smithi100.stdout:0/430: stat dd/d31/d91/d56/d59 0 2024-03-18T14:51:02.497 INFO:tasks.workunit.client.1.smithi100.stdout:0/431: stat dd/d2b/d95/d79/d7d/f9d 0 2024-03-18T14:51:02.497 INFO:tasks.workunit.client.1.smithi100.stdout:0/432: dread - dd/d31/d91/d56/d71/f90 zero size 2024-03-18T14:51:02.498 INFO:tasks.workunit.client.0.smithi012.stdout:5/670: rename d3/d1e/fa4 to d3/d15/d42/dc6/d73/d6b/d8f/fda 0 2024-03-18T14:51:02.498 INFO:tasks.workunit.client.0.smithi012.stdout:5/671: stat d3/db6/fca 0 2024-03-18T14:51:02.500 INFO:tasks.workunit.client.1.smithi100.stdout:3/356: creat d3/d16/d13/d1f/d7c/f83 x:0 0 0 2024-03-18T14:51:02.500 INFO:tasks.workunit.client.1.smithi100.stdout:3/357: chown d3/d16/d13/d1f/d3f/d4f/d65/l7b 43936540 1 2024-03-18T14:51:02.500 INFO:tasks.workunit.client.1.smithi100.stdout:3/358: creat d3/d16/f84 x:0 0 0 2024-03-18T14:51:02.502 INFO:tasks.workunit.client.0.smithi012.stdout:4/610: dread d6/f25 [0,4194304] 0 2024-03-18T14:51:02.504 INFO:tasks.workunit.client.0.smithi012.stdout:0/713: rename d1/d20/d4e/d54/d67/dbc/cdf to d1/d20/d23/ce9 0 2024-03-18T14:51:02.505 INFO:tasks.workunit.client.1.smithi100.stdout:6/315: creat d6/f5e x:0 0 0 2024-03-18T14:51:02.507 INFO:tasks.workunit.client.0.smithi012.stdout:5/672: link d3/fc d3/d29/d36/d9b/d59/d8b/fdb 0 2024-03-18T14:51:02.507 INFO:tasks.workunit.client.0.smithi012.stdout:5/673: chown d3/d15/f1b 221729 1 2024-03-18T14:51:02.508 INFO:tasks.workunit.client.0.smithi012.stdout:2/726: rename d0/d3c/d40/d47/d85/lde to d0/d3c/d40/d47/le4 0 2024-03-18T14:51:02.509 INFO:tasks.workunit.client.0.smithi012.stdout:0/714: mkdir d1/d20/d23/d3b/d33/dea 0 2024-03-18T14:51:02.509 INFO:tasks.workunit.client.0.smithi012.stdout:0/715: readlink d1/la6 0 2024-03-18T14:51:02.509 INFO:tasks.workunit.client.0.smithi012.stdout:0/716: read d1/d16/f17 [625030,22126] 0 2024-03-18T14:51:02.509 INFO:tasks.workunit.client.0.smithi012.stdout:0/717: chown d1/d16/d19/d8d 4433 1 2024-03-18T14:51:02.510 INFO:tasks.workunit.client.0.smithi012.stdout:3/625: dwrite dc/d30/d35/f70 [0,4194304] 0 2024-03-18T14:51:02.511 INFO:tasks.workunit.client.0.smithi012.stdout:8/847: dwrite d0/d5a/d5e/ddf/f5c [0,4194304] 0 2024-03-18T14:51:02.511 INFO:tasks.workunit.client.0.smithi012.stdout:8/848: dread - d0/d5a/d55/f105 zero size 2024-03-18T14:51:02.511 INFO:tasks.workunit.client.1.smithi100.stdout:7/331: rmdir d5/d14/d16 39 2024-03-18T14:51:02.514 INFO:tasks.workunit.client.0.smithi012.stdout:7/669: rename dd/d13/d23/d60/f95 to dd/d13/d23/d76/d6b/fe8 0 2024-03-18T14:51:02.517 INFO:tasks.workunit.client.0.smithi012.stdout:0/718: dread d1/d20/d23/d26/f90 [0,4194304] 0 2024-03-18T14:51:02.518 INFO:tasks.workunit.client.1.smithi100.stdout:9/348: creat da/d46/f63 x:0 0 0 2024-03-18T14:51:02.520 INFO:tasks.workunit.client.1.smithi100.stdout:0/433: symlink dd/d31/d91/d7b/la2 0 2024-03-18T14:51:02.520 INFO:tasks.workunit.client.1.smithi100.stdout:0/434: write dd/d20/fa0 [477776,124276] 0 2024-03-18T14:51:02.524 INFO:tasks.workunit.client.1.smithi100.stdout:3/359: rmdir d3/d16/d13/d1f/d37 39 2024-03-18T14:51:02.526 INFO:tasks.workunit.client.0.smithi012.stdout:3/626: symlink dc/d91/lcc 0 2024-03-18T14:51:02.526 INFO:tasks.workunit.client.0.smithi012.stdout:3/627: write dc/d3c/f4d [45030,61283] 0 2024-03-18T14:51:02.526 INFO:tasks.workunit.client.0.smithi012.stdout:3/628: readlink dc/d30/l3b 0 2024-03-18T14:51:02.530 INFO:tasks.workunit.client.1.smithi100.stdout:4/342: rename d9/la to d9/d13/l66 0 2024-03-18T14:51:02.532 INFO:tasks.workunit.client.1.smithi100.stdout:9/349: write da/db/f13 [3022645,20817] 0 2024-03-18T14:51:02.533 INFO:tasks.workunit.client.1.smithi100.stdout:0/435: dread dd/d2b/f3d [0,4194304] 0 2024-03-18T14:51:02.534 INFO:tasks.workunit.client.1.smithi100.stdout:6/316: dwrite d6/d10/d2b/d36/d4f/f58 [0,4194304] 0 2024-03-18T14:51:02.534 INFO:tasks.workunit.client.1.smithi100.stdout:6/317: creat d6/d7/f5f x:0 0 0 2024-03-18T14:51:02.534 INFO:tasks.workunit.client.1.smithi100.stdout:6/318: chown d6/d7/f59 30303 1 2024-03-18T14:51:02.535 INFO:tasks.workunit.client.1.smithi100.stdout:3/360: link d3/d16/c2f d3/dd/d21/c85 0 2024-03-18T14:51:02.535 INFO:tasks.workunit.client.1.smithi100.stdout:3/361: dread - d3/dd/d21/f55 zero size 2024-03-18T14:51:02.535 INFO:tasks.workunit.client.0.smithi012.stdout:8/849: creat d0/d5a/d5e/dd7/d84/d54/d9f/f10f x:0 0 0 2024-03-18T14:51:02.537 INFO:tasks.workunit.client.0.smithi012.stdout:5/674: getdents d3/d29/d36/d9b/d59/d8b 0 2024-03-18T14:51:02.537 INFO:tasks.workunit.client.0.smithi012.stdout:5/675: chown d3/d15/d5f/c54 2712524 1 2024-03-18T14:51:02.539 INFO:tasks.workunit.client.1.smithi100.stdout:8/355: rename d4/dc/c2a to d4/dc/dd/de/d19/d37/c76 0 2024-03-18T14:51:02.540 INFO:tasks.workunit.client.1.smithi100.stdout:8/356: fsync d4/dc/d1f/f32 0 2024-03-18T14:51:02.540 INFO:tasks.workunit.client.1.smithi100.stdout:8/357: chown d4/dc/dd/de/l29 769 1 2024-03-18T14:51:02.540 INFO:tasks.workunit.client.1.smithi100.stdout:7/332: dwrite d5/d14/d16/d44/f76 [0,4194304] 0 2024-03-18T14:51:02.540 INFO:tasks.workunit.client.0.smithi012.stdout:2/727: dwrite d0/d6b/dbf/d33/d80/fc8 [0,4194304] 0 2024-03-18T14:51:02.540 INFO:tasks.workunit.client.0.smithi012.stdout:2/728: chown d0/l30 10 1 2024-03-18T14:51:02.543 INFO:tasks.workunit.client.0.smithi012.stdout:7/670: mknod dd/d13/d23/d33/d37/ce9 0 2024-03-18T14:51:02.543 INFO:tasks.workunit.client.0.smithi012.stdout:7/671: creat dd/d13/d23/d76/d6f/db3/dc7/fea x:0 0 0 2024-03-18T14:51:02.544 INFO:tasks.workunit.client.0.smithi012.stdout:7/672: fdatasync fa 0 2024-03-18T14:51:02.544 INFO:tasks.workunit.client.0.smithi012.stdout:7/673: truncate dd/d13/d23/d2f/d7a/db6/f9b 101473 0 2024-03-18T14:51:02.545 INFO:tasks.workunit.client.0.smithi012.stdout:2/729: dread d0/d6b/dbf/f3a [0,4194304] 0 2024-03-18T14:51:02.547 INFO:tasks.workunit.client.0.smithi012.stdout:3/629: creat dc/d3c/d59/d86/fcd x:0 0 0 2024-03-18T14:51:02.547 INFO:tasks.workunit.client.0.smithi012.stdout:3/630: stat dc/d30/d35/l34 0 2024-03-18T14:51:02.549 INFO:tasks.workunit.client.0.smithi012.stdout:8/850: creat d0/d5a/d5e/dd7/d84/d54/d62/daf/de8/f110 x:0 0 0 2024-03-18T14:51:02.550 INFO:tasks.workunit.client.0.smithi012.stdout:5/676: rename d3/d1e/cac to d3/d15/cdc 0 2024-03-18T14:51:02.553 INFO:tasks.workunit.client.0.smithi012.stdout:0/719: dread d1/d20/d7b/f81 [0,4194304] 0 2024-03-18T14:51:02.554 INFO:tasks.workunit.client.0.smithi012.stdout:0/720: stat d1/d20/d23/d3b 0 2024-03-18T14:51:02.558 INFO:tasks.workunit.client.0.smithi012.stdout:7/674: rmdir dd/d4f 39 2024-03-18T14:51:02.561 INFO:tasks.workunit.client.0.smithi012.stdout:7/675: dread dd/d4f/f69 [0,4194304] 0 2024-03-18T14:51:02.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/730: mkdir d0/d6b/dbf/d33/d71/de5 0 2024-03-18T14:51:02.563 INFO:tasks.workunit.client.1.smithi100.stdout:6/319: symlink d6/d10/d2b/d36/l60 0 2024-03-18T14:51:02.564 INFO:tasks.workunit.client.0.smithi012.stdout:3/631: symlink dc/d3c/d4c/lce 0 2024-03-18T14:51:02.566 INFO:tasks.workunit.client.1.smithi100.stdout:3/362: rmdir d3 39 2024-03-18T14:51:02.566 INFO:tasks.workunit.client.1.smithi100.stdout:3/363: chown d3/d16/d13/d1f/d3f/d4f/c50 3 1 2024-03-18T14:51:02.567 INFO:tasks.workunit.client.0.smithi012.stdout:8/851: symlink d0/d5a/d5e/dd7/d84/d2a/l111 0 2024-03-18T14:51:02.572 INFO:tasks.workunit.client.1.smithi100.stdout:2/312: rename d3/d13/d40/d53/c5a to d3/d1a/d69/c73 0 2024-03-18T14:51:02.575 INFO:tasks.workunit.client.0.smithi012.stdout:7/676: mknod dd/d13/d23/ceb 0 2024-03-18T14:51:02.576 INFO:tasks.workunit.client.0.smithi012.stdout:7/677: creat dd/d13/d23/d2f/d53/fec x:0 0 0 2024-03-18T14:51:02.576 INFO:tasks.workunit.client.0.smithi012.stdout:7/678: dread - dd/d13/d23/d76/d6f/dd9/fdf zero size 2024-03-18T14:51:02.576 INFO:tasks.workunit.client.0.smithi012.stdout:7/679: readlink dd/d13/d23/d2f/d7a/dab/lba 0 2024-03-18T14:51:02.577 INFO:tasks.workunit.client.1.smithi100.stdout:6/320: dread d6/f2f [0,4194304] 0 2024-03-18T14:51:02.578 INFO:tasks.workunit.client.0.smithi012.stdout:2/731: creat d0/d6b/dbf/d4f/fe6 x:0 0 0 2024-03-18T14:51:02.578 INFO:tasks.workunit.client.0.smithi012.stdout:2/732: write d0/d10/f13 [3131461,38891] 0 2024-03-18T14:51:02.578 INFO:tasks.workunit.client.0.smithi012.stdout:2/733: write d0/d6b/dbf/d33/dc9/fcd [864594,44647] 0 2024-03-18T14:51:02.579 INFO:tasks.workunit.client.0.smithi012.stdout:5/677: dwrite d3/d15/d42/dc6/d73/d6b/fcd [0,4194304] 0 2024-03-18T14:51:02.580 INFO:tasks.workunit.client.0.smithi012.stdout:3/632: symlink dc/d3c/d4c/d5f/lcf 0 2024-03-18T14:51:02.580 INFO:tasks.workunit.client.0.smithi012.stdout:3/633: dread - dc/fcb zero size 2024-03-18T14:51:02.582 INFO:tasks.workunit.client.0.smithi012.stdout:8/852: creat d0/d5a/d5e/dd7/d84/f112 x:0 0 0 2024-03-18T14:51:02.583 INFO:tasks.workunit.client.1.smithi100.stdout:7/333: rmdir d5/d14/d16/d21 39 2024-03-18T14:51:02.583 INFO:tasks.workunit.client.1.smithi100.stdout:7/334: dread - d5/d14/d16/d2c/f74 zero size 2024-03-18T14:51:02.583 INFO:tasks.workunit.client.1.smithi100.stdout:7/335: read d5/f35 [3122076,56232] 0 2024-03-18T14:51:02.584 INFO:tasks.workunit.client.0.smithi012.stdout:3/634: dread dc/f48 [0,4194304] 0 2024-03-18T14:51:02.584 INFO:tasks.workunit.client.0.smithi012.stdout:3/635: chown dc/d3c/d72/l7d 33179999 1 2024-03-18T14:51:02.584 INFO:tasks.workunit.client.0.smithi012.stdout:3/636: chown dc/d3c/d4c/d5f/lcf 21 1 2024-03-18T14:51:02.584 INFO:tasks.workunit.client.1.smithi100.stdout:2/313: dread d3/d6/f44 [0,4194304] 0 2024-03-18T14:51:02.585 INFO:tasks.workunit.client.1.smithi100.stdout:1/350: sync 2024-03-18T14:51:02.586 INFO:tasks.workunit.client.0.smithi012.stdout:0/721: dwrite d1/d20/f2e [0,4194304] 0 2024-03-18T14:51:02.586 INFO:tasks.workunit.client.0.smithi012.stdout:0/722: write d1/d16/d31/f50 [5628785,8145] 0 2024-03-18T14:51:02.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/364: dwrite d3/d16/d13/d1f/f78 [0,4194304] 0 2024-03-18T14:51:02.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/365: chown d3/dd/d49/f72 21 1 2024-03-18T14:51:02.586 INFO:tasks.workunit.client.1.smithi100.stdout:9/350: symlink da/db/l64 0 2024-03-18T14:51:02.587 INFO:tasks.workunit.client.0.smithi012.stdout:7/680: rmdir dd/d13/d23/d60 39 2024-03-18T14:51:02.587 INFO:tasks.workunit.client.0.smithi012.stdout:7/681: fsync dd/d13/d23/d2f/d7a/db6/f9b 0 2024-03-18T14:51:02.589 INFO:tasks.workunit.client.1.smithi100.stdout:0/436: rename dd/d61/f8e to dd/d31/d91/d56/d71/fa3 0 2024-03-18T14:51:02.589 INFO:tasks.workunit.client.0.smithi012.stdout:2/734: rmdir d0/d6b 39 2024-03-18T14:51:02.592 INFO:tasks.workunit.client.1.smithi100.stdout:8/358: link d4/dc/dd/de/d21/d23/d25/l3e d4/l77 0 2024-03-18T14:51:02.592 INFO:tasks.workunit.client.1.smithi100.stdout:8/359: fsync d4/dc/dd/de/d21/d23/f62 0 2024-03-18T14:51:02.595 INFO:tasks.workunit.client.1.smithi100.stdout:2/314: read d3/f10 [4037159,90347] 0 2024-03-18T14:51:02.595 INFO:tasks.workunit.client.1.smithi100.stdout:6/321: link d6/d7/df/d26/l28 d6/d7/df/l61 0 2024-03-18T14:51:02.596 INFO:tasks.workunit.client.1.smithi100.stdout:1/351: dread d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:51:02.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/437: dread dd/d20/f2c [0,4194304] 0 2024-03-18T14:51:02.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/438: readlink dd/d20/d44/l6c 0 2024-03-18T14:51:02.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/439: chown dd/d31/d91/d56 1999473 1 2024-03-18T14:51:02.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/440: chown dd/d20/f4d 1 1 2024-03-18T14:51:02.598 INFO:tasks.workunit.client.1.smithi100.stdout:5/318: sync 2024-03-18T14:51:02.599 INFO:tasks.workunit.client.1.smithi100.stdout:7/336: creat d5/d14/d16/d2e/f7d x:0 0 0 2024-03-18T14:51:02.599 INFO:tasks.workunit.client.0.smithi012.stdout:8/853: dwrite d0/d5a/f9a [0,4194304] 0 2024-03-18T14:51:02.599 INFO:tasks.workunit.client.0.smithi012.stdout:8/854: chown d0 0 1 2024-03-18T14:51:02.601 INFO:tasks.workunit.client.0.smithi012.stdout:5/678: rename d3/d29/cd8 to d3/cdd 0 2024-03-18T14:51:02.603 INFO:tasks.workunit.client.0.smithi012.stdout:3/637: creat dc/d30/dab/fd0 x:0 0 0 2024-03-18T14:51:02.604 INFO:tasks.workunit.client.0.smithi012.stdout:3/638: dread - dc/d3c/d4c/fc6 zero size 2024-03-18T14:51:02.607 INFO:tasks.workunit.client.1.smithi100.stdout:1/352: write d8/d1e/f36 [171665,99732] 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:3/366: creat d3/d16/d13/d1f/d37/f86 x:0 0 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:3/367: write d3/d16/d13/d1f/d37/f54 [1458239,36947] 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:3/368: stat d3/d16/l18 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/337: dread d5/f52 [0,4194304] 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/338: fsync d5/d22/d24/f79 0 2024-03-18T14:51:02.608 INFO:tasks.workunit.client.1.smithi100.stdout:7/339: fdatasync d5/d30/d4d/d5a/f78 0 2024-03-18T14:51:02.609 INFO:tasks.workunit.client.0.smithi012.stdout:0/723: creat d1/d20/d23/d6f/feb x:0 0 0 2024-03-18T14:51:02.609 INFO:tasks.workunit.client.0.smithi012.stdout:0/724: creat d1/d16/d31/fec x:0 0 0 2024-03-18T14:51:02.610 INFO:tasks.workunit.client.1.smithi100.stdout:9/351: symlink da/dd/d3c/l65 0 2024-03-18T14:51:02.613 INFO:tasks.workunit.client.1.smithi100.stdout:9/352: dread da/dd/d29/f2e [0,4194304] 0 2024-03-18T14:51:02.614 INFO:tasks.workunit.client.1.smithi100.stdout:8/360: getdents d4/dc 0 2024-03-18T14:51:02.614 INFO:tasks.workunit.client.1.smithi100.stdout:8/361: write d4/dc/dd/de/d21/d23/f28 [999737,115010] 0 2024-03-18T14:51:02.615 INFO:tasks.workunit.client.1.smithi100.stdout:8/362: creat d4/dc/dd/de/d21/d23/f78 x:0 0 0 2024-03-18T14:51:02.615 INFO:tasks.workunit.client.0.smithi012.stdout:7/682: truncate dd/d13/d23/d60/f89 18906 0 2024-03-18T14:51:02.616 INFO:tasks.workunit.client.1.smithi100.stdout:2/315: stat d3/da/de/c31 0 2024-03-18T14:51:02.617 INFO:tasks.workunit.client.0.smithi012.stdout:2/735: creat d0/d6b/fe7 x:0 0 0 2024-03-18T14:51:02.622 INFO:tasks.workunit.client.1.smithi100.stdout:0/441: creat dd/d2b/d95/d79/fa4 x:0 0 0 2024-03-18T14:51:02.622 INFO:tasks.workunit.client.1.smithi100.stdout:0/442: dread - dd/d2b/d95/d79/d7d/f7e zero size 2024-03-18T14:51:02.622 INFO:tasks.workunit.client.1.smithi100.stdout:0/443: fsync dd/d20/d44/f9e 0 2024-03-18T14:51:02.622 INFO:tasks.workunit.client.1.smithi100.stdout:0/444: dread - dd/d2b/d95/d79/d7d/f9d zero size 2024-03-18T14:51:02.625 INFO:tasks.workunit.client.1.smithi100.stdout:5/319: unlink d29/d2d/d32/c4b 0 2024-03-18T14:51:02.625 INFO:tasks.workunit.client.1.smithi100.stdout:5/320: read - d29/d2d/f5e zero size 2024-03-18T14:51:02.629 INFO:tasks.workunit.client.1.smithi100.stdout:9/353: dwrite da/fc [4194304,4194304] 0 2024-03-18T14:51:02.629 INFO:tasks.workunit.client.1.smithi100.stdout:9/354: creat da/d25/f66 x:0 0 0 2024-03-18T14:51:02.629 INFO:tasks.workunit.client.1.smithi100.stdout:1/353: creat d8/d30/d61/f63 x:0 0 0 2024-03-18T14:51:02.631 INFO:tasks.workunit.client.1.smithi100.stdout:3/369: link d3/dd/d21/d3e/f7e d3/dd/d49/f87 0 2024-03-18T14:51:02.633 INFO:tasks.workunit.client.1.smithi100.stdout:7/340: mknod d5/d22/c7e 0 2024-03-18T14:51:02.637 INFO:tasks.workunit.client.0.smithi012.stdout:8/855: creat d0/d5a/d19/d63/dbe/f113 x:0 0 0 2024-03-18T14:51:02.639 INFO:tasks.workunit.client.0.smithi012.stdout:5/679: link d3/d29/d36/d9b/fd9 d3/d29/d36/d9b/d59/fde 0 2024-03-18T14:51:02.639 INFO:tasks.workunit.client.0.smithi012.stdout:5/680: stat d3/d1e 0 2024-03-18T14:51:02.643 INFO:tasks.workunit.client.1.smithi100.stdout:8/363: symlink d4/dc/dd/l79 0 2024-03-18T14:51:02.643 INFO:tasks.workunit.client.1.smithi100.stdout:8/364: chown d4/dc/c10 284 1 2024-03-18T14:51:02.643 INFO:tasks.workunit.client.1.smithi100.stdout:8/365: write d4/dc/d1f/f32 [564745,66162] 0 2024-03-18T14:51:02.643 INFO:tasks.workunit.client.1.smithi100.stdout:8/366: readlink d4/dc/dd/de/d19/l22 0 2024-03-18T14:51:02.645 INFO:tasks.workunit.client.1.smithi100.stdout:2/316: link d3/d13/d4f/c62 d3/da/de/c74 0 2024-03-18T14:51:02.645 INFO:tasks.workunit.client.1.smithi100.stdout:2/317: fdatasync d3/d13/d40/d53/f65 0 2024-03-18T14:51:02.645 INFO:tasks.workunit.client.1.smithi100.stdout:1/354: dwrite d8/d1e/d11/f21 [0,4194304] 0 2024-03-18T14:51:02.645 INFO:tasks.workunit.client.1.smithi100.stdout:1/355: chown d8/d1e/d11/d27/f4d 23896297 1 2024-03-18T14:51:02.646 INFO:tasks.workunit.client.0.smithi012.stdout:0/725: link d1/d20/d23/d3b/d33/d64/c6b d1/d16/d6e/dcc/ced 0 2024-03-18T14:51:02.646 INFO:tasks.workunit.client.0.smithi012.stdout:0/726: write d1/d20/d23/d88/fd2 [171891,21710] 0 2024-03-18T14:51:02.646 INFO:tasks.workunit.client.0.smithi012.stdout:0/727: chown d1/d20/d23/d82/ld9 25 1 2024-03-18T14:51:02.647 INFO:tasks.workunit.client.1.smithi100.stdout:6/322: getdents d6/d7/df/d26/d32/d48 0 2024-03-18T14:51:02.647 INFO:tasks.workunit.client.1.smithi100.stdout:6/323: write d6/d7/df/d26/f56 [696050,118848] 0 2024-03-18T14:51:02.649 INFO:tasks.workunit.client.0.smithi012.stdout:7/683: link dd/d13/d23/d2f/f9f dd/d13/d23/d60/fed 0 2024-03-18T14:51:02.649 INFO:tasks.workunit.client.1.smithi100.stdout:4/343: sync 2024-03-18T14:51:02.649 INFO:tasks.workunit.client.1.smithi100.stdout:1/356: read f5 [648079,17446] 0 2024-03-18T14:51:02.649 INFO:tasks.workunit.client.1.smithi100.stdout:0/445: rmdir dd/d31/d91/d7b 39 2024-03-18T14:51:02.653 INFO:tasks.workunit.client.1.smithi100.stdout:5/321: mkdir d29/d2d/d39/d60 0 2024-03-18T14:51:02.658 INFO:tasks.workunit.client.0.smithi012.stdout:8/856: mkdir d0/d5a/d5e/ddf/d50/d114 0 2024-03-18T14:51:02.658 INFO:tasks.workunit.client.0.smithi012.stdout:8/857: write d0/d47/fba [84248,126258] 0 2024-03-18T14:51:02.658 INFO:tasks.workunit.client.0.smithi012.stdout:8/858: write d0/d5a/f1d [598649,86397] 0 2024-03-18T14:51:02.659 INFO:tasks.workunit.client.1.smithi100.stdout:9/355: mkdir da/d46/d4d/d67 0 2024-03-18T14:51:02.663 INFO:tasks.workunit.client.1.smithi100.stdout:8/367: getdents d4/dc/dd 0 2024-03-18T14:51:02.663 INFO:tasks.workunit.client.1.smithi100.stdout:8/368: write d4/dc/dd/de/d21/d23/d47/f50 [5197938,87392] 0 2024-03-18T14:51:02.663 INFO:tasks.workunit.client.1.smithi100.stdout:8/369: truncate d4/dc/dd/d48/f5f 810188 0 2024-03-18T14:51:02.663 INFO:tasks.workunit.client.0.smithi012.stdout:3/639: rmdir dc/d30 39 2024-03-18T14:51:02.664 INFO:tasks.workunit.client.0.smithi012.stdout:3/640: write dc/d3c/d4c/f94 [616814,66476] 0 2024-03-18T14:51:02.664 INFO:tasks.workunit.client.1.smithi100.stdout:2/318: symlink d3/d13/l75 0 2024-03-18T14:51:02.665 INFO:tasks.workunit.client.0.smithi012.stdout:2/736: getdents d0/d6b 0 2024-03-18T14:51:02.667 INFO:tasks.workunit.client.1.smithi100.stdout:6/324: mknod d6/d10/d2b/d36/c62 0 2024-03-18T14:51:02.669 INFO:tasks.workunit.client.0.smithi012.stdout:8/859: creat d0/d5a/d5e/dd7/d84/d54/d62/daf/f115 x:0 0 0 2024-03-18T14:51:02.669 INFO:tasks.workunit.client.0.smithi012.stdout:8/860: read - d0/d5a/d19/fce zero size 2024-03-18T14:51:02.669 INFO:tasks.workunit.client.0.smithi012.stdout:8/861: dread - d0/d5a/d5e/dd7/fff zero size 2024-03-18T14:51:02.670 INFO:tasks.workunit.client.1.smithi100.stdout:4/344: getdents d9/d13/d39/d59 0 2024-03-18T14:51:02.671 INFO:tasks.workunit.client.0.smithi012.stdout:7/684: dwrite dd/d13/f72 [0,4194304] 0 2024-03-18T14:51:02.671 INFO:tasks.workunit.client.0.smithi012.stdout:7/685: dread - dd/d4f/fa9 zero size 2024-03-18T14:51:02.671 INFO:tasks.workunit.client.1.smithi100.stdout:1/357: dwrite d8/f52 [0,4194304] 0 2024-03-18T14:51:02.671 INFO:tasks.workunit.client.1.smithi100.stdout:1/358: chown d8/d30/f54 58866 1 2024-03-18T14:51:02.671 INFO:tasks.workunit.client.1.smithi100.stdout:1/359: chown d8/d30/d61 77 1 2024-03-18T14:51:02.674 INFO:tasks.workunit.client.0.smithi012.stdout:3/641: creat dc/d30/d3a/da0/fd1 x:0 0 0 2024-03-18T14:51:02.678 INFO:tasks.workunit.client.0.smithi012.stdout:0/728: truncate d1/d20/d7b/ff 1596576 0 2024-03-18T14:51:02.680 INFO:tasks.workunit.client.1.smithi100.stdout:5/322: mkdir d29/d2d/d39/d61 0 2024-03-18T14:51:02.680 INFO:tasks.workunit.client.1.smithi100.stdout:5/323: creat d29/d2d/f62 x:0 0 0 2024-03-18T14:51:02.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/862: symlink d0/d5a/d5e/ddf/d50/d114/l116 0 2024-03-18T14:51:02.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/863: read d0/d5a/d19/dad/f46 [1353447,102780] 0 2024-03-18T14:51:02.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/864: chown d0/d5a/d19/d63 4717979 1 2024-03-18T14:51:02.681 INFO:tasks.workunit.client.0.smithi012.stdout:8/865: fdatasync d0/d5a/d5e/dd7/d84/d2a/fa1 0 2024-03-18T14:51:02.682 INFO:tasks.workunit.client.1.smithi100.stdout:9/356: creat da/d46/d4d/d67/f68 x:0 0 0 2024-03-18T14:51:02.682 INFO:tasks.workunit.client.1.smithi100.stdout:9/357: chown da/d51/f52 190192435 1 2024-03-18T14:51:02.682 INFO:tasks.workunit.client.1.smithi100.stdout:9/358: dread - da/d51/d55/f5c zero size 2024-03-18T14:51:02.682 INFO:tasks.workunit.client.1.smithi100.stdout:9/359: creat da/d51/f69 x:0 0 0 2024-03-18T14:51:02.684 INFO:tasks.workunit.client.0.smithi012.stdout:7/686: rmdir dd/d13/d23/d60/d9c 39 2024-03-18T14:51:02.686 INFO:tasks.workunit.client.1.smithi100.stdout:4/345: dread d9/f3b [0,4194304] 0 2024-03-18T14:51:02.686 INFO:tasks.workunit.client.1.smithi100.stdout:2/319: mkdir d3/d1a/d76 0 2024-03-18T14:51:02.686 INFO:tasks.workunit.client.1.smithi100.stdout:2/320: fsync d3/d13/d40/d53/f66 0 2024-03-18T14:51:02.689 INFO:tasks.workunit.client.1.smithi100.stdout:8/370: dwrite d4/dc/dd/de/d21/d23/f28 [0,4194304] 0 2024-03-18T14:51:02.689 INFO:tasks.workunit.client.1.smithi100.stdout:1/360: rename d8/d30/f54 to d8/d1e/d11/d27/f64 0 2024-03-18T14:51:02.689 INFO:tasks.workunit.client.1.smithi100.stdout:1/361: read - d8/d1e/d11/f60 zero size 2024-03-18T14:51:02.690 INFO:tasks.workunit.client.0.smithi012.stdout:3/642: mknod dc/d30/d3a/cd2 0 2024-03-18T14:51:02.690 INFO:tasks.workunit.client.0.smithi012.stdout:3/643: write dc/d30/f6d [126357,76869] 0 2024-03-18T14:51:02.691 INFO:tasks.workunit.client.0.smithi012.stdout:0/729: mknod d1/d16/d31/cee 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:8/866: truncate d0/d5a/d5e/ddf/d50/fa5 1412501 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:8/867: fsync d0/d5a/d19/d63/dbe/f113 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:8/868: fsync d0/d5a/f25 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:8/869: write d0/d5a/d5e/dd7/d84/f112 [661201,68942] 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.0.smithi012.stdout:8/870: creat d0/d5a/d5e/dd7/d84/d54/d62/dd0/f117 x:0 0 0 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.1.smithi100.stdout:0/446: rmdir dd/d2b 39 2024-03-18T14:51:02.693 INFO:tasks.workunit.client.1.smithi100.stdout:0/447: write fa [831697,120625] 0 2024-03-18T14:51:02.694 INFO:tasks.workunit.client.1.smithi100.stdout:6/325: dwrite d6/d12/f3a [0,4194304] 0 2024-03-18T14:51:02.694 INFO:tasks.workunit.client.1.smithi100.stdout:6/326: write d6/d12/f5c [991002,102632] 0 2024-03-18T14:51:02.694 INFO:tasks.workunit.client.1.smithi100.stdout:6/327: stat d6/d7/f8 0 2024-03-18T14:51:02.695 INFO:tasks.workunit.client.0.smithi012.stdout:3/644: mkdir dc/d30/d3a/db7/dd3 0 2024-03-18T14:51:02.695 INFO:tasks.workunit.client.0.smithi012.stdout:0/730: write d1/d20/d7b/f81 [2878932,24743] 0 2024-03-18T14:51:02.698 INFO:tasks.workunit.client.1.smithi100.stdout:5/324: mknod d29/c63 0 2024-03-18T14:51:02.701 INFO:tasks.workunit.client.0.smithi012.stdout:3/645: creat dc/d3c/d59/d86/d9a/d64/fd4 x:0 0 0 2024-03-18T14:51:02.701 INFO:tasks.workunit.client.0.smithi012.stdout:3/646: stat dc/d3c/d4c/d57/c8a 0 2024-03-18T14:51:02.702 INFO:tasks.workunit.client.1.smithi100.stdout:9/360: symlink da/dd/d3b/l6a 0 2024-03-18T14:51:02.704 INFO:tasks.workunit.client.0.smithi012.stdout:3/647: creat dc/d3c/d59/d86/d9a/fd5 x:0 0 0 2024-03-18T14:51:02.706 INFO:tasks.workunit.client.1.smithi100.stdout:2/321: creat d3/da/de/d48/f77 x:0 0 0 2024-03-18T14:51:02.711 INFO:tasks.workunit.client.1.smithi100.stdout:1/362: symlink d8/dd/l65 0 2024-03-18T14:51:02.714 INFO:tasks.workunit.client.1.smithi100.stdout:0/448: symlink dd/d2b/la5 0 2024-03-18T14:51:02.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/328: mkdir d6/d10/d27/d63 0 2024-03-18T14:51:02.718 INFO:tasks.workunit.client.1.smithi100.stdout:5/325: symlink d29/d2d/d32/l64 0 2024-03-18T14:51:02.718 INFO:tasks.workunit.client.1.smithi100.stdout:5/326: fdatasync d29/d2d/f62 0 2024-03-18T14:51:02.720 INFO:tasks.workunit.client.1.smithi100.stdout:9/361: mkdir da/dd/d3c/d6b 0 2024-03-18T14:51:02.720 INFO:tasks.workunit.client.1.smithi100.stdout:4/346: rmdir d9/d13/d25 39 2024-03-18T14:51:02.722 INFO:tasks.workunit.client.0.smithi012.stdout:0/731: dwrite d1/d20/d23/d3b/d33/d51/fae [0,4194304] 0 2024-03-18T14:51:02.722 INFO:tasks.workunit.client.0.smithi012.stdout:0/732: creat d1/d16/d6e/fef x:0 0 0 2024-03-18T14:51:02.729 INFO:tasks.workunit.client.0.smithi012.stdout:0/733: getdents d1/d20/d23/d3b/d33/d51 0 2024-03-18T14:51:02.731 INFO:tasks.workunit.client.0.smithi012.stdout:0/734: symlink d1/d16/d6e/dcc/lf0 0 2024-03-18T14:51:02.732 INFO:tasks.workunit.client.1.smithi100.stdout:2/322: mkdir d3/da/d78 0 2024-03-18T14:51:02.733 INFO:tasks.workunit.client.1.smithi100.stdout:8/371: mknod d4/c7a 0 2024-03-18T14:51:02.733 INFO:tasks.workunit.client.1.smithi100.stdout:8/372: chown d4/f9 204 1 2024-03-18T14:51:02.735 INFO:tasks.workunit.client.0.smithi012.stdout:0/735: link d1/d20/d23/d7f/f97 d1/d16/d79/ff1 0 2024-03-18T14:51:02.737 INFO:tasks.workunit.client.1.smithi100.stdout:6/329: mkdir d6/d10/d27/d64 0 2024-03-18T14:51:02.737 INFO:tasks.workunit.client.1.smithi100.stdout:6/330: write d6/d10/f5b [762664,52375] 0 2024-03-18T14:51:02.737 INFO:tasks.workunit.client.1.smithi100.stdout:6/331: readlink d6/d7/df/d26/l28 0 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/362: unlink da/db/f36 0 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/363: readlink da/d46/d4d/l47 0 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/364: chown da/d46/f63 2 1 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/365: dread - da/dd/f42 zero size 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/366: fdatasync da/dd/f42 0 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/367: readlink da/l22 0 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/368: chown da/dd/d29/f2e 5945 1 2024-03-18T14:51:02.739 INFO:tasks.workunit.client.1.smithi100.stdout:4/347: symlink d9/d27/l67 0 2024-03-18T14:51:02.740 INFO:tasks.workunit.client.0.smithi012.stdout:0/736: getdents d1/d20/d23/d3b/d33 0 2024-03-18T14:51:02.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:02 smithi012 ceph-mon[29182]: pgmap v5: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:02.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:02 smithi012 ceph-mon[29182]: mgrmap e24: smithi100.rwnrcn(active, since 4s) 2024-03-18T14:51:02.742 INFO:tasks.workunit.client.1.smithi100.stdout:2/323: unlink d3/da/de/l15 0 2024-03-18T14:51:02.743 INFO:tasks.workunit.client.0.smithi012.stdout:0/737: symlink d1/d16/d19/dd0/lf2 0 2024-03-18T14:51:02.743 INFO:tasks.workunit.client.1.smithi100.stdout:1/363: dwrite d8/d1e/f28 [4194304,4194304] 0 2024-03-18T14:51:02.746 INFO:tasks.workunit.client.1.smithi100.stdout:9/369: rename l3 to da/dd/d3c/d4b/l6c 0 2024-03-18T14:51:02.751 INFO:tasks.workunit.client.1.smithi100.stdout:1/364: rmdir d8/d1e/d11 39 2024-03-18T14:51:02.751 INFO:tasks.workunit.client.1.smithi100.stdout:1/365: chown d8/dd/c1c 0 1 2024-03-18T14:51:02.751 INFO:tasks.workunit.client.1.smithi100.stdout:1/366: fsync d8/d2a/d37/d50/f5e 0 2024-03-18T14:51:02.752 INFO:tasks.workunit.client.0.smithi012.stdout:3/648: dwrite dc/d3c/d72/fc0 [0,4194304] 0 2024-03-18T14:51:02.752 INFO:tasks.workunit.client.1.smithi100.stdout:2/324: dread d3/da/de/d20/f24 [0,4194304] 0 2024-03-18T14:51:02.752 INFO:tasks.workunit.client.1.smithi100.stdout:0/449: dwrite dd/d2b/d95/d79/fa4 [0,4194304] 0 2024-03-18T14:51:02.757 INFO:tasks.workunit.client.1.smithi100.stdout:2/325: rename d3/d13/d4f/c62 to d3/d1a/d6c/c79 0 2024-03-18T14:51:02.757 INFO:tasks.workunit.client.1.smithi100.stdout:1/367: dread d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/368: fsync d8/d1e/f28 0 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:5/327: dwrite d29/f5a [0,4194304] 0 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/369: chown d8/d2a/d37/d50 54302032 1 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/370: read - d8/dd/f56 zero size 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:5/328: write d29/d2d/d39/f4e [262794,101090] 0 2024-03-18T14:51:02.758 INFO:tasks.workunit.client.1.smithi100.stdout:0/450: dread dd/f1b [0,4194304] 0 2024-03-18T14:51:02.761 INFO:tasks.workunit.client.1.smithi100.stdout:2/326: link d3/d13/d40/d53/f6f d3/d6/d29/d36/d49/f7a 0 2024-03-18T14:51:02.765 INFO:tasks.workunit.client.1.smithi100.stdout:1/371: stat l3 0 2024-03-18T14:51:02.765 INFO:tasks.workunit.client.1.smithi100.stdout:1/372: chown d8/d1e/c58 36561 1 2024-03-18T14:51:02.769 INFO:tasks.workunit.client.1.smithi100.stdout:1/373: rmdir d8/d30/d61 39 2024-03-18T14:51:02.769 INFO:tasks.workunit.client.1.smithi100.stdout:1/374: write d8/d30/f35 [6453,27402] 0 2024-03-18T14:51:02.771 INFO:tasks.workunit.client.1.smithi100.stdout:2/327: write d3/f4 [2950591,39914] 0 2024-03-18T14:51:02.771 INFO:tasks.workunit.client.1.smithi100.stdout:1/375: rename d8/d1e/d11/f55 to d8/d2a/d37/f66 0 2024-03-18T14:51:02.772 INFO:tasks.workunit.client.0.smithi012.stdout:8/871: truncate d0/d5a/d5e/ddf/d50/fa5 788229 0 2024-03-18T14:51:02.772 INFO:tasks.workunit.client.0.smithi012.stdout:8/872: chown d0/d5a/d5e/dd7/d84/d54/d9f/c107 12 1 2024-03-18T14:51:02.773 INFO:tasks.workunit.client.1.smithi100.stdout:2/328: mknod d3/da/c7b 0 2024-03-18T14:51:02.773 INFO:tasks.workunit.client.1.smithi100.stdout:2/329: readlink l1 0 2024-03-18T14:51:02.773 INFO:tasks.workunit.client.1.smithi100.stdout:2/330: stat d3/d6/l1f 0 2024-03-18T14:51:02.775 INFO:tasks.workunit.client.1.smithi100.stdout:1/376: symlink d8/d1e/l67 0 2024-03-18T14:51:02.775 INFO:tasks.workunit.client.1.smithi100.stdout:6/332: dwrite d6/d12/f3a [4194304,4194304] 0 2024-03-18T14:51:02.775 INFO:tasks.workunit.client.0.smithi012.stdout:8/873: symlink d0/d5a/d19/l118 0 2024-03-18T14:51:02.777 INFO:tasks.workunit.client.1.smithi100.stdout:1/377: mknod d8/d1e/d11/d27/d3f/c68 0 2024-03-18T14:51:02.778 INFO:tasks.workunit.client.1.smithi100.stdout:4/348: dwrite d9/f3b [0,4194304] 0 2024-03-18T14:51:02.779 INFO:tasks.workunit.client.1.smithi100.stdout:7/341: sync 2024-03-18T14:51:02.779 INFO:tasks.workunit.client.1.smithi100.stdout:3/370: sync 2024-03-18T14:51:02.779 INFO:tasks.workunit.client.1.smithi100.stdout:3/371: write d3/d16/f45 [165128,6528] 0 2024-03-18T14:51:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:8/874: dread d0/d5a/d5e/dd7/d84/d2a/d3e/f5f [0,4194304] 0 2024-03-18T14:51:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:8/875: write d0/d5a/d55/fe3 [4568071,78114] 0 2024-03-18T14:51:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/738: dwrite d1/d20/d4e/d54/d8a/fb9 [0,4194304] 0 2024-03-18T14:51:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/739: write d1/d20/d23/d26/fa4 [1762576,100831] 0 2024-03-18T14:51:02.780 INFO:tasks.workunit.client.0.smithi012.stdout:0/740: readlink d1/d20/d23/d7f/l8c 0 2024-03-18T14:51:02.782 INFO:tasks.workunit.client.1.smithi100.stdout:6/333: mkdir d6/d7/df/d26/d32/d65 0 2024-03-18T14:51:02.784 INFO:tasks.workunit.client.1.smithi100.stdout:1/378: creat d8/f69 x:0 0 0 2024-03-18T14:51:02.786 INFO:tasks.workunit.client.0.smithi012.stdout:3/649: dwrite dc/f43 [0,4194304] 0 2024-03-18T14:51:02.786 INFO:tasks.workunit.client.1.smithi100.stdout:5/329: dwrite f20 [0,4194304] 0 2024-03-18T14:51:02.788 INFO:tasks.workunit.client.0.smithi012.stdout:8/876: mkdir d0/d5a/d5e/dd7/d84/d2a/d3e/d119 0 2024-03-18T14:51:02.794 INFO:tasks.workunit.client.1.smithi100.stdout:4/349: symlink d9/d27/d62/l68 0 2024-03-18T14:51:02.794 INFO:tasks.workunit.client.1.smithi100.stdout:4/350: readlink d9/d17/l4f 0 2024-03-18T14:51:02.794 INFO:tasks.workunit.client.1.smithi100.stdout:3/372: mkdir d3/d16/d13/d17/d88 0 2024-03-18T14:51:02.794 INFO:tasks.workunit.client.1.smithi100.stdout:4/351: fsync d9/fe 0 2024-03-18T14:51:02.794 INFO:tasks.workunit.client.1.smithi100.stdout:3/373: creat d3/d16/d13/d1f/d7c/f89 x:0 0 0 2024-03-18T14:51:02.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:02 smithi100 ceph-mon[37807]: pgmap v5: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:02.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:02 smithi100 ceph-mon[37807]: mgrmap e24: smithi100.rwnrcn(active, since 4s) 2024-03-18T14:51:02.798 INFO:tasks.workunit.client.1.smithi100.stdout:2/331: dwrite d3/fc [4194304,4194304] 0 2024-03-18T14:51:02.799 INFO:tasks.workunit.client.1.smithi100.stdout:2/332: dread - d3/d6/f67 zero size 2024-03-18T14:51:02.799 INFO:tasks.workunit.client.1.smithi100.stdout:2/333: write d3/d6/d29/d36/d4a/f56 [1013161,44339] 0 2024-03-18T14:51:02.799 INFO:tasks.workunit.client.1.smithi100.stdout:2/334: fsync d3/da/de/f3c 0 2024-03-18T14:51:02.799 INFO:tasks.workunit.client.0.smithi012.stdout:0/741: creat d1/d20/d23/d3b/d33/d51/ff3 x:0 0 0 2024-03-18T14:51:02.799 INFO:tasks.workunit.client.0.smithi012.stdout:0/742: truncate d1/f9 1309359 0 2024-03-18T14:51:02.806 INFO:tasks.workunit.client.1.smithi100.stdout:4/352: dread d9/f1f [0,4194304] 0 2024-03-18T14:51:02.807 INFO:tasks.workunit.client.1.smithi100.stdout:1/379: symlink d8/d1e/d11/d27/d3e/l6a 0 2024-03-18T14:51:02.808 INFO:tasks.workunit.client.1.smithi100.stdout:4/353: dread d9/f1b [4194304,4194304] 0 2024-03-18T14:51:02.808 INFO:tasks.workunit.client.1.smithi100.stdout:4/354: dread - d9/d13/d25/f4d zero size 2024-03-18T14:51:02.808 INFO:tasks.workunit.client.1.smithi100.stdout:2/335: write d3/d6/d29/d36/f4e [342344,85076] 0 2024-03-18T14:51:02.809 INFO:tasks.workunit.client.0.smithi012.stdout:0/743: dread d1/d16/d31/f1d [0,4194304] 0 2024-03-18T14:51:02.809 INFO:tasks.workunit.client.0.smithi012.stdout:0/744: fdatasync d1/d16/fe 0 2024-03-18T14:51:02.817 INFO:tasks.workunit.client.1.smithi100.stdout:7/342: dwrite d5/d14/f40 [0,4194304] 0 2024-03-18T14:51:02.817 INFO:tasks.workunit.client.1.smithi100.stdout:7/343: stat d5/d22/d3b/d47 0 2024-03-18T14:51:02.817 INFO:tasks.workunit.client.1.smithi100.stdout:7/344: chown d5/d14/d16/d2e 21474 1 2024-03-18T14:51:02.819 INFO:tasks.workunit.client.1.smithi100.stdout:1/380: truncate d8/f52 3644067 0 2024-03-18T14:51:02.821 INFO:tasks.workunit.client.1.smithi100.stdout:4/355: rename f0 to d9/d3a/f69 0 2024-03-18T14:51:02.822 INFO:tasks.workunit.client.1.smithi100.stdout:6/334: dwrite d6/d7/f5f [0,4194304] 0 2024-03-18T14:51:02.822 INFO:tasks.workunit.client.1.smithi100.stdout:2/336: mkdir d3/d1a/d76/d7c 0 2024-03-18T14:51:02.827 INFO:tasks.workunit.client.0.smithi012.stdout:8/877: rmdir d0/d5a/d5e/ddf/d50/d6a 39 2024-03-18T14:51:02.827 INFO:tasks.workunit.client.0.smithi012.stdout:8/878: write d0/d5a/d19/dad/f3c [1734197,117937] 0 2024-03-18T14:51:02.828 INFO:tasks.workunit.client.0.smithi012.stdout:3/650: dwrite dc/d3c/d4c/faa [0,4194304] 0 2024-03-18T14:51:02.831 INFO:tasks.workunit.client.0.smithi012.stdout:0/745: rename d1/d20/d4e/d54/d67/fbf to d1/d20/d4e/d54/d67/ff4 0 2024-03-18T14:51:02.832 INFO:tasks.workunit.client.1.smithi100.stdout:7/345: creat d5/d14/d16/d21/f7f x:0 0 0 2024-03-18T14:51:02.834 INFO:tasks.workunit.client.0.smithi012.stdout:8/879: symlink d0/d5a/d5e/dd7/d84/d54/d62/l11a 0 2024-03-18T14:51:02.836 INFO:tasks.workunit.client.1.smithi100.stdout:5/330: dwrite d29/f37 [0,4194304] 0 2024-03-18T14:51:02.836 INFO:tasks.workunit.client.1.smithi100.stdout:5/331: chown l1d 123 1 2024-03-18T14:51:02.837 INFO:tasks.workunit.client.1.smithi100.stdout:4/356: rename f8 to d9/d27/d42/f6a 0 2024-03-18T14:51:02.837 INFO:tasks.workunit.client.1.smithi100.stdout:4/357: chown d9/d27/d42/f4a 33977 1 2024-03-18T14:51:02.837 INFO:tasks.workunit.client.1.smithi100.stdout:4/358: fdatasync d9/d13/d25/f4d 0 2024-03-18T14:51:02.838 INFO:tasks.workunit.client.0.smithi012.stdout:0/746: dread d1/d20/d23/d3b/d33/d51/fa8 [0,4194304] 0 2024-03-18T14:51:02.838 INFO:tasks.workunit.client.0.smithi012.stdout:0/747: creat d1/d16/d79/ff5 x:0 0 0 2024-03-18T14:51:02.841 INFO:tasks.workunit.client.1.smithi100.stdout:6/335: read d6/d7/f5f [2442969,112422] 0 2024-03-18T14:51:02.841 INFO:tasks.workunit.client.1.smithi100.stdout:3/374: dwrite d3/dd/d49/f5d [0,4194304] 0 2024-03-18T14:51:02.841 INFO:tasks.workunit.client.1.smithi100.stdout:3/375: chown d3/c12 429 1 2024-03-18T14:51:02.842 INFO:tasks.workunit.client.0.smithi012.stdout:3/651: readlink dc/l1c 0 2024-03-18T14:51:02.842 INFO:tasks.workunit.client.0.smithi012.stdout:3/652: write dc/d3c/f74 [659444,7237] 0 2024-03-18T14:51:02.843 INFO:tasks.workunit.client.1.smithi100.stdout:1/381: dwrite d8/d1e/f45 [0,4194304] 0 2024-03-18T14:51:02.844 INFO:tasks.workunit.client.1.smithi100.stdout:1/382: dread - d8/d1e/d11/f60 zero size 2024-03-18T14:51:02.845 INFO:tasks.workunit.client.1.smithi100.stdout:2/337: mkdir d3/d13/d40/d7d 0 2024-03-18T14:51:02.845 INFO:tasks.workunit.client.1.smithi100.stdout:2/338: stat d3/d1a/d69/f70 0 2024-03-18T14:51:02.846 INFO:tasks.workunit.client.1.smithi100.stdout:7/346: dread d5/d14/d16/d44/f76 [0,4194304] 0 2024-03-18T14:51:02.847 INFO:tasks.workunit.client.1.smithi100.stdout:6/336: read d6/d12/d2c/f4c [640560,59915] 0 2024-03-18T14:51:02.850 INFO:tasks.workunit.client.1.smithi100.stdout:1/383: dread d8/d2a/f47 [0,4194304] 0 2024-03-18T14:51:02.852 INFO:tasks.workunit.client.1.smithi100.stdout:4/359: unlink d9/d13/d5a/d5b/f53 0 2024-03-18T14:51:02.853 INFO:tasks.workunit.client.0.smithi012.stdout:3/653: creat dc/d30/fd6 x:0 0 0 2024-03-18T14:51:02.858 INFO:tasks.workunit.client.1.smithi100.stdout:3/376: truncate d3/dd/f11 3650433 0 2024-03-18T14:51:02.859 INFO:tasks.workunit.client.0.smithi012.stdout:3/654: mkdir dc/d3c/d59/d86/d9a/dd7 0 2024-03-18T14:51:02.861 INFO:tasks.workunit.client.0.smithi012.stdout:1/637: sync 2024-03-18T14:51:02.862 INFO:tasks.workunit.client.0.smithi012.stdout:6/626: sync 2024-03-18T14:51:02.862 INFO:tasks.workunit.client.0.smithi012.stdout:0/748: write d1/faa [732556,85220] 0 2024-03-18T14:51:02.867 INFO:tasks.workunit.client.0.smithi012.stdout:1/638: mknod d8/db/d10/d16/da1/db9/ccd 0 2024-03-18T14:51:02.867 INFO:tasks.workunit.client.1.smithi100.stdout:5/332: dwrite f17 [4194304,4194304] 0 2024-03-18T14:51:02.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/749: stat d1/d16/d19/dd0/cd6 0 2024-03-18T14:51:02.872 INFO:tasks.workunit.client.1.smithi100.stdout:3/377: dread d3/dd/d21/d3e/f79 [0,4194304] 0 2024-03-18T14:51:02.872 INFO:tasks.workunit.client.1.smithi100.stdout:4/360: chown d9/d13/f1e 607224096 1 2024-03-18T14:51:02.874 INFO:tasks.workunit.client.0.smithi012.stdout:6/627: dread d17/d19/f3e [0,4194304] 0 2024-03-18T14:51:02.874 INFO:tasks.workunit.client.1.smithi100.stdout:5/333: mkdir d29/d2d/d39/d65 0 2024-03-18T14:51:02.876 INFO:tasks.workunit.client.0.smithi012.stdout:6/628: truncate d17/d18/f9e 3008264 0 2024-03-18T14:51:02.876 INFO:tasks.workunit.client.0.smithi012.stdout:6/629: fsync d17/d26/d36/d4d/d82/f8b 0 2024-03-18T14:51:02.879 INFO:tasks.workunit.client.0.smithi012.stdout:6/630: creat d17/d19/d27/d6e/fc0 x:0 0 0 2024-03-18T14:51:02.879 INFO:tasks.workunit.client.0.smithi012.stdout:6/631: write d17/d26/d36/f7a [687673,18971] 0 2024-03-18T14:51:02.879 INFO:tasks.workunit.client.0.smithi012.stdout:6/632: stat d17/d33/l3a 0 2024-03-18T14:51:02.879 INFO:tasks.workunit.client.0.smithi012.stdout:6/633: write d17/d19/d27/d2f/f62 [4316730,83242] 0 2024-03-18T14:51:02.880 INFO:tasks.workunit.client.1.smithi100.stdout:3/378: creat d3/f8a x:0 0 0 2024-03-18T14:51:02.881 INFO:tasks.workunit.client.1.smithi100.stdout:6/337: rename d6/d10/d2b to d6/d7/df/d26/d4e/d66 0 2024-03-18T14:51:02.883 INFO:tasks.workunit.client.1.smithi100.stdout:5/334: rename d29/d2d/f62 to d29/d2d/d39/d65/f66 0 2024-03-18T14:51:02.883 INFO:tasks.workunit.client.1.smithi100.stdout:5/335: fsync d29/d2d/d39/f56 0 2024-03-18T14:51:02.883 INFO:tasks.workunit.client.1.smithi100.stdout:5/336: read d29/d2d/d39/f4c [1740003,58751] 0 2024-03-18T14:51:02.883 INFO:tasks.workunit.client.1.smithi100.stdout:5/337: chown d29/d2d/d39 183707 1 2024-03-18T14:51:02.883 INFO:tasks.workunit.client.1.smithi100.stdout:6/338: link d6/d7/df/l57 d6/d7/df/d26/d32/l67 0 2024-03-18T14:51:02.884 INFO:tasks.workunit.client.1.smithi100.stdout:3/379: getdents d3/dd/d21/d3e 0 2024-03-18T14:51:02.884 INFO:tasks.workunit.client.1.smithi100.stdout:3/380: write d3/d16/d13/d17/f7a [951627,119951] 0 2024-03-18T14:51:02.884 INFO:tasks.workunit.client.1.smithi100.stdout:4/361: read d9/d17/f34 [4352939,44624] 0 2024-03-18T14:51:02.884 INFO:tasks.workunit.client.1.smithi100.stdout:4/362: creat d9/d27/f6b x:0 0 0 2024-03-18T14:51:02.885 INFO:tasks.workunit.client.1.smithi100.stdout:3/381: mknod d3/d16/d13/d1f/d3f/c8b 0 2024-03-18T14:51:02.886 INFO:tasks.workunit.client.1.smithi100.stdout:5/338: getdents d29/d2d/d32/d38 0 2024-03-18T14:51:02.888 INFO:tasks.workunit.client.1.smithi100.stdout:3/382: mknod d3/d16/d13/d1f/d7c/c8c 0 2024-03-18T14:51:02.888 INFO:tasks.workunit.client.1.smithi100.stdout:4/363: creat d9/d27/d62/f6c x:0 0 0 2024-03-18T14:51:02.889 INFO:tasks.workunit.client.1.smithi100.stdout:2/339: dwrite d3/da/f43 [0,4194304] 0 2024-03-18T14:51:02.890 INFO:tasks.workunit.client.0.smithi012.stdout:8/880: dwrite d0/d47/d59/dc9/ffe [0,4194304] 0 2024-03-18T14:51:02.891 INFO:tasks.workunit.client.1.smithi100.stdout:4/364: link d9/d27/f2e d9/f6d 0 2024-03-18T14:51:02.893 INFO:tasks.workunit.client.1.smithi100.stdout:6/339: write d6/d12/f3a [108743,97612] 0 2024-03-18T14:51:02.894 INFO:tasks.workunit.client.1.smithi100.stdout:1/384: dwrite d8/d2a/f44 [0,4194304] 0 2024-03-18T14:51:02.894 INFO:tasks.workunit.client.1.smithi100.stdout:1/385: fdatasync d8/f69 0 2024-03-18T14:51:02.896 INFO:tasks.workunit.client.0.smithi012.stdout:6/634: dread d17/f1b [0,4194304] 0 2024-03-18T14:51:02.896 INFO:tasks.workunit.client.0.smithi012.stdout:6/635: chown d17/d19/d27/db1 4237691 1 2024-03-18T14:51:02.896 INFO:tasks.workunit.client.0.smithi012.stdout:3/655: dwrite dc/d30/f98 [0,4194304] 0 2024-03-18T14:51:02.897 INFO:tasks.workunit.client.1.smithi100.stdout:4/365: chown d9/d13/l30 71 1 2024-03-18T14:51:02.901 INFO:tasks.workunit.client.1.smithi100.stdout:6/340: rename d6/d7/f3d to d6/d7/df/d26/d4e/d66/f68 0 2024-03-18T14:51:02.911 INFO:tasks.workunit.client.1.smithi100.stdout:6/341: unlink d6/d7/df/l61 0 2024-03-18T14:51:02.911 INFO:tasks.workunit.client.1.smithi100.stdout:6/342: write d6/d7/df/d26/f56 [736032,107563] 0 2024-03-18T14:51:02.913 INFO:tasks.workunit.client.1.smithi100.stdout:1/386: write d8/d1e/f45 [2308306,97307] 0 2024-03-18T14:51:02.913 INFO:tasks.workunit.client.1.smithi100.stdout:1/387: fdatasync d8/d1e/d11/f21 0 2024-03-18T14:51:02.913 INFO:tasks.workunit.client.0.smithi012.stdout:8/881: dwrite d0/d5a/d55/fef [0,4194304] 0 2024-03-18T14:51:02.913 INFO:tasks.workunit.client.0.smithi012.stdout:8/882: chown d0/d15/d4e/ffa 98199238 1 2024-03-18T14:51:02.914 INFO:tasks.workunit.client.0.smithi012.stdout:8/883: dread - d0/d5a/d5e/dd7/d84/d54/d62/dd0/fd5 zero size 2024-03-18T14:51:02.914 INFO:tasks.workunit.client.0.smithi012.stdout:8/884: dread - d0/d5a/d5e/ddf/d50/d6a/fe4 zero size 2024-03-18T14:51:02.914 INFO:tasks.workunit.client.1.smithi100.stdout:4/366: read d9/d17/f24 [2764004,128496] 0 2024-03-18T14:51:02.915 INFO:tasks.workunit.client.1.smithi100.stdout:8/373: sync 2024-03-18T14:51:02.916 INFO:tasks.workunit.client.1.smithi100.stdout:6/343: rename d6/d7/l41 to d6/d7/df/d26/d32/d48/l69 0 2024-03-18T14:51:02.917 INFO:tasks.workunit.client.0.smithi012.stdout:6/636: truncate f11 2677129 0 2024-03-18T14:51:02.918 INFO:tasks.workunit.client.0.smithi012.stdout:6/637: chown d17/d18/db8/f55 24 1 2024-03-18T14:51:02.918 INFO:tasks.workunit.client.0.smithi012.stdout:6/638: dread - d17/d26/fb3 zero size 2024-03-18T14:51:02.918 INFO:tasks.workunit.client.0.smithi012.stdout:1/639: write d8/d1e/d27/f3c [801293,1492] 0 2024-03-18T14:51:02.918 INFO:tasks.workunit.client.0.smithi012.stdout:1/640: chown d8/db/d10 3860787 1 2024-03-18T14:51:02.920 INFO:tasks.workunit.client.1.smithi100.stdout:3/383: dwrite d3/dd/d21/d3e/f32 [0,4194304] 0 2024-03-18T14:51:02.921 INFO:tasks.workunit.client.0.smithi012.stdout:3/656: unlink dc/d3c/d59/d86/d9a/fa7 0 2024-03-18T14:51:02.921 INFO:tasks.workunit.client.0.smithi012.stdout:3/657: readlink dc/l1e 0 2024-03-18T14:51:02.923 INFO:tasks.workunit.client.1.smithi100.stdout:4/367: mknod d9/c6e 0 2024-03-18T14:51:02.923 INFO:tasks.workunit.client.1.smithi100.stdout:4/368: dread - d9/d13/d25/d51/f5d zero size 2024-03-18T14:51:02.926 INFO:tasks.workunit.client.1.smithi100.stdout:8/374: rename d4/dc/dd/de/f17 to d4/dc/f7b 0 2024-03-18T14:51:02.927 INFO:tasks.workunit.client.1.smithi100.stdout:8/375: read d4/dc/d1f/d33/f36 [392919,98656] 0 2024-03-18T14:51:02.929 INFO:tasks.workunit.client.1.smithi100.stdout:6/344: rmdir d6/d7/df/d26/d4e/d66/d36 39 2024-03-18T14:51:02.929 INFO:tasks.workunit.client.1.smithi100.stdout:8/376: read d4/dc/d31/f3d [555360,74573] 0 2024-03-18T14:51:02.929 INFO:tasks.workunit.client.1.smithi100.stdout:9/370: sync 2024-03-18T14:51:02.929 INFO:tasks.workunit.client.1.smithi100.stdout:0/451: sync 2024-03-18T14:51:02.930 INFO:tasks.workunit.client.0.smithi012.stdout:8/885: mknod d0/d5a/d55/c11b 0 2024-03-18T14:51:02.930 INFO:tasks.workunit.client.0.smithi012.stdout:8/886: chown d0/d5a/f1d 83665751 1 2024-03-18T14:51:02.932 INFO:tasks.workunit.client.0.smithi012.stdout:8/887: dread d0/d5a/f1d [0,4194304] 0 2024-03-18T14:51:02.933 INFO:tasks.workunit.client.0.smithi012.stdout:8/888: write d0/f6 [2866028,62277] 0 2024-03-18T14:51:02.937 INFO:tasks.workunit.client.0.smithi012.stdout:1/641: creat d8/db/d10/d31/d43/d5e/d65/d70/dc7/fce x:0 0 0 2024-03-18T14:51:02.938 INFO:tasks.workunit.client.0.smithi012.stdout:3/658: mkdir dc/d30/d3a/da0/dc5/dc7/dd8 0 2024-03-18T14:51:02.939 INFO:tasks.workunit.client.1.smithi100.stdout:6/345: write d6/d7/f59 [3185854,106742] 0 2024-03-18T14:51:02.939 INFO:tasks.workunit.client.1.smithi100.stdout:6/346: fdatasync d6/d12/f3b 0 2024-03-18T14:51:02.940 INFO:tasks.workunit.client.1.smithi100.stdout:6/347: chown d6/d10/f13 1980115534 1 2024-03-18T14:51:02.940 INFO:tasks.workunit.client.1.smithi100.stdout:0/452: dread f3 [0,4194304] 0 2024-03-18T14:51:02.940 INFO:tasks.workunit.client.1.smithi100.stdout:0/453: creat dd/fa6 x:0 0 0 2024-03-18T14:51:02.940 INFO:tasks.workunit.client.1.smithi100.stdout:8/377: mkdir d4/dc/d1f/d59/d6c/d7c 0 2024-03-18T14:51:02.940 INFO:tasks.workunit.client.1.smithi100.stdout:8/378: write d4/dc/d1f/d33/f36 [628510,63913] 0 2024-03-18T14:51:02.941 INFO:tasks.workunit.client.1.smithi100.stdout:9/371: mknod da/dd/d3b/c6d 0 2024-03-18T14:51:02.942 INFO:tasks.workunit.client.0.smithi012.stdout:8/889: mknod d0/d5a/c11c 0 2024-03-18T14:51:02.943 INFO:tasks.workunit.client.0.smithi012.stdout:8/890: truncate d0/d5a/d5e/f82 493937 0 2024-03-18T14:51:02.943 INFO:tasks.workunit.client.0.smithi012.stdout:8/891: chown d0/d5a/d5e/dd7/d84/d54/ff6 9543214 1 2024-03-18T14:51:02.943 INFO:tasks.workunit.client.0.smithi012.stdout:8/892: dread - d0/d5a/d19/fce zero size 2024-03-18T14:51:02.944 INFO:tasks.workunit.client.1.smithi100.stdout:3/384: rmdir d3/d16/d13/d1f/d7c 39 2024-03-18T14:51:02.947 INFO:tasks.workunit.client.0.smithi012.stdout:6/639: dwrite d17/d26/fb3 [0,4194304] 0 2024-03-18T14:51:02.947 INFO:tasks.workunit.client.0.smithi012.stdout:6/640: fdatasync d17/d38/d51/f53 0 2024-03-18T14:51:02.947 INFO:tasks.workunit.client.0.smithi012.stdout:1/642: creat d8/db/d10/d31/d43/d5e/d65/d84/fcf x:0 0 0 2024-03-18T14:51:02.949 INFO:tasks.workunit.client.1.smithi100.stdout:4/369: dwrite d9/d17/f34 [4194304,4194304] 0 2024-03-18T14:51:02.949 INFO:tasks.workunit.client.1.smithi100.stdout:6/348: mknod d6/d7/df/d26/d32/c6a 0 2024-03-18T14:51:02.952 INFO:tasks.workunit.client.0.smithi012.stdout:3/659: creat dc/d30/d3a/fd9 x:0 0 0 2024-03-18T14:51:02.956 INFO:tasks.workunit.client.0.smithi012.stdout:8/893: rename d0/d5a/d5e/dd7/d84/d2a/l111 to d0/d5a/l11d 0 2024-03-18T14:51:02.956 INFO:tasks.workunit.client.0.smithi012.stdout:8/894: write d0/d47/d59/d67/fda [1201164,5478] 0 2024-03-18T14:51:02.956 INFO:tasks.workunit.client.0.smithi012.stdout:8/895: readlink d0/d5a/d5e/ddf/d50/d114/l116 0 2024-03-18T14:51:02.956 INFO:tasks.workunit.client.0.smithi012.stdout:8/896: chown d0/d5a/d19/d63/dbe/ce7 160589 1 2024-03-18T14:51:02.957 INFO:tasks.workunit.client.0.smithi012.stdout:6/641: dread d17/f1a [0,4194304] 0 2024-03-18T14:51:02.959 INFO:tasks.workunit.client.1.smithi100.stdout:8/379: symlink d4/dc/dd/de/d19/l7d 0 2024-03-18T14:51:02.960 INFO:tasks.workunit.client.0.smithi012.stdout:6/642: dread d17/d26/d36/d4d/f81 [0,4194304] 0 2024-03-18T14:51:02.961 INFO:tasks.workunit.client.1.smithi100.stdout:3/385: dwrite d3/dd/d49/f70 [0,4194304] 0 2024-03-18T14:51:02.962 INFO:tasks.workunit.client.0.smithi012.stdout:9/732: sync 2024-03-18T14:51:02.963 INFO:tasks.workunit.client.0.smithi012.stdout:1/643: unlink d8/db/d10/d31/d43/d5e/c64 0 2024-03-18T14:51:02.964 INFO:tasks.workunit.client.0.smithi012.stdout:1/644: fsync d8/d1e/d27/d44/f59 0 2024-03-18T14:51:02.964 INFO:tasks.workunit.client.0.smithi012.stdout:1/645: chown d8/db/d10/d31/d43/d5e 6258807 1 2024-03-18T14:51:02.966 INFO:tasks.workunit.client.1.smithi100.stdout:6/349: rename d6/d7/df/c3f to d6/d7/df/c6b 0 2024-03-18T14:51:02.966 INFO:tasks.workunit.client.1.smithi100.stdout:6/350: chown d6/d7/df/d26/d4e/d53 52379937 1 2024-03-18T14:51:02.968 INFO:tasks.workunit.client.1.smithi100.stdout:3/386: dread d3/d16/d13/d17/f4c [0,4194304] 0 2024-03-18T14:51:02.968 INFO:tasks.workunit.client.1.smithi100.stdout:3/387: stat d3/d16/d13/d1f/l77 0 2024-03-18T14:51:02.968 INFO:tasks.workunit.client.1.smithi100.stdout:3/388: chown d3/d16/d13/d1f/d37/c48 17378 1 2024-03-18T14:51:02.974 INFO:tasks.workunit.client.0.smithi012.stdout:4/611: sync 2024-03-18T14:51:02.974 INFO:tasks.workunit.client.0.smithi012.stdout:4/612: write d6/d19/f3e [1561995,5820] 0 2024-03-18T14:51:02.974 INFO:tasks.workunit.client.0.smithi012.stdout:4/613: chown d6/d1a/d30/c9d 0 1 2024-03-18T14:51:02.975 INFO:tasks.workunit.client.1.smithi100.stdout:8/380: dwrite d4/dc/dd/de/f57 [0,4194304] 0 2024-03-18T14:51:02.975 INFO:tasks.workunit.client.1.smithi100.stdout:8/381: getdents d4/dc/d31/d66 0 2024-03-18T14:51:02.975 INFO:tasks.workunit.client.1.smithi100.stdout:3/389: mknod d3/d16/d13/d1f/d3f/d4f/c8d 0 2024-03-18T14:51:02.977 INFO:tasks.workunit.client.0.smithi012.stdout:3/660: creat dc/d3c/fda x:0 0 0 2024-03-18T14:51:02.977 INFO:tasks.workunit.client.0.smithi012.stdout:3/661: chown dc/d3c/d4c 18 1 2024-03-18T14:51:02.977 INFO:tasks.workunit.client.0.smithi012.stdout:3/662: write dc/d30/dab/fb8 [1010779,83035] 0 2024-03-18T14:51:02.981 INFO:tasks.workunit.client.0.smithi012.stdout:6/643: rename d17/d26/d36/d4d/da3 to d17/d18/dc1 0 2024-03-18T14:51:02.984 INFO:tasks.workunit.client.0.smithi012.stdout:9/733: mkdir d3/de6/dbf/d77/d52/d99/da0/ddf/dee 0 2024-03-18T14:51:02.984 INFO:tasks.workunit.client.0.smithi012.stdout:9/734: chown d3/de6/dbf/d66/d69/d7f/d8c/lbe 313892 1 2024-03-18T14:51:02.987 INFO:tasks.workunit.client.0.smithi012.stdout:1/646: mknod d8/db/d10/cd0 0 2024-03-18T14:51:02.989 INFO:tasks.workunit.client.1.smithi100.stdout:8/382: link d4/dc/d1f/d33/c52 d4/dc/dd/de/c7e 0 2024-03-18T14:51:02.989 INFO:tasks.workunit.client.1.smithi100.stdout:8/383: chown d4/dc/dd/de/f57 806 1 2024-03-18T14:51:02.989 INFO:tasks.workunit.client.1.smithi100.stdout:8/384: chown d4/dc/dd/de/d21/d23/d25/f71 21603759 1 2024-03-18T14:51:02.989 INFO:tasks.workunit.client.1.smithi100.stdout:8/385: fdatasync d4/dc/d1f/d33/f4e 0 2024-03-18T14:51:02.992 INFO:tasks.workunit.client.1.smithi100.stdout:3/390: rename d3/d16/l18 to d3/d16/d13/d1f/d3f/d4f/d60/l8e 0 2024-03-18T14:51:02.993 INFO:tasks.workunit.client.1.smithi100.stdout:3/391: stat d3/d16/d13/d1f/d3f/d4f/d60/f6b 0 2024-03-18T14:51:02.995 INFO:tasks.workunit.client.1.smithi100.stdout:0/454: write dd/d2b/f3d [475867,101082] 0 2024-03-18T14:51:02.997 INFO:tasks.workunit.client.1.smithi100.stdout:3/392: rename d3/d16/f62 to d3/dd/d49/f8f 0 2024-03-18T14:51:03.000 INFO:tasks.workunit.client.1.smithi100.stdout:8/386: getdents d4/dc/d1f/d33 0 2024-03-18T14:51:03.001 INFO:tasks.workunit.client.0.smithi012.stdout:3/663: truncate dc/d30/d35/f75 506108 0 2024-03-18T14:51:03.001 INFO:tasks.workunit.client.0.smithi012.stdout:3/664: truncate dc/d91/fbb 1346192 0 2024-03-18T14:51:03.003 INFO:tasks.workunit.client.1.smithi100.stdout:0/455: mknod dd/d31/d91/d56/d62/ca7 0 2024-03-18T14:51:03.003 INFO:tasks.workunit.client.1.smithi100.stdout:0/456: write dd/d20/f5f [4709101,90209] 0 2024-03-18T14:51:03.003 INFO:tasks.workunit.client.1.smithi100.stdout:0/457: stat dd/d20/d44/l4b 0 2024-03-18T14:51:03.006 INFO:tasks.workunit.client.0.smithi012.stdout:8/897: getdents d0/d5a/d5e/dd7/d84 0 2024-03-18T14:51:03.008 INFO:tasks.workunit.client.1.smithi100.stdout:8/387: link d4/dc/dd/de/d21/d23/f78 d4/dc/dd/de/d19/d37/f7f 0 2024-03-18T14:51:03.011 INFO:tasks.workunit.client.0.smithi012.stdout:4/614: rename d6/d19/fa0 to d6/d1a/d64/fcf 0 2024-03-18T14:51:03.011 INFO:tasks.workunit.client.0.smithi012.stdout:4/615: readlink d6/da/d89/da1/la3 0 2024-03-18T14:51:03.013 INFO:tasks.workunit.client.0.smithi012.stdout:5/681: sync 2024-03-18T14:51:03.013 INFO:tasks.workunit.client.0.smithi012.stdout:7/687: sync 2024-03-18T14:51:03.013 INFO:tasks.workunit.client.0.smithi012.stdout:2/737: sync 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:5/682: readlink d3/d29/d35/l81 0 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:5/683: fdatasync d3/d84/f85 0 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:7/688: write dd/d13/d23/fce [600911,25823] 0 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:7/689: chown dd/f45 30000 1 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:4/616: dread d6/da/d58/f8d [0,4194304] 0 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.0.smithi012.stdout:6/644: link d17/d18/dc1/lb5 d17/d19/d27/d6e/d84/lc2 0 2024-03-18T14:51:03.014 INFO:tasks.workunit.client.1.smithi100.stdout:0/458: dread f3 [0,4194304] 0 2024-03-18T14:51:03.015 INFO:tasks.workunit.client.0.smithi012.stdout:8/898: dread d0/d5a/d19/dad/f46 [0,4194304] 0 2024-03-18T14:51:03.016 INFO:tasks.workunit.client.0.smithi012.stdout:1/647: mkdir d8/db/d10/d16/d6b/dd1 0 2024-03-18T14:51:03.019 INFO:tasks.workunit.client.1.smithi100.stdout:8/388: unlink d4/c7a 0 2024-03-18T14:51:03.024 INFO:tasks.workunit.client.1.smithi100.stdout:8/389: dread d4/dc/d1f/d59/f64 [0,4194304] 0 2024-03-18T14:51:03.024 INFO:tasks.workunit.client.0.smithi012.stdout:9/735: rename d3/de6/dbf/d66/d21/d35/d48/l91 to d3/de6/dbf/d66/d69/d7f/lef 0 2024-03-18T14:51:03.024 INFO:tasks.workunit.client.0.smithi012.stdout:9/736: write d3/f6 [4544676,98201] 0 2024-03-18T14:51:03.024 INFO:tasks.workunit.client.0.smithi012.stdout:8/899: dread d0/d5a/d19/d57/f69 [0,4194304] 0 2024-03-18T14:51:03.026 INFO:tasks.workunit.client.1.smithi100.stdout:0/459: rename dd/f63 to dd/d2b/d95/d79/d7d/fa8 0 2024-03-18T14:51:03.026 INFO:tasks.workunit.client.1.smithi100.stdout:7/347: sync 2024-03-18T14:51:03.027 INFO:tasks.workunit.client.0.smithi012.stdout:2/738: creat d0/d10/fe8 x:0 0 0 2024-03-18T14:51:03.027 INFO:tasks.workunit.client.0.smithi012.stdout:2/739: truncate d0/d6b/dbf/d4f/fb1 1987986 0 2024-03-18T14:51:03.028 INFO:tasks.workunit.client.1.smithi100.stdout:3/393: dwrite d3/dd/d21/d3e/f79 [0,4194304] 0 2024-03-18T14:51:03.029 INFO:tasks.workunit.client.1.smithi100.stdout:8/390: creat d4/dc/d1f/d59/d6c/f80 x:0 0 0 2024-03-18T14:51:03.029 INFO:tasks.workunit.client.1.smithi100.stdout:8/391: truncate d4/dc/d1f/d59/d6c/f80 861767 0 2024-03-18T14:51:03.031 INFO:tasks.workunit.client.0.smithi012.stdout:5/684: mkdir d3/d29/d36/d9b/ddf 0 2024-03-18T14:51:03.031 INFO:tasks.workunit.client.0.smithi012.stdout:5/685: truncate d3/f9e 1716066 0 2024-03-18T14:51:03.033 INFO:tasks.workunit.client.0.smithi012.stdout:9/737: write d3/de6/dbf/d77/d52/d78/f7e [2951555,127789] 0 2024-03-18T14:51:03.035 INFO:tasks.workunit.client.0.smithi012.stdout:6/645: creat d17/d19/d27/d58/d91/fc3 x:0 0 0 2024-03-18T14:51:03.035 INFO:tasks.workunit.client.0.smithi012.stdout:6/646: chown d17/d26/l69 310790637 1 2024-03-18T14:51:03.037 INFO:tasks.workunit.client.0.smithi012.stdout:3/665: rename dc/d3c/d4c/d57/lc3 to dc/d3c/d59/d86/ldb 0 2024-03-18T14:51:03.037 INFO:tasks.workunit.client.0.smithi012.stdout:3/666: creat dc/d3c/d59/d86/fdc x:0 0 0 2024-03-18T14:51:03.037 INFO:tasks.workunit.client.0.smithi012.stdout:0/750: sync 2024-03-18T14:51:03.038 INFO:tasks.workunit.client.0.smithi012.stdout:8/900: rmdir d0/d5a/d5e/dd7/d84/d54/d62 39 2024-03-18T14:51:03.038 INFO:tasks.workunit.client.0.smithi012.stdout:8/901: readlink d0/lae 0 2024-03-18T14:51:03.040 INFO:tasks.workunit.client.1.smithi100.stdout:3/394: rmdir d3/d16/d13/d1f/d3f/d4f/d60 39 2024-03-18T14:51:03.040 INFO:tasks.workunit.client.1.smithi100.stdout:7/348: dread d5/d22/d3b/d47/f49 [0,4194304] 0 2024-03-18T14:51:03.040 INFO:tasks.workunit.client.1.smithi100.stdout:7/349: creat d5/d30/d4d/d5a/f80 x:0 0 0 2024-03-18T14:51:03.040 INFO:tasks.workunit.client.1.smithi100.stdout:7/350: chown d5/d14/d16/d2e/d62/d66/d4f 222 1 2024-03-18T14:51:03.041 INFO:tasks.workunit.client.1.smithi100.stdout:7/351: chown d5/d14/d16/d21/f50 403 1 2024-03-18T14:51:03.041 INFO:tasks.workunit.client.0.smithi012.stdout:2/740: truncate d0/d6b/dbf/f3a 2008283 0 2024-03-18T14:51:03.044 INFO:tasks.workunit.client.1.smithi100.stdout:8/392: unlink d4/l8 0 2024-03-18T14:51:03.044 INFO:tasks.workunit.client.1.smithi100.stdout:8/393: fdatasync d4/dc/dd/de/d21/d23/f28 0 2024-03-18T14:51:03.044 INFO:tasks.workunit.client.1.smithi100.stdout:8/394: fdatasync d4/dc/d1f/d33/f36 0 2024-03-18T14:51:03.045 INFO:tasks.workunit.client.0.smithi012.stdout:7/690: dwrite dd/d13/d23/d76/d6f/db3/dc7/fd8 [0,4194304] 0 2024-03-18T14:51:03.045 INFO:tasks.workunit.client.0.smithi012.stdout:7/691: chown dd/d13/d23/d76/d6b/fe8 82492 1 2024-03-18T14:51:03.045 INFO:tasks.workunit.client.0.smithi012.stdout:4/617: getdents d6/d19/d6c/d96 0 2024-03-18T14:51:03.047 INFO:tasks.workunit.client.1.smithi100.stdout:0/460: getdents dd/d31 0 2024-03-18T14:51:03.047 INFO:tasks.workunit.client.1.smithi100.stdout:0/461: chown dd/d31/d91/d56 7 1 2024-03-18T14:51:03.050 INFO:tasks.workunit.client.0.smithi012.stdout:6/647: dread f16 [0,4194304] 0 2024-03-18T14:51:03.051 INFO:tasks.workunit.client.0.smithi012.stdout:9/738: getdents d3/de6/dbf/d66/d21 0 2024-03-18T14:51:03.052 INFO:tasks.workunit.client.0.smithi012.stdout:1/648: dwrite d8/db/d10/d16/d54/fa6 [0,4194304] 0 2024-03-18T14:51:03.052 INFO:tasks.workunit.client.0.smithi012.stdout:1/649: truncate d8/db/d10/d16/f98 378954 0 2024-03-18T14:51:03.052 INFO:tasks.workunit.client.0.smithi012.stdout:5/686: dwrite d3/d84/f85 [0,4194304] 0 2024-03-18T14:51:03.053 INFO:tasks.workunit.client.0.smithi012.stdout:2/741: dread d0/d10/f13 [0,4194304] 0 2024-03-18T14:51:03.053 INFO:tasks.workunit.client.0.smithi012.stdout:2/742: creat d0/d6b/dbf/d33/d80/fe9 x:0 0 0 2024-03-18T14:51:03.058 INFO:tasks.workunit.client.1.smithi100.stdout:7/352: link d5/d14/d16/c3a d5/d22/c81 0 2024-03-18T14:51:03.058 INFO:tasks.workunit.client.1.smithi100.stdout:7/353: dread - d5/d14/f77 zero size 2024-03-18T14:51:03.058 INFO:tasks.workunit.client.1.smithi100.stdout:7/354: creat d5/d30/d4d/d5a/f82 x:0 0 0 2024-03-18T14:51:03.061 INFO:tasks.workunit.client.0.smithi012.stdout:3/667: rename dc/f44 to dc/d3c/d59/fdd 0 2024-03-18T14:51:03.061 INFO:tasks.workunit.client.0.smithi012.stdout:3/668: truncate dc/d3c/d59/fc8 471349 0 2024-03-18T14:51:03.062 INFO:tasks.workunit.client.1.smithi100.stdout:8/395: mknod d4/dc/dd/de/d21/d23/d25/c81 0 2024-03-18T14:51:03.065 INFO:tasks.workunit.client.0.smithi012.stdout:6/648: dread d17/f35 [0,4194304] 0 2024-03-18T14:51:03.067 INFO:tasks.workunit.client.0.smithi012.stdout:0/751: creat d1/d20/d23/d3b/ff6 x:0 0 0 2024-03-18T14:51:03.067 INFO:tasks.workunit.client.0.smithi012.stdout:0/752: readlink d1/d20/d4e/le4 0 2024-03-18T14:51:03.070 INFO:tasks.workunit.client.0.smithi012.stdout:5/687: dread d3/d15/f1c [0,4194304] 0 2024-03-18T14:51:03.070 INFO:tasks.workunit.client.1.smithi100.stdout:0/462: mkdir dd/d2b/d95/d79/d7d/da9 0 2024-03-18T14:51:03.071 INFO:tasks.workunit.client.1.smithi100.stdout:0/463: write dd/f43 [934113,105398] 0 2024-03-18T14:51:03.073 INFO:tasks.workunit.client.1.smithi100.stdout:3/395: dwrite d3/d16/d13/d17/f7f [0,4194304] 0 2024-03-18T14:51:03.074 INFO:tasks.workunit.client.0.smithi012.stdout:9/739: dwrite d3/de6/dbf/d66/d69/d7f/d8c/d8b/f9f [0,4194304] 0 2024-03-18T14:51:03.074 INFO:tasks.workunit.client.0.smithi012.stdout:8/902: link d0/d15/f28 d0/d5a/d5e/dd7/d84/d2a/f11e 0 2024-03-18T14:51:03.078 INFO:tasks.workunit.client.0.smithi012.stdout:2/743: write d0/d6b/dbf/d33/d71/d90/fae [3025108,107325] 0 2024-03-18T14:51:03.078 INFO:tasks.workunit.client.0.smithi012.stdout:2/744: stat d0/d3c/d40/f5e 0 2024-03-18T14:51:03.078 INFO:tasks.workunit.client.0.smithi012.stdout:2/745: write d0/da2/f67 [4048187,81562] 0 2024-03-18T14:51:03.078 INFO:tasks.workunit.client.1.smithi100.stdout:5/339: sync 2024-03-18T14:51:03.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/340: sync 2024-03-18T14:51:03.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/341: write d3/f10 [291842,13664] 0 2024-03-18T14:51:03.079 INFO:tasks.workunit.client.1.smithi100.stdout:5/340: write d29/d2d/d32/f5c [140636,95780] 0 2024-03-18T14:51:03.079 INFO:tasks.workunit.client.1.smithi100.stdout:7/355: mknod d5/c83 0 2024-03-18T14:51:03.082 INFO:tasks.workunit.client.1.smithi100.stdout:0/464: rename dd/d2b/l36 to dd/d31/d91/d56/d59/laa 0 2024-03-18T14:51:03.086 INFO:tasks.workunit.client.0.smithi012.stdout:7/692: symlink dd/d13/d23/d76/d6b/lee 0 2024-03-18T14:51:03.086 INFO:tasks.workunit.client.0.smithi012.stdout:7/693: chown dd/f45 15 1 2024-03-18T14:51:03.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/618: symlink d6/da/d89/d95/ld0 0 2024-03-18T14:51:03.089 INFO:tasks.workunit.client.1.smithi100.stdout:5/341: read d29/f2b [821368,7555] 0 2024-03-18T14:51:03.091 INFO:tasks.workunit.client.0.smithi012.stdout:2/746: write d0/d10/f6a [1880316,81701] 0 2024-03-18T14:51:03.091 INFO:tasks.workunit.client.0.smithi012.stdout:2/747: fdatasync d0/d6b/dbf/d57/d61/f6d 0 2024-03-18T14:51:03.093 INFO:tasks.workunit.client.0.smithi012.stdout:1/650: creat d8/db/dbe/fd2 x:0 0 0 2024-03-18T14:51:03.094 INFO:tasks.workunit.client.0.smithi012.stdout:1/651: creat d8/db/d10/d31/d81/fd3 x:0 0 0 2024-03-18T14:51:03.097 INFO:tasks.workunit.client.1.smithi100.stdout:8/396: dwrite d4/dc/dd/de/f63 [0,4194304] 0 2024-03-18T14:51:03.097 INFO:tasks.workunit.client.0.smithi012.stdout:5/688: getdents d3/d29 0 2024-03-18T14:51:03.097 INFO:tasks.workunit.client.0.smithi012.stdout:3/669: dwrite dc/d3c/d4c/f94 [0,4194304] 0 2024-03-18T14:51:03.099 INFO:tasks.workunit.client.1.smithi100.stdout:3/396: dwrite d3/dd/d21/d3e/f75 [0,4194304] 0 2024-03-18T14:51:03.099 INFO:tasks.workunit.client.0.smithi012.stdout:9/740: mknod d3/de6/dbf/d77/d52/cf0 0 2024-03-18T14:51:03.101 INFO:tasks.workunit.client.1.smithi100.stdout:2/342: rmdir d3/da/d78 0 2024-03-18T14:51:03.101 INFO:tasks.workunit.client.1.smithi100.stdout:2/343: read d3/d6/f2f [94221,35720] 0 2024-03-18T14:51:03.102 INFO:tasks.workunit.client.0.smithi012.stdout:8/903: symlink d0/d5a/d5e/ddf/d50/d6a/l11f 0 2024-03-18T14:51:03.105 INFO:tasks.workunit.client.1.smithi100.stdout:7/356: creat d5/d22/d24/d6e/f84 x:0 0 0 2024-03-18T14:51:03.112 INFO:tasks.workunit.client.1.smithi100.stdout:8/397: mkdir d4/dc/dd/de/d21/d23/d47/d82 0 2024-03-18T14:51:03.113 INFO:tasks.workunit.client.1.smithi100.stdout:8/398: dread - d4/dc/dd/de/d19/d37/f7f zero size 2024-03-18T14:51:03.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/397: dread d3/d16/d13/f47 [0,4194304] 0 2024-03-18T14:51:03.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/398: write d3/d16/d13/d1f/d7c/f83 [212205,75540] 0 2024-03-18T14:51:03.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/399: fdatasync d3/d16/d13/d17/f33 0 2024-03-18T14:51:03.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/400: creat d3/d16/d13/d1f/f90 x:0 0 0 2024-03-18T14:51:03.117 INFO:tasks.workunit.client.0.smithi012.stdout:4/619: rmdir d6/da/dc0 0 2024-03-18T14:51:03.117 INFO:tasks.workunit.client.0.smithi012.stdout:4/620: truncate d6/d1a/d30/d46/d6f/d7c/faa 696326 0 2024-03-18T14:51:03.121 INFO:tasks.workunit.client.0.smithi012.stdout:2/748: mkdir d0/d6b/dbf/d33/d71/d90/dea 0 2024-03-18T14:51:03.123 INFO:tasks.workunit.client.1.smithi100.stdout:7/357: mkdir d5/d14/d16/d21/d85 0 2024-03-18T14:51:03.125 INFO:tasks.workunit.client.1.smithi100.stdout:8/399: mknod d4/dc/dd/c83 0 2024-03-18T14:51:03.125 INFO:tasks.workunit.client.1.smithi100.stdout:8/400: write d4/dc/d31/f69 [323294,44748] 0 2024-03-18T14:51:03.129 INFO:tasks.workunit.client.0.smithi012.stdout:0/753: symlink d1/d20/d7b/lf7 0 2024-03-18T14:51:03.130 INFO:tasks.workunit.client.1.smithi100.stdout:3/401: creat d3/d16/d13/d1f/d3f/d4f/d65/f91 x:0 0 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:5/689: mkdir d3/de0 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:5/690: fdatasync d3/d29/d36/fb3 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:5/691: write d3/d29/d9d/dba/fd7 [569883,79332] 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:5/692: readlink d3/d29/d35/d5b/lc2 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.0.smithi012.stdout:5/693: dread d3/f22 [0,4194304] 0 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.1.smithi100.stdout:9/372: sync 2024-03-18T14:51:03.134 INFO:tasks.workunit.client.1.smithi100.stdout:1/388: sync 2024-03-18T14:51:03.135 INFO:tasks.workunit.client.1.smithi100.stdout:1/389: chown d8/d1e/d11/d27/d3f/c68 1553 1 2024-03-18T14:51:03.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/465: dwrite dd/d2b/f89 [0,4194304] 0 2024-03-18T14:51:03.135 INFO:tasks.workunit.client.1.smithi100.stdout:0/466: write dd/f5d [4933795,42295] 0 2024-03-18T14:51:03.135 INFO:tasks.workunit.client.1.smithi100.stdout:8/401: creat d4/d5a/f84 x:0 0 0 2024-03-18T14:51:03.138 INFO:tasks.workunit.client.1.smithi100.stdout:3/402: mknod d3/d16/d13/d17/c92 0 2024-03-18T14:51:03.138 INFO:tasks.workunit.client.1.smithi100.stdout:3/403: readlink d3/dd/d21/l73 0 2024-03-18T14:51:03.138 INFO:tasks.workunit.client.1.smithi100.stdout:3/404: write d3/d16/d13/d1f/d37/f86 [904818,19595] 0 2024-03-18T14:51:03.139 INFO:tasks.workunit.client.0.smithi012.stdout:5/694: dread d3/d1e/f1a [0,4194304] 0 2024-03-18T14:51:03.139 INFO:tasks.workunit.client.0.smithi012.stdout:5/695: creat d3/d15/d5f/fe1 x:0 0 0 2024-03-18T14:51:03.139 INFO:tasks.workunit.client.1.smithi100.stdout:3/405: read d3/f57 [139682,80491] 0 2024-03-18T14:51:03.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/406: dread d3/d16/f36 [0,4194304] 0 2024-03-18T14:51:03.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/407: stat d3/d16/d13/d1f/f6c 0 2024-03-18T14:51:03.141 INFO:tasks.workunit.client.0.smithi012.stdout:9/741: mknod d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/cf1 0 2024-03-18T14:51:03.142 INFO:tasks.workunit.client.0.smithi012.stdout:8/904: symlink d0/d5a/d19/dad/l120 0 2024-03-18T14:51:03.142 INFO:tasks.workunit.client.0.smithi012.stdout:8/905: creat d0/d47/d59/dc9/d87/f121 x:0 0 0 2024-03-18T14:51:03.143 INFO:tasks.workunit.client.0.smithi012.stdout:3/670: dwrite dc/d3c/d4c/d57/fa1 [0,4194304] 0 2024-03-18T14:51:03.143 INFO:tasks.workunit.client.0.smithi012.stdout:3/671: readlink dc/d3c/d4c/lce 0 2024-03-18T14:51:03.143 INFO:tasks.workunit.client.0.smithi012.stdout:3/672: creat dc/d3c/fde x:0 0 0 2024-03-18T14:51:03.144 INFO:tasks.workunit.client.0.smithi012.stdout:4/621: rmdir d6/d19/d6c/d96 39 2024-03-18T14:51:03.145 INFO:tasks.workunit.client.1.smithi100.stdout:9/373: dread da/db/f4c [0,4194304] 0 2024-03-18T14:51:03.146 INFO:tasks.workunit.client.1.smithi100.stdout:9/374: write da/d2c/d2f/f30 [1256263,93381] 0 2024-03-18T14:51:03.146 INFO:tasks.workunit.client.1.smithi100.stdout:9/375: chown f9 802135 1 2024-03-18T14:51:03.146 INFO:tasks.workunit.client.1.smithi100.stdout:6/351: sync 2024-03-18T14:51:03.146 INFO:tasks.workunit.client.1.smithi100.stdout:4/370: sync 2024-03-18T14:51:03.147 INFO:tasks.workunit.client.0.smithi012.stdout:1/652: dwrite d8/db/d10/d31/f4d [0,4194304] 0 2024-03-18T14:51:03.148 INFO:tasks.workunit.client.0.smithi012.stdout:2/749: dwrite d0/d3c/f46 [0,4194304] 0 2024-03-18T14:51:03.148 INFO:tasks.workunit.client.0.smithi012.stdout:0/754: rename d1/d20/d23/d3b/d33/fbd to d1/d16/d19/dd0/ff8 0 2024-03-18T14:51:03.148 INFO:tasks.workunit.client.1.smithi100.stdout:2/344: dwrite d3/d6/f44 [0,4194304] 0 2024-03-18T14:51:03.148 INFO:tasks.workunit.client.1.smithi100.stdout:2/345: truncate d3/da/f3a 766625 0 2024-03-18T14:51:03.148 INFO:tasks.workunit.client.1.smithi100.stdout:2/346: chown d3/da/de/c64 24 1 2024-03-18T14:51:03.149 INFO:tasks.workunit.client.1.smithi100.stdout:0/467: creat dd/d2b/d54/fab x:0 0 0 2024-03-18T14:51:03.149 INFO:tasks.workunit.client.1.smithi100.stdout:0/468: write dd/d31/d91/d65/f74 [341589,106070] 0 2024-03-18T14:51:03.149 INFO:tasks.workunit.client.1.smithi100.stdout:0/469: truncate dd/d2b/d95/d79/d7d/f7e 474338 0 2024-03-18T14:51:03.151 INFO:tasks.workunit.client.1.smithi100.stdout:8/402: mkdir d4/dc/d31/d85 0 2024-03-18T14:51:03.151 INFO:tasks.workunit.client.1.smithi100.stdout:8/403: dread - d4/dc/dd/de/d19/d37/f7f zero size 2024-03-18T14:51:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/696: mknod d3/d15/d42/dc6/d73/d6b/d8f/ce2 0 2024-03-18T14:51:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/697: write d3/d84/f9c [664846,76248] 0 2024-03-18T14:51:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/698: creat d3/d29/d36/fe3 x:0 0 0 2024-03-18T14:51:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/699: stat d3/d29/c56 0 2024-03-18T14:51:03.152 INFO:tasks.workunit.client.0.smithi012.stdout:5/700: creat d3/d29/d36/fe4 x:0 0 0 2024-03-18T14:51:03.155 INFO:tasks.workunit.client.0.smithi012.stdout:9/742: creat d3/de6/dbf/d66/d6e/dc6/ff2 x:0 0 0 2024-03-18T14:51:03.155 INFO:tasks.workunit.client.0.smithi012.stdout:9/743: truncate d3/de6/dbf/d77/d52/d78/f7e 8839015 0 2024-03-18T14:51:03.155 INFO:tasks.workunit.client.0.smithi012.stdout:9/744: write d3/de6/dbf/d66/d69/d7f/d8c/dac/fcb [412878,97931] 0 2024-03-18T14:51:03.157 INFO:tasks.workunit.client.1.smithi100.stdout:3/408: mknod d3/d16/d13/d17/d7d/c93 0 2024-03-18T14:51:03.161 INFO:tasks.workunit.client.1.smithi100.stdout:9/376: truncate f9 622795 0 2024-03-18T14:51:03.164 INFO:tasks.workunit.client.0.smithi012.stdout:8/906: mkdir d0/d5a/d5e/dd7/d122 0 2024-03-18T14:51:03.164 INFO:tasks.workunit.client.0.smithi012.stdout:8/907: write d0/d5a/d55/fef [4412376,86843] 0 2024-03-18T14:51:03.166 INFO:tasks.workunit.client.1.smithi100.stdout:6/352: rmdir d6/d7/df/d26/d4e/d66/d36/d4f 39 2024-03-18T14:51:03.166 INFO:tasks.workunit.client.1.smithi100.stdout:6/353: creat d6/f6c x:0 0 0 2024-03-18T14:51:03.169 INFO:tasks.workunit.client.1.smithi100.stdout:4/371: unlink d9/d13/d5a/f5f 0 2024-03-18T14:51:03.169 INFO:tasks.workunit.client.1.smithi100.stdout:4/372: write d9/fe [2006013,28935] 0 2024-03-18T14:51:03.171 INFO:tasks.workunit.client.1.smithi100.stdout:2/347: truncate d3/f10 1084043 0 2024-03-18T14:51:03.174 INFO:tasks.workunit.client.1.smithi100.stdout:0/470: mkdir dd/d2b/d95/d79/d7d/da9/dac 0 2024-03-18T14:51:03.175 INFO:tasks.workunit.client.0.smithi012.stdout:4/622: mknod d6/d1a/d92/cd1 0 2024-03-18T14:51:03.175 INFO:tasks.workunit.client.0.smithi012.stdout:8/908: write d0/d47/f4f [2765324,123025] 0 2024-03-18T14:51:03.176 INFO:tasks.workunit.client.1.smithi100.stdout:8/404: link d4/dc/f7b d4/dc/dd/de/d21/d23/d47/f86 0 2024-03-18T14:51:03.178 INFO:tasks.workunit.client.0.smithi012.stdout:5/701: dwrite d3/db6/fb7 [0,4194304] 0 2024-03-18T14:51:03.180 INFO:tasks.workunit.client.0.smithi012.stdout:9/745: dwrite d3/de6/dbf/d66/d69/d7f/d8c/dac/fb8 [0,4194304] 0 2024-03-18T14:51:03.181 INFO:tasks.workunit.client.1.smithi100.stdout:6/354: mknod d6/d7/df/d26/d32/d65/c6d 0 2024-03-18T14:51:03.182 INFO:tasks.workunit.client.0.smithi012.stdout:1/653: rmdir d8/db/d10/d31/d43/d5e/d65 39 2024-03-18T14:51:03.182 INFO:tasks.workunit.client.0.smithi012.stdout:1/654: read d8/d1e/d27/fb7 [214896,120624] 0 2024-03-18T14:51:03.184 INFO:tasks.workunit.client.1.smithi100.stdout:7/358: rename d5/d14/d16/d21 to d5/d14/d16/d2c/d86 0 2024-03-18T14:51:03.184 INFO:tasks.workunit.client.1.smithi100.stdout:2/348: dread d3/d6/d29/f3d [0,4194304] 0 2024-03-18T14:51:03.185 INFO:tasks.workunit.client.1.smithi100.stdout:4/373: write d9/f32 [1984679,78641] 0 2024-03-18T14:51:03.185 INFO:tasks.workunit.client.1.smithi100.stdout:4/374: write d9/d27/f6b [111576,23798] 0 2024-03-18T14:51:03.185 INFO:tasks.workunit.client.1.smithi100.stdout:8/405: read d4/dc/dd/de/d21/d23/f27 [6387633,109580] 0 2024-03-18T14:51:03.186 INFO:tasks.workunit.client.0.smithi012.stdout:8/909: dread d0/d15/f28 [0,4194304] 0 2024-03-18T14:51:03.187 INFO:tasks.workunit.client.1.smithi100.stdout:4/375: write d9/f1f [1967799,38930] 0 2024-03-18T14:51:03.189 INFO:tasks.workunit.client.0.smithi012.stdout:2/750: mkdir d0/da2/deb 0 2024-03-18T14:51:03.191 INFO:tasks.workunit.client.1.smithi100.stdout:8/406: write d4/f15 [2436126,83162] 0 2024-03-18T14:51:03.192 INFO:tasks.workunit.client.1.smithi100.stdout:8/407: read d4/dc/d1f/d59/d6c/f80 [557667,85028] 0 2024-03-18T14:51:03.193 INFO:tasks.workunit.client.0.smithi012.stdout:1/655: dread d8/db/fd [0,4194304] 0 2024-03-18T14:51:03.198 INFO:tasks.workunit.client.0.smithi012.stdout:9/746: dread d3/d9/f29 [0,4194304] 0 2024-03-18T14:51:03.198 INFO:tasks.workunit.client.0.smithi012.stdout:9/747: stat d3/de6/dbf/d66/d69/d7f/d8c/dac/fec 0 2024-03-18T14:51:03.203 INFO:tasks.workunit.client.0.smithi012.stdout:8/910: dwrite d0/d5a/d5e/ddf/d50/fa5 [0,4194304] 0 2024-03-18T14:51:03.207 INFO:tasks.workunit.client.1.smithi100.stdout:0/471: unlink dd/d2b/c30 0 2024-03-18T14:51:03.207 INFO:tasks.workunit.client.1.smithi100.stdout:0/472: chown dd/d20/c51 3615 1 2024-03-18T14:51:03.208 INFO:tasks.workunit.client.0.smithi012.stdout:0/755: creat d1/d20/d23/d3b/d33/d64/ff9 x:0 0 0 2024-03-18T14:51:03.208 INFO:tasks.workunit.client.0.smithi012.stdout:0/756: dread - d1/d20/d23/d7f/f97 zero size 2024-03-18T14:51:03.209 INFO:tasks.workunit.client.0.smithi012.stdout:9/748: read d3/de6/dbf/d77/da8/f96 [1225975,48880] 0 2024-03-18T14:51:03.209 INFO:tasks.workunit.client.0.smithi012.stdout:9/749: chown d3/de6/dbf/d66/l70 516 1 2024-03-18T14:51:03.210 INFO:tasks.workunit.client.1.smithi100.stdout:6/355: creat d6/d7/df/d26/d4e/d66/d36/f6e x:0 0 0 2024-03-18T14:51:03.210 INFO:tasks.workunit.client.1.smithi100.stdout:6/356: chown d6/d7/df/d26/d32/d65 335 1 2024-03-18T14:51:03.210 INFO:tasks.workunit.client.0.smithi012.stdout:9/750: dread d3/de6/dbf/d66/d69/d7f/d8c/dac/fae [0,4194304] 0 2024-03-18T14:51:03.213 INFO:tasks.workunit.client.1.smithi100.stdout:1/390: rename d8/d1e/d11/f33 to d8/d2a/f6b 0 2024-03-18T14:51:03.214 INFO:tasks.workunit.client.0.smithi012.stdout:1/656: dwrite d8/d1e/d27/f48 [0,4194304] 0 2024-03-18T14:51:03.214 INFO:tasks.workunit.client.0.smithi012.stdout:1/657: creat d8/db/d10/d16/da1/d2e/fd4 x:0 0 0 2024-03-18T14:51:03.215 INFO:tasks.workunit.client.1.smithi100.stdout:1/391: dread d8/d30/f35 [0,4194304] 0 2024-03-18T14:51:03.215 INFO:tasks.workunit.client.1.smithi100.stdout:7/359: truncate d5/d14/f1e 1516372 0 2024-03-18T14:51:03.215 INFO:tasks.workunit.client.1.smithi100.stdout:7/360: fdatasync d5/d14/d16/d2e/d62/d66/f54 0 2024-03-18T14:51:03.216 INFO:tasks.workunit.client.0.smithi012.stdout:4/623: mkdir d6/da/d58/dd2 0 2024-03-18T14:51:03.217 INFO:tasks.workunit.client.1.smithi100.stdout:2/349: mknod d3/d1a/d69/c7e 0 2024-03-18T14:51:03.219 INFO:tasks.workunit.client.1.smithi100.stdout:4/376: readlink d9/d13/l44 0 2024-03-18T14:51:03.221 INFO:tasks.workunit.client.0.smithi012.stdout:5/702: truncate d3/d15/d5f/f70 2319122 0 2024-03-18T14:51:03.222 INFO:tasks.workunit.client.1.smithi100.stdout:7/361: dread d5/d22/d24/d39/f45 [0,4194304] 0 2024-03-18T14:51:03.223 INFO:tasks.workunit.client.1.smithi100.stdout:8/408: symlink d4/dc/dd/de/d21/d23/d47/d56/l87 0 2024-03-18T14:51:03.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/751: creat d0/d6b/dbf/d31/fec x:0 0 0 2024-03-18T14:51:03.225 INFO:tasks.workunit.client.0.smithi012.stdout:2/752: write d0/d10/f13 [1507606,115973] 0 2024-03-18T14:51:03.227 INFO:tasks.workunit.client.1.smithi100.stdout:6/357: symlink d6/d7/df/d26/d4e/d53/l6f 0 2024-03-18T14:51:03.229 INFO:tasks.workunit.client.0.smithi012.stdout:8/911: mkdir d0/d15/d4e/d123 0 2024-03-18T14:51:03.232 INFO:tasks.workunit.client.1.smithi100.stdout:3/409: rename d3/f30 to d3/d16/d13/d1f/d3f/d4f/d60/f94 0 2024-03-18T14:51:03.233 INFO:tasks.workunit.client.1.smithi100.stdout:6/358: dread d6/d7/f5f [0,4194304] 0 2024-03-18T14:51:03.233 INFO:tasks.workunit.client.1.smithi100.stdout:7/362: write d5/d14/d16/d2e/d62/d66/f54 [3611503,27497] 0 2024-03-18T14:51:03.233 INFO:tasks.workunit.client.1.smithi100.stdout:7/363: fsync d5/d14/d16/f1d 0 2024-03-18T14:51:03.233 INFO:tasks.workunit.client.1.smithi100.stdout:7/364: truncate d5/f72 909507 0 2024-03-18T14:51:03.234 INFO:tasks.workunit.client.0.smithi012.stdout:0/757: mknod d1/d20/d4e/d54/d67/dbc/cfa 0 2024-03-18T14:51:03.234 INFO:tasks.workunit.client.0.smithi012.stdout:0/758: readlink d1/d16/d31/l28 0 2024-03-18T14:51:03.236 INFO:tasks.workunit.client.1.smithi100.stdout:1/392: getdents d8/d1e 0 2024-03-18T14:51:03.239 INFO:tasks.workunit.client.0.smithi012.stdout:9/751: rename d3/de6/dbf/d66/c3f to d3/de6/dbf/d77/d52/d95/da9/cf3 0 2024-03-18T14:51:03.239 INFO:tasks.workunit.client.0.smithi012.stdout:9/752: write d3/de6/dbf/d77/d52/d78/f7e [4138044,112673] 0 2024-03-18T14:51:03.247 INFO:tasks.workunit.client.1.smithi100.stdout:3/410: link d3/d16/d13/d17/c64 d3/d16/d13/d17/d7d/c95 0 2024-03-18T14:51:03.249 INFO:tasks.workunit.client.0.smithi012.stdout:1/658: mknod d8/cd5 0 2024-03-18T14:51:03.251 INFO:tasks.workunit.client.0.smithi012.stdout:5/703: dwrite d3/d15/d42/dc6/f3e [0,4194304] 0 2024-03-18T14:51:03.252 INFO:tasks.workunit.client.1.smithi100.stdout:9/377: rename f6 to da/d46/d4d/d67/f6e 0 2024-03-18T14:51:03.253 INFO:tasks.workunit.client.0.smithi012.stdout:4/624: stat d6/l7 0 2024-03-18T14:51:03.253 INFO:tasks.workunit.client.0.smithi012.stdout:4/625: dread - d6/d1a/d92/f5d zero size 2024-03-18T14:51:03.253 INFO:tasks.workunit.client.0.smithi012.stdout:4/626: truncate d6/f5c 4527609 0 2024-03-18T14:51:03.254 INFO:tasks.workunit.client.0.smithi012.stdout:4/627: stat d6/d1a/d98/daf 0 2024-03-18T14:51:03.254 INFO:tasks.workunit.client.1.smithi100.stdout:2/350: dwrite d3/da/f25 [0,4194304] 0 2024-03-18T14:51:03.256 INFO:tasks.workunit.client.1.smithi100.stdout:7/365: mkdir d5/d14/d16/d2c/d86/d85/d87 0 2024-03-18T14:51:03.259 INFO:tasks.workunit.client.1.smithi100.stdout:5/342: sync 2024-03-18T14:51:03.260 INFO:tasks.workunit.client.1.smithi100.stdout:4/377: link d9/d13/d5a/d5b/c47 d9/d13/c6f 0 2024-03-18T14:51:03.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/473: rename dd/f43 to dd/d31/d91/d7b/fad 0 2024-03-18T14:51:03.264 INFO:tasks.workunit.client.1.smithi100.stdout:0/474: write dd/f18 [129225,86663] 0 2024-03-18T14:51:03.264 INFO:tasks.workunit.client.1.smithi100.stdout:0/475: chown dd/d2b/d95/d79/d7d/f7e 41 1 2024-03-18T14:51:03.266 INFO:tasks.workunit.client.0.smithi012.stdout:2/753: mkdir d0/d6b/dbf/d33/d71/de5/ded 0 2024-03-18T14:51:03.266 INFO:tasks.workunit.client.1.smithi100.stdout:8/409: dwrite d4/dc/dd/d48/f5f [0,4194304] 0 2024-03-18T14:51:03.269 INFO:tasks.workunit.client.0.smithi012.stdout:5/704: dread d3/d29/d35/f48 [0,4194304] 0 2024-03-18T14:51:03.269 INFO:tasks.workunit.client.1.smithi100.stdout:9/378: mknod da/dd/d3c/d4b/c6f 0 2024-03-18T14:51:03.272 INFO:tasks.workunit.client.0.smithi012.stdout:0/759: fdatasync d1/d20/d7b/fa1 0 2024-03-18T14:51:03.276 INFO:tasks.workunit.client.1.smithi100.stdout:2/351: symlink d3/d6/l7f 0 2024-03-18T14:51:03.282 INFO:tasks.workunit.client.1.smithi100.stdout:9/379: read da/d46/d4d/f56 [2360935,8730] 0 2024-03-18T14:51:03.282 INFO:tasks.workunit.client.1.smithi100.stdout:9/380: read - da/d51/f69 zero size 2024-03-18T14:51:03.282 INFO:tasks.workunit.client.1.smithi100.stdout:5/343: symlink d29/d2d/d39/d61/l67 0 2024-03-18T14:51:03.282 INFO:tasks.workunit.client.1.smithi100.stdout:5/344: creat d29/d2d/d32/f68 x:0 0 0 2024-03-18T14:51:03.283 INFO:tasks.workunit.client.1.smithi100.stdout:6/359: dwrite d6/d7/df/d26/d32/f33 [0,4194304] 0 2024-03-18T14:51:03.283 INFO:tasks.workunit.client.0.smithi012.stdout:9/753: symlink d3/d49/lf4 0 2024-03-18T14:51:03.283 INFO:tasks.workunit.client.0.smithi012.stdout:9/754: fsync d3/de6/dbf/d66/d6e/fbc 0 2024-03-18T14:51:03.289 INFO:tasks.workunit.client.1.smithi100.stdout:0/476: rename dd/c33 to dd/d2b/d95/d79/d7d/cae 0 2024-03-18T14:51:03.291 INFO:tasks.workunit.client.1.smithi100.stdout:1/393: dwrite f5 [0,4194304] 0 2024-03-18T14:51:03.291 INFO:tasks.workunit.client.1.smithi100.stdout:3/411: dwrite d3/d16/d13/d1f/f59 [0,4194304] 0 2024-03-18T14:51:03.292 INFO:tasks.workunit.client.0.smithi012.stdout:2/754: mknod d0/d6b/dbf/d57/d61/cee 0 2024-03-18T14:51:03.292 INFO:tasks.workunit.client.0.smithi012.stdout:2/755: truncate d0/d3c/fd9 678657 0 2024-03-18T14:51:03.294 INFO:tasks.workunit.client.1.smithi100.stdout:0/477: dread dd/f2e [0,4194304] 0 2024-03-18T14:51:03.294 INFO:tasks.workunit.client.1.smithi100.stdout:2/352: link d3/da/l11 d3/d6/d5f/d6a/l80 0 2024-03-18T14:51:03.294 INFO:tasks.workunit.client.1.smithi100.stdout:2/353: dread - d3/d6/f67 zero size 2024-03-18T14:51:03.298 INFO:tasks.workunit.client.1.smithi100.stdout:6/360: rename d6/d7/f40 to d6/d10/d27/f70 0 2024-03-18T14:51:03.301 INFO:tasks.workunit.client.1.smithi100.stdout:1/394: symlink d8/l6c 0 2024-03-18T14:51:03.305 INFO:tasks.workunit.client.0.smithi012.stdout:2/756: stat d0/d6b/fc2 0 2024-03-18T14:51:03.307 INFO:tasks.workunit.client.0.smithi012.stdout:5/705: symlink d3/d1e/d33/le5 0 2024-03-18T14:51:03.309 INFO:tasks.workunit.client.1.smithi100.stdout:9/381: write da/d25/f35 [3464120,113251] 0 2024-03-18T14:51:03.309 INFO:tasks.workunit.client.1.smithi100.stdout:0/478: symlink dd/d2b/d95/d79/d7d/da9/laf 0 2024-03-18T14:51:03.309 INFO:tasks.workunit.client.0.smithi012.stdout:1/659: dwrite d8/db/d10/d31/d43/d5e/f62 [0,4194304] 0 2024-03-18T14:51:03.309 INFO:tasks.workunit.client.0.smithi012.stdout:1/660: creat d8/db/d10/d16/d6b/d7e/fd6 x:0 0 0 2024-03-18T14:51:03.312 INFO:tasks.workunit.client.0.smithi012.stdout:4/628: rename d6/d76 to d6/da/d89/d97/dc5/dd3 0 2024-03-18T14:51:03.314 INFO:tasks.workunit.client.1.smithi100.stdout:2/354: mknod d3/da/de/d48/c81 0 2024-03-18T14:51:03.314 INFO:tasks.workunit.client.1.smithi100.stdout:7/366: dwrite d5/d22/f57 [0,4194304] 0 2024-03-18T14:51:03.317 INFO:tasks.workunit.client.1.smithi100.stdout:1/395: rename d8/d1e/d11/d27/d3c to d8/d1e/d11/d6d 0 2024-03-18T14:51:03.317 INFO:tasks.workunit.client.1.smithi100.stdout:1/396: fsync d8/f69 0 2024-03-18T14:51:03.318 INFO:tasks.workunit.client.1.smithi100.stdout:0/479: dread dd/d31/d91/d56/d71/fa3 [0,4194304] 0 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.0.smithi012.stdout:0/760: dwrite d1/d16/d31/f3f [0,4194304] 0 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.0.smithi012.stdout:0/761: chown d1/d20/d23/d6f/f75 683712 1 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.1.smithi100.stdout:4/378: dwrite d9/d27/d42/f6a [0,4194304] 0 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.1.smithi100.stdout:8/410: dwrite d4/dc/dd/de/d21/d23/d47/f86 [0,4194304] 0 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.1.smithi100.stdout:8/411: dread - d4/f60 zero size 2024-03-18T14:51:03.321 INFO:tasks.workunit.client.1.smithi100.stdout:9/382: mkdir da/d2c/d70 0 2024-03-18T14:51:03.322 INFO:tasks.workunit.client.0.smithi012.stdout:1/661: rmdir d8/db/d10/d16/da1/db9 39 2024-03-18T14:51:03.323 INFO:tasks.workunit.client.1.smithi100.stdout:3/412: dwrite d3/dd/d21/d3e/f32 [4194304,4194304] 0 2024-03-18T14:51:03.324 INFO:tasks.workunit.client.1.smithi100.stdout:7/367: rmdir d5/d14/d16/d2c/d86/d85 39 2024-03-18T14:51:03.327 INFO:tasks.workunit.client.1.smithi100.stdout:2/355: dread d3/fc [0,4194304] 0 2024-03-18T14:51:03.327 INFO:tasks.workunit.client.1.smithi100.stdout:2/356: dread - d3/d1a/f60 zero size 2024-03-18T14:51:03.328 INFO:tasks.workunit.client.1.smithi100.stdout:1/397: rmdir d8/d1e/d11/d27 39 2024-03-18T14:51:03.328 INFO:tasks.workunit.client.1.smithi100.stdout:2/357: read d3/da/f33 [342909,106635] 0 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.1.smithi100.stdout:6/361: dwrite d6/d10/f52 [0,4194304] 0 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/706: dwrite d3/d29/f9a [0,4194304] 0 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/707: write d3/d15/d5f/fd2 [62387,53579] 0 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/708: chown d3/d15/d42/dc6/d73/d6b/d8f/cc1 2470 1 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/709: write d3/d15/d5f/fd2 [685529,5618] 0 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/710: chown d3/d84 5752 1 2024-03-18T14:51:03.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/711: chown d3/d15/l16 5 1 2024-03-18T14:51:03.330 INFO:tasks.workunit.client.0.smithi012.stdout:4/629: read d6/da/f91 [5744,66871] 0 2024-03-18T14:51:03.330 INFO:tasks.workunit.client.1.smithi100.stdout:4/379: dread d9/d17/f34 [0,4194304] 0 2024-03-18T14:51:03.330 INFO:tasks.workunit.client.1.smithi100.stdout:4/380: fdatasync d9/fe 0 2024-03-18T14:51:03.332 INFO:tasks.workunit.client.1.smithi100.stdout:0/480: mkdir dd/d2b/d95/d79/d7d/da9/db0 0 2024-03-18T14:51:03.332 INFO:tasks.workunit.client.1.smithi100.stdout:0/481: creat dd/fb1 x:0 0 0 2024-03-18T14:51:03.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/630: dread d6/da/d58/f8d [0,4194304] 0 2024-03-18T14:51:03.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/631: creat d6/da/d89/d95/fd4 x:0 0 0 2024-03-18T14:51:03.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/632: write d6/f21 [1707948,29807] 0 2024-03-18T14:51:03.335 INFO:tasks.workunit.client.0.smithi012.stdout:9/755: rename d3/de6/dbf/d66/c6b to d3/de6/dbf/d66/d21/d35/cf5 0 2024-03-18T14:51:03.335 INFO:tasks.workunit.client.0.smithi012.stdout:9/756: write d3/f2e [442142,93502] 0 2024-03-18T14:51:03.336 INFO:tasks.workunit.client.0.smithi012.stdout:9/757: chown d3/d49/c5b 479794 1 2024-03-18T14:51:03.336 INFO:tasks.workunit.client.1.smithi100.stdout:2/358: write d3/da/f43 [17191,2034] 0 2024-03-18T14:51:03.336 INFO:tasks.workunit.client.1.smithi100.stdout:2/359: readlink d3/d13/l14 0 2024-03-18T14:51:03.337 INFO:tasks.workunit.client.1.smithi100.stdout:7/368: write d5/d14/f40 [3817541,108649] 0 2024-03-18T14:51:03.337 INFO:tasks.workunit.client.1.smithi100.stdout:7/369: dread - d5/d30/f5d zero size 2024-03-18T14:51:03.338 INFO:tasks.workunit.client.1.smithi100.stdout:7/370: read d5/f72 [84103,75839] 0 2024-03-18T14:51:03.342 INFO:tasks.workunit.client.0.smithi012.stdout:4/633: write d6/d1a/d30/d46/f6b [778684,75893] 0 2024-03-18T14:51:03.342 INFO:tasks.workunit.client.0.smithi012.stdout:4/634: creat d6/d1a/d30/fd5 x:0 0 0 2024-03-18T14:51:03.343 INFO:tasks.workunit.client.0.smithi012.stdout:0/762: mknod d1/d16/d19/cfb 0 2024-03-18T14:51:03.344 INFO:tasks.workunit.client.1.smithi100.stdout:2/360: write d3/d6/d29/f3d [1455309,75454] 0 2024-03-18T14:51:03.344 INFO:tasks.workunit.client.1.smithi100.stdout:2/361: write d3/da/f25 [4938465,75627] 0 2024-03-18T14:51:03.344 INFO:tasks.workunit.client.1.smithi100.stdout:2/362: write d3/d1a/d69/f70 [128439,89104] 0 2024-03-18T14:51:03.344 INFO:tasks.workunit.client.1.smithi100.stdout:8/412: symlink d4/dc/d1f/d59/d6c/d7c/l88 0 2024-03-18T14:51:03.345 INFO:tasks.workunit.client.1.smithi100.stdout:3/413: dwrite d3/dd/f11 [0,4194304] 0 2024-03-18T14:51:03.345 INFO:tasks.workunit.client.1.smithi100.stdout:3/414: chown d3/l34 19350896 1 2024-03-18T14:51:03.345 INFO:tasks.workunit.client.1.smithi100.stdout:3/415: truncate d3/d16/f58 167157 0 2024-03-18T14:51:03.345 INFO:tasks.workunit.client.0.smithi012.stdout:1/662: mknod d8/db/d10/d31/d43/d5e/d65/d70/dc7/cd7 0 2024-03-18T14:51:03.346 INFO:tasks.workunit.client.1.smithi100.stdout:9/383: mkdir da/dd/d29/d71 0 2024-03-18T14:51:03.348 INFO:tasks.workunit.client.1.smithi100.stdout:7/371: write d5/d30/d4d/d5a/f6f [2060937,126444] 0 2024-03-18T14:51:03.348 INFO:tasks.workunit.client.1.smithi100.stdout:3/416: dread d3/d16/f14 [0,4194304] 0 2024-03-18T14:51:03.357 INFO:tasks.workunit.client.0.smithi012.stdout:5/712: rename d3/d15/d42/lc5 to d3/d29/d36/d9b/d59/le6 0 2024-03-18T14:51:03.357 INFO:tasks.workunit.client.0.smithi012.stdout:5/713: readlink d3/l6 0 2024-03-18T14:51:03.359 INFO:tasks.workunit.client.0.smithi012.stdout:9/758: symlink d3/de6/dbf/d66/d6e/lf6 0 2024-03-18T14:51:03.363 INFO:tasks.workunit.client.0.smithi012.stdout:4/635: mkdir d6/da/d89/d94/dbe/dd6 0 2024-03-18T14:51:03.364 INFO:tasks.workunit.client.1.smithi100.stdout:3/417: dwrite d3/dd/d49/f4a [0,4194304] 0 2024-03-18T14:51:03.364 INFO:tasks.workunit.client.1.smithi100.stdout:3/418: fsync d3/d16/d13/d17/f61 0 2024-03-18T14:51:03.366 INFO:tasks.workunit.client.0.smithi012.stdout:0/763: symlink d1/d20/d23/d6f/d93/dba/lfc 0 2024-03-18T14:51:03.366 INFO:tasks.workunit.client.0.smithi012.stdout:0/764: write d1/d16/fe [555483,120597] 0 2024-03-18T14:51:03.376 INFO:tasks.workunit.client.1.smithi100.stdout:3/419: dread d3/d16/d13/d17/f7f [0,4194304] 0 2024-03-18T14:51:03.377 INFO:tasks.workunit.client.0.smithi012.stdout:1/663: symlink d8/db/d10/d31/d43/ld8 0 2024-03-18T14:51:03.379 INFO:tasks.workunit.client.0.smithi012.stdout:4/636: read d6/fe [2433065,59603] 0 2024-03-18T14:51:03.379 INFO:tasks.workunit.client.1.smithi100.stdout:3/420: dread d3/f57 [0,4194304] 0 2024-03-18T14:51:03.379 INFO:tasks.workunit.client.1.smithi100.stdout:3/421: creat d3/d16/d13/d17/f96 x:0 0 0 2024-03-18T14:51:03.379 INFO:tasks.workunit.client.1.smithi100.stdout:3/422: stat d3/d16/d13/d1f/d37 0 2024-03-18T14:51:03.381 INFO:tasks.workunit.client.1.smithi100.stdout:1/398: creat d8/d1e/d11/d2d/f6e x:0 0 0 2024-03-18T14:51:03.384 INFO:tasks.workunit.client.0.smithi012.stdout:9/759: link d3/de6/dbf/d77/d52/d65/l9d d3/de6/dbf/d77/d52/lf7 0 2024-03-18T14:51:03.387 INFO:tasks.workunit.client.1.smithi100.stdout:1/399: dread d8/d2a/f6b [0,4194304] 0 2024-03-18T14:51:03.395 INFO:tasks.workunit.client.1.smithi100.stdout:4/381: creat d9/d13/d5a/d5b/f70 x:0 0 0 2024-03-18T14:51:03.395 INFO:tasks.workunit.client.1.smithi100.stdout:4/382: read d9/fe [236825,96094] 0 2024-03-18T14:51:03.395 INFO:tasks.workunit.client.1.smithi100.stdout:4/383: write d9/d13/d25/f48 [4594137,19699] 0 2024-03-18T14:51:03.395 INFO:tasks.workunit.client.1.smithi100.stdout:4/384: stat d9/d27/d62 0 2024-03-18T14:51:03.399 INFO:tasks.workunit.client.1.smithi100.stdout:0/482: mknod dd/d31/d91/d56/d71/cb2 0 2024-03-18T14:51:03.401 INFO:tasks.workunit.client.0.smithi012.stdout:1/664: unlink d8/db/d10/d31/d43/d5e/d65/c85 0 2024-03-18T14:51:03.402 INFO:tasks.workunit.client.0.smithi012.stdout:4/637: mknod d6/d27/cd7 0 2024-03-18T14:51:03.404 INFO:tasks.workunit.client.0.smithi012.stdout:5/714: link d3/d29/d36/d9b/d59/c7b d3/d1e/d33/ce7 0 2024-03-18T14:51:03.406 INFO:tasks.workunit.client.0.smithi012.stdout:4/638: dread d6/d1a/d30/d46/d6f/d7c/f81 [0,4194304] 0 2024-03-18T14:51:03.406 INFO:tasks.workunit.client.1.smithi100.stdout:2/363: mkdir d3/da/de/d20/d82 0 2024-03-18T14:51:03.414 INFO:tasks.workunit.client.1.smithi100.stdout:2/364: dread d3/da/f25 [0,4194304] 0 2024-03-18T14:51:03.483 INFO:tasks.workunit.client.1.smithi100.stdout:8/413: mknod d4/dc/d31/d85/c89 0 2024-03-18T14:51:03.485 INFO:tasks.workunit.client.1.smithi100.stdout:9/384: mkdir da/dd/d29/d71/d72 0 2024-03-18T14:51:03.485 INFO:tasks.workunit.client.1.smithi100.stdout:9/385: creat da/dd/f73 x:0 0 0 2024-03-18T14:51:03.486 INFO:tasks.workunit.client.1.smithi100.stdout:7/372: creat d5/d14/d16/d2c/d86/d85/d87/f88 x:0 0 0 2024-03-18T14:51:03.486 INFO:tasks.workunit.client.1.smithi100.stdout:7/373: write d5/f72 [1588726,19327] 0 2024-03-18T14:51:03.486 INFO:tasks.workunit.client.1.smithi100.stdout:7/374: readlink d5/l2f 0 2024-03-18T14:51:03.486 INFO:tasks.workunit.client.1.smithi100.stdout:7/375: dread - d5/d14/d16/d2c/d86/f7f zero size 2024-03-18T14:51:03.488 INFO:tasks.workunit.client.0.smithi012.stdout:9/760: creat d3/ff8 x:0 0 0 2024-03-18T14:51:03.491 INFO:tasks.workunit.client.0.smithi012.stdout:5/715: mknod d3/d29/d36/d9b/d59/d8b/ce8 0 2024-03-18T14:51:03.494 INFO:tasks.workunit.client.0.smithi012.stdout:4/639: symlink d6/ld8 0 2024-03-18T14:51:03.494 INFO:tasks.workunit.client.0.smithi012.stdout:4/640: write d6/d1a/d92/f79 [1240412,76688] 0 2024-03-18T14:51:03.496 INFO:tasks.workunit.client.0.smithi012.stdout:5/716: creat d3/d15/d42/dc6/d73/d6b/d8f/fe9 x:0 0 0 2024-03-18T14:51:03.497 INFO:tasks.workunit.client.0.smithi012.stdout:4/641: creat d6/db6/fd9 x:0 0 0 2024-03-18T14:51:03.499 INFO:tasks.workunit.client.0.smithi012.stdout:5/717: dread d3/f22 [0,4194304] 0 2024-03-18T14:51:03.501 INFO:tasks.workunit.client.1.smithi100.stdout:3/423: link d3/d16/d13/d17/f61 d3/d16/d13/d1f/f97 0 2024-03-18T14:51:03.502 INFO:tasks.workunit.client.0.smithi012.stdout:9/761: dwrite d3/f63 [0,4194304] 0 2024-03-18T14:51:03.503 INFO:tasks.workunit.client.0.smithi012.stdout:5/718: unlink d3/d29/d35/l53 0 2024-03-18T14:51:03.506 INFO:tasks.workunit.client.1.smithi100.stdout:1/400: mkdir d8/d2a/d37/d50/d6f 0 2024-03-18T14:51:03.506 INFO:tasks.workunit.client.1.smithi100.stdout:1/401: write d8/d1e/f36 [4709205,82760] 0 2024-03-18T14:51:03.514 INFO:tasks.workunit.client.0.smithi012.stdout:5/719: creat d3/d29/d35/fea x:0 0 0 2024-03-18T14:51:03.515 INFO:tasks.workunit.client.1.smithi100.stdout:0/483: symlink dd/d31/d91/d56/d59/lb3 0 2024-03-18T14:51:03.516 INFO:tasks.workunit.client.1.smithi100.stdout:2/365: stat d3/c18 0 2024-03-18T14:51:03.516 INFO:tasks.workunit.client.1.smithi100.stdout:3/424: dread d3/f29 [0,4194304] 0 2024-03-18T14:51:03.517 INFO:tasks.workunit.client.0.smithi012.stdout:5/720: creat d3/d29/d35/dcc/feb x:0 0 0 2024-03-18T14:51:03.517 INFO:tasks.workunit.client.0.smithi012.stdout:5/721: creat d3/db6/fec x:0 0 0 2024-03-18T14:51:03.517 INFO:tasks.workunit.client.0.smithi012.stdout:5/722: write d3/d29/d35/f5a [348133,99679] 0 2024-03-18T14:51:03.519 INFO:tasks.workunit.client.1.smithi100.stdout:8/414: creat d4/dc/f8a x:0 0 0 2024-03-18T14:51:03.520 INFO:tasks.workunit.client.1.smithi100.stdout:0/484: dread dd/f16 [0,4194304] 0 2024-03-18T14:51:03.522 INFO:tasks.workunit.client.0.smithi012.stdout:5/723: link d3/d29/d36/d9b/cbb d3/d29/d36/d9b/ddf/ced 0 2024-03-18T14:51:03.524 INFO:tasks.workunit.client.1.smithi100.stdout:9/386: mkdir da/d46/d4d/d67/d74 0 2024-03-18T14:51:03.524 INFO:tasks.workunit.client.1.smithi100.stdout:9/387: truncate da/d46/f63 696140 0 2024-03-18T14:51:03.524 INFO:tasks.workunit.client.0.smithi012.stdout:9/762: dwrite d3/de6/dbf/d77/d52/d95/fcf [0,4194304] 0 2024-03-18T14:51:03.527 INFO:tasks.workunit.client.0.smithi012.stdout:5/724: creat d3/d15/d42/dc6/d73/fee x:0 0 0 2024-03-18T14:51:03.527 INFO:tasks.workunit.client.0.smithi012.stdout:5/725: stat d3/d1e/f8c 0 2024-03-18T14:51:03.527 INFO:tasks.workunit.client.0.smithi012.stdout:5/726: chown d3/d15 0 1 2024-03-18T14:51:03.528 INFO:tasks.workunit.client.1.smithi100.stdout:1/402: symlink d8/d30/d61/l70 0 2024-03-18T14:51:03.529 INFO:tasks.workunit.client.1.smithi100.stdout:6/362: rename d6/d10 to d6/d7/df/d26/d4e/d66/d36/d4f/d71 0 2024-03-18T14:51:03.530 INFO:tasks.workunit.client.0.smithi012.stdout:9/763: creat d3/d9/ff9 x:0 0 0 2024-03-18T14:51:03.530 INFO:tasks.workunit.client.0.smithi012.stdout:9/764: truncate d3/de6/dbf/d66/d21/d35/d48/fa4 445172 0 2024-03-18T14:51:03.531 INFO:tasks.workunit.client.1.smithi100.stdout:6/363: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f5d [0,4194304] 0 2024-03-18T14:51:03.531 INFO:tasks.workunit.client.1.smithi100.stdout:6/364: chown d6/d7 313 1 2024-03-18T14:51:03.531 INFO:tasks.workunit.client.1.smithi100.stdout:6/365: write d6/d7/df/d26/d32/d48/f4a [2336776,51060] 0 2024-03-18T14:51:03.531 INFO:tasks.workunit.client.1.smithi100.stdout:2/366: creat d3/d13/d40/f83 x:0 0 0 2024-03-18T14:51:03.532 INFO:tasks.workunit.client.0.smithi012.stdout:5/727: mkdir d3/d29/d36/d9b/ddf/def 0 2024-03-18T14:51:03.532 INFO:tasks.workunit.client.1.smithi100.stdout:3/425: getdents d3/d16/d13/d71 0 2024-03-18T14:51:03.534 INFO:tasks.workunit.client.1.smithi100.stdout:8/415: mknod d4/dc/dd/de/d21/d23/d47/c8b 0 2024-03-18T14:51:03.535 INFO:tasks.workunit.client.0.smithi012.stdout:5/728: mkdir d3/db6/df0 0 2024-03-18T14:51:03.535 INFO:tasks.workunit.client.0.smithi012.stdout:5/729: fsync d3/d15/d42/dc6/d65/f6c 0 2024-03-18T14:51:03.535 INFO:tasks.workunit.client.0.smithi012.stdout:5/730: chown d3/d29/d36/d9b/l2a 730 1 2024-03-18T14:51:03.535 INFO:tasks.workunit.client.0.smithi012.stdout:5/731: dread - d3/d29/f57 zero size 2024-03-18T14:51:03.535 INFO:tasks.workunit.client.0.smithi012.stdout:5/732: write d3/d29/d36/d9b/f71 [4923831,80492] 0 2024-03-18T14:51:03.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/403: mkdir d8/d2a/d37/d50/d6f/d71 0 2024-03-18T14:51:03.539 INFO:tasks.workunit.client.0.smithi012.stdout:9/765: write d3/de6/dbf/d77/da8/f76 [2475706,124608] 0 2024-03-18T14:51:03.539 INFO:tasks.workunit.client.0.smithi012.stdout:9/766: chown d3/de6/dbf/d77/d52/d99 2 1 2024-03-18T14:51:03.541 INFO:tasks.workunit.client.1.smithi100.stdout:7/376: rename d5/f46 to d5/d22/d24/d6e/d7a/f89 0 2024-03-18T14:51:03.543 INFO:tasks.workunit.client.1.smithi100.stdout:2/367: unlink d3/d6/d5f/d6a/l80 0 2024-03-18T14:51:03.543 INFO:tasks.workunit.client.1.smithi100.stdout:2/368: write d3/d6/d29/d36/d49/f5b [461965,50237] 0 2024-03-18T14:51:03.544 INFO:tasks.workunit.client.1.smithi100.stdout:2/369: dread d3/d6/f2f [0,4194304] 0 2024-03-18T14:51:03.545 INFO:tasks.workunit.client.0.smithi012.stdout:5/733: write d3/d29/d36/d9b/f49 [789368,111441] 0 2024-03-18T14:51:03.546 INFO:tasks.workunit.client.1.smithi100.stdout:3/426: mknod d3/d16/d13/d1f/d37/c98 0 2024-03-18T14:51:03.548 INFO:tasks.workunit.client.1.smithi100.stdout:3/427: dread d3/dd/f2a [0,4194304] 0 2024-03-18T14:51:03.548 INFO:tasks.workunit.client.1.smithi100.stdout:8/416: unlink d4/dc/dd/de/d21/d23/c4f 0 2024-03-18T14:51:03.548 INFO:tasks.workunit.client.1.smithi100.stdout:8/417: read d4/f15 [3421344,55427] 0 2024-03-18T14:51:03.550 INFO:tasks.workunit.client.1.smithi100.stdout:1/404: symlink d8/d1e/d11/l72 0 2024-03-18T14:51:03.553 INFO:tasks.workunit.client.0.smithi012.stdout:9/767: rename d3/c24 to d3/d9/d5c/dbd/cfa 0 2024-03-18T14:51:03.553 INFO:tasks.workunit.client.0.smithi012.stdout:9/768: chown d3/d49 57080468 1 2024-03-18T14:51:03.554 INFO:tasks.workunit.client.1.smithi100.stdout:0/485: rename dd/d2b/d95/d79/d7d/da9/laf to dd/d31/d91/d56/lb4 0 2024-03-18T14:51:03.556 INFO:tasks.workunit.client.0.smithi012.stdout:5/734: link d3/d29/d35/d5b/l66 d3/d1e/d33/lf1 0 2024-03-18T14:51:03.556 INFO:tasks.workunit.client.0.smithi012.stdout:5/735: write d3/d29/d36/d9b/f49 [2827215,13001] 0 2024-03-18T14:51:03.557 INFO:tasks.workunit.client.1.smithi100.stdout:8/418: dread d4/dc/dd/de/d19/f1e [4194304,4194304] 0 2024-03-18T14:51:03.557 INFO:tasks.workunit.client.1.smithi100.stdout:7/377: mknod d5/d14/d16/d2e/d62/c8a 0 2024-03-18T14:51:03.557 INFO:tasks.workunit.client.1.smithi100.stdout:7/378: dread - d5/d14/d16/d2e/d7b/f65 zero size 2024-03-18T14:51:03.557 INFO:tasks.workunit.client.1.smithi100.stdout:7/379: creat d5/d22/d24/f8b x:0 0 0 2024-03-18T14:51:03.559 INFO:tasks.workunit.client.1.smithi100.stdout:2/370: symlink d3/d13/d40/d53/l84 0 2024-03-18T14:51:03.560 INFO:tasks.workunit.client.0.smithi012.stdout:9/769: mknod d3/de6/dbf/d77/d52/d99/da0/ddf/cfb 0 2024-03-18T14:51:03.560 INFO:tasks.workunit.client.0.smithi012.stdout:9/770: chown d3/de6/dc1 0 1 2024-03-18T14:51:03.560 INFO:tasks.workunit.client.0.smithi012.stdout:9/771: chown d3/de6/dbf/d66/f2b 2634279 1 2024-03-18T14:51:03.562 INFO:tasks.workunit.client.1.smithi100.stdout:3/428: mknod d3/d16/d13/d1f/d3f/d4f/d60/c99 0 2024-03-18T14:51:03.564 INFO:tasks.workunit.client.0.smithi012.stdout:5/736: mkdir d3/dad/df2 0 2024-03-18T14:51:03.564 INFO:tasks.workunit.client.0.smithi012.stdout:5/737: fdatasync d3/d15/d42/dc6/d73/f99 0 2024-03-18T14:51:03.565 INFO:tasks.workunit.client.1.smithi100.stdout:1/405: mkdir d8/d1e/d11/d27/d73 0 2024-03-18T14:51:03.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/366: rename d6/d7/df/d26/d32 to d6/d7/df/d72 0 2024-03-18T14:51:03.567 INFO:tasks.workunit.client.1.smithi100.stdout:7/380: read d5/f52 [5102317,11864] 0 2024-03-18T14:51:03.569 INFO:tasks.workunit.client.1.smithi100.stdout:8/419: symlink d4/dc/dd/de/d21/d23/d47/d56/l8c 0 2024-03-18T14:51:03.570 INFO:tasks.workunit.client.0.smithi012.stdout:9/772: dread d3/de6/dbf/d66/d69/d7f/d8c/d8b/f9f [0,4194304] 0 2024-03-18T14:51:03.571 INFO:tasks.workunit.client.1.smithi100.stdout:0/486: dwrite dd/d2b/d95/d79/fa4 [0,4194304] 0 2024-03-18T14:51:03.572 INFO:tasks.workunit.client.1.smithi100.stdout:2/371: link d3/d6/d29/d36/d49/f7a d3/d13/d40/d53/f85 0 2024-03-18T14:51:03.573 INFO:tasks.workunit.client.1.smithi100.stdout:3/429: creat d3/d16/d13/d71/f9a x:0 0 0 2024-03-18T14:51:03.573 INFO:tasks.workunit.client.1.smithi100.stdout:3/430: creat d3/d16/f9b x:0 0 0 2024-03-18T14:51:03.574 INFO:tasks.workunit.client.0.smithi012.stdout:9/773: mknod d3/de6/dbf/d77/d52/d99/da0/ddf/cfc 0 2024-03-18T14:51:03.575 INFO:tasks.workunit.client.0.smithi012.stdout:9/774: write d3/de6/dbf/d66/d69/d7f/d8c/dac/fec [282236,43926] 0 2024-03-18T14:51:03.576 INFO:tasks.workunit.client.1.smithi100.stdout:1/406: creat d8/d1e/d11/d27/d3f/f74 x:0 0 0 2024-03-18T14:51:03.577 INFO:tasks.workunit.client.1.smithi100.stdout:7/381: truncate d5/d14/d16/f1d 2595851 0 2024-03-18T14:51:03.578 INFO:tasks.workunit.client.0.smithi012.stdout:9/775: creat d3/de6/dbf/d77/d52/d99/da0/ddf/dee/ffd x:0 0 0 2024-03-18T14:51:03.579 INFO:tasks.workunit.client.1.smithi100.stdout:8/420: truncate d4/dc/d1f/d59/f64 724154 0 2024-03-18T14:51:03.581 INFO:tasks.workunit.client.1.smithi100.stdout:6/367: dwrite d6/f5e [0,4194304] 0 2024-03-18T14:51:03.581 INFO:tasks.workunit.client.1.smithi100.stdout:6/368: chown d6/d7/df/d72/d48/l69 2544 1 2024-03-18T14:51:03.582 INFO:tasks.workunit.client.0.smithi012.stdout:5/738: rename d3/d29/d36/d9b/d59 to d3/d15/d42/dc6/d73/d6b/df3 0 2024-03-18T14:51:03.582 INFO:tasks.workunit.client.1.smithi100.stdout:3/431: dread d3/dd/f11 [0,4194304] 0 2024-03-18T14:51:03.583 INFO:tasks.workunit.client.0.smithi012.stdout:9/776: mkdir d3/de6/dbf/d77/d52/d95/dfe 0 2024-03-18T14:51:03.583 INFO:tasks.workunit.client.0.smithi012.stdout:9/777: write d3/f4 [1385322,2905] 0 2024-03-18T14:51:03.583 INFO:tasks.workunit.client.1.smithi100.stdout:7/382: dread d5/d14/d16/d44/f76 [0,4194304] 0 2024-03-18T14:51:03.588 INFO:tasks.workunit.client.0.smithi012.stdout:5/739: getdents d3/d29/d9d 0 2024-03-18T14:51:03.590 INFO:tasks.workunit.client.1.smithi100.stdout:2/372: symlink d3/da/de/d20/l86 0 2024-03-18T14:51:03.590 INFO:tasks.workunit.client.1.smithi100.stdout:2/373: chown d3/da/de 2235 1 2024-03-18T14:51:03.594 INFO:tasks.workunit.client.1.smithi100.stdout:8/421: read d4/fa [1406853,34733] 0 2024-03-18T14:51:03.596 INFO:tasks.workunit.client.1.smithi100.stdout:1/407: mknod d8/d1e/d11/d27/d3f/c75 0 2024-03-18T14:51:03.600 INFO:tasks.workunit.client.0.smithi012.stdout:5/740: write d3/d1e/f93 [1990067,80698] 0 2024-03-18T14:51:03.602 INFO:tasks.workunit.client.1.smithi100.stdout:7/383: creat d5/d14/d16/d2e/d62/f8c x:0 0 0 2024-03-18T14:51:03.607 INFO:tasks.workunit.client.1.smithi100.stdout:2/374: mknod d3/d6/d29/d36/d49/c87 0 2024-03-18T14:51:03.607 INFO:tasks.workunit.client.1.smithi100.stdout:2/375: chown d3/d13/d4f/c55 10285 1 2024-03-18T14:51:03.609 INFO:tasks.workunit.client.1.smithi100.stdout:8/422: creat d4/dc/d1f/d33/f8d x:0 0 0 2024-03-18T14:51:03.612 INFO:tasks.workunit.client.1.smithi100.stdout:7/384: symlink d5/d14/d16/d2c/d86/d85/d87/l8d 0 2024-03-18T14:51:03.614 INFO:tasks.workunit.client.1.smithi100.stdout:0/487: rename dd/d2b/d54 to dd/d20/db5 0 2024-03-18T14:51:03.614 INFO:tasks.workunit.client.1.smithi100.stdout:6/369: dwrite d6/d12/f3b [0,4194304] 0 2024-03-18T14:51:03.615 INFO:tasks.workunit.client.1.smithi100.stdout:0/488: dread dd/d2b/d95/d79/d7d/f7e [0,4194304] 0 2024-03-18T14:51:03.617 INFO:tasks.workunit.client.1.smithi100.stdout:8/423: rmdir d4/dc/dd/de/d21/d23/d47/d82 0 2024-03-18T14:51:03.617 INFO:tasks.workunit.client.1.smithi100.stdout:3/432: dwrite d3/dd/f27 [0,4194304] 0 2024-03-18T14:51:03.618 INFO:tasks.workunit.client.1.smithi100.stdout:3/433: chown d3/dd/d21/d3e/c3d 29731 1 2024-03-18T14:51:03.618 INFO:tasks.workunit.client.1.smithi100.stdout:3/434: chown d3/d16/d13/d17/f7a 739 1 2024-03-18T14:51:03.618 INFO:tasks.workunit.client.1.smithi100.stdout:3/435: write d3/d16/d13/d1f/d3f/d4f/d60/f63 [92113,64311] 0 2024-03-18T14:51:03.620 INFO:tasks.workunit.client.1.smithi100.stdout:2/376: rename d3/d6/d29/d36/d49/l4b to d3/d6/d29/d36/d4a/l88 0 2024-03-18T14:51:03.624 INFO:tasks.workunit.client.1.smithi100.stdout:1/408: dwrite d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:03.625 INFO:tasks.workunit.client.1.smithi100.stdout:1/409: read d8/d1e/d11/d27/f64 [378320,33567] 0 2024-03-18T14:51:03.626 INFO:tasks.workunit.client.1.smithi100.stdout:6/370: creat d6/d7/df/d26/d4e/d66/d36/d4f/f73 x:0 0 0 2024-03-18T14:51:03.627 INFO:tasks.workunit.client.1.smithi100.stdout:8/424: mkdir d4/dc/d31/d85/d8e 0 2024-03-18T14:51:03.627 INFO:tasks.workunit.client.1.smithi100.stdout:8/425: write d4/fa [3047205,123399] 0 2024-03-18T14:51:03.630 INFO:tasks.workunit.client.1.smithi100.stdout:3/436: rmdir d3/d16/d13/d1f/d3f/d4f/d60 39 2024-03-18T14:51:03.631 INFO:tasks.workunit.client.1.smithi100.stdout:3/437: fsync d3/dd/d49/f72 0 2024-03-18T14:51:03.633 INFO:tasks.workunit.client.1.smithi100.stdout:5/345: sync 2024-03-18T14:51:03.633 INFO:tasks.workunit.client.1.smithi100.stdout:5/346: chown d29/f37 295576 1 2024-03-18T14:51:03.633 INFO:tasks.workunit.client.1.smithi100.stdout:4/385: sync 2024-03-18T14:51:03.633 INFO:tasks.workunit.client.1.smithi100.stdout:4/386: truncate d9/d13/d5a/d5b/f70 132958 0 2024-03-18T14:51:03.633 INFO:tasks.workunit.client.1.smithi100.stdout:2/377: unlink d3/d1a/c59 0 2024-03-18T14:51:03.634 INFO:tasks.workunit.client.1.smithi100.stdout:1/410: dread d8/d2a/f41 [0,4194304] 0 2024-03-18T14:51:03.636 INFO:tasks.workunit.client.1.smithi100.stdout:6/371: link d6/c16 d6/d7/df/d72/d65/c74 0 2024-03-18T14:51:03.637 INFO:tasks.workunit.client.1.smithi100.stdout:8/426: unlink d4/d5a/f84 0 2024-03-18T14:51:03.639 INFO:tasks.workunit.client.1.smithi100.stdout:3/438: creat d3/d16/d13/f9c x:0 0 0 2024-03-18T14:51:03.641 INFO:tasks.workunit.client.1.smithi100.stdout:4/387: symlink d9/d13/d25/d51/l71 0 2024-03-18T14:51:03.644 INFO:tasks.workunit.client.1.smithi100.stdout:2/378: rmdir d3/d1a/d76/d7c 0 2024-03-18T14:51:03.646 INFO:tasks.workunit.client.1.smithi100.stdout:6/372: link d6/d7/df/d26/f1c d6/d12/f75 0 2024-03-18T14:51:03.648 INFO:tasks.workunit.client.1.smithi100.stdout:8/427: symlink d4/dc/dd/l8f 0 2024-03-18T14:51:03.648 INFO:tasks.workunit.client.1.smithi100.stdout:8/428: chown d4/dc/d31/f46 6883937 1 2024-03-18T14:51:03.649 INFO:tasks.workunit.client.1.smithi100.stdout:3/439: symlink d3/d16/l9d 0 2024-03-18T14:51:03.650 INFO:tasks.workunit.client.0.smithi012.stdout:3/673: sync 2024-03-18T14:51:03.650 INFO:tasks.workunit.client.0.smithi012.stdout:7/694: sync 2024-03-18T14:51:03.650 INFO:tasks.workunit.client.0.smithi012.stdout:6/649: sync 2024-03-18T14:51:03.650 INFO:tasks.workunit.client.0.smithi012.stdout:8/912: sync 2024-03-18T14:51:03.651 INFO:tasks.workunit.client.1.smithi100.stdout:6/373: dread d6/d12/d2c/f4c [0,4194304] 0 2024-03-18T14:51:03.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/489: dwrite dd/d20/f32 [0,4194304] 0 2024-03-18T14:51:03.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/385: dwrite d5/d14/d16/d2c/d86/f50 [0,4194304] 0 2024-03-18T14:51:03.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/386: chown d5/d30/d4d/d5a/f80 4933 1 2024-03-18T14:51:03.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/387: write d5/d14/f40 [525243,120171] 0 2024-03-18T14:51:03.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/388: dread - d5/d14/d16/d2e/f7d zero size 2024-03-18T14:51:03.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/389: readlink d5/d22/l36 0 2024-03-18T14:51:03.653 INFO:tasks.workunit.client.1.smithi100.stdout:1/411: dwrite d8/d2a/d37/d50/f5e [4194304,4194304] 0 2024-03-18T14:51:03.653 INFO:tasks.workunit.client.1.smithi100.stdout:1/412: fsync d8/d1e/d11/f1b 0 2024-03-18T14:51:03.655 INFO:tasks.workunit.client.1.smithi100.stdout:5/347: unlink l27 0 2024-03-18T14:51:03.655 INFO:tasks.workunit.client.0.smithi012.stdout:6/650: symlink d17/d26/lc4 0 2024-03-18T14:51:03.655 INFO:tasks.workunit.client.0.smithi012.stdout:6/651: dread - d17/d38/d51/fa9 zero size 2024-03-18T14:51:03.655 INFO:tasks.workunit.client.0.smithi012.stdout:6/652: truncate d17/d18/f94 253763 0 2024-03-18T14:51:03.661 INFO:tasks.workunit.client.1.smithi100.stdout:2/379: rename d3/c18 to d3/d1a/d6c/c89 0 2024-03-18T14:51:03.663 INFO:tasks.workunit.client.0.smithi012.stdout:7/695: rename dd/d13/d23/d76/d6f/db3/dc7/fe6 to dd/d13/d23/d2f/d53/d8d/fef 0 2024-03-18T14:51:03.665 INFO:tasks.workunit.client.1.smithi100.stdout:8/429: unlink d4/dc/dd/l8f 0 2024-03-18T14:51:03.666 INFO:tasks.workunit.client.0.smithi012.stdout:8/913: symlink d0/d5a/d5e/dd7/d84/d2a/l124 0 2024-03-18T14:51:03.667 INFO:tasks.workunit.client.1.smithi100.stdout:3/440: truncate d3/d16/d13/d17/f2b 647561 0 2024-03-18T14:51:03.669 INFO:tasks.workunit.client.0.smithi012.stdout:6/653: rmdir d17/d38/d51/d66/db9 0 2024-03-18T14:51:03.669 INFO:tasks.workunit.client.0.smithi012.stdout:6/654: chown d17/d18/f23 6 1 2024-03-18T14:51:03.672 INFO:tasks.workunit.client.0.smithi012.stdout:7/696: rmdir dd 39 2024-03-18T14:51:03.672 INFO:tasks.workunit.client.0.smithi012.stdout:7/697: fdatasync dd/d13/d23/d33/f78 0 2024-03-18T14:51:03.673 INFO:tasks.workunit.client.0.smithi012.stdout:7/698: dread dd/d13/d23/d2f/d7a/db6/f9b [0,4194304] 0 2024-03-18T14:51:03.673 INFO:tasks.workunit.client.0.smithi012.stdout:7/699: dread - dd/d13/d23/d2f/d53/fec zero size 2024-03-18T14:51:03.675 INFO:tasks.workunit.client.0.smithi012.stdout:8/914: truncate d0/d5a/fcb 3638509 0 2024-03-18T14:51:03.680 INFO:tasks.workunit.client.1.smithi100.stdout:4/388: dwrite d9/f3b [0,4194304] 0 2024-03-18T14:51:03.680 INFO:tasks.workunit.client.1.smithi100.stdout:4/389: chown d9/d27/d42/f4a 46009707 1 2024-03-18T14:51:03.680 INFO:tasks.workunit.client.1.smithi100.stdout:6/374: dwrite d6/d7/df/f55 [0,4194304] 0 2024-03-18T14:51:03.680 INFO:tasks.workunit.client.1.smithi100.stdout:7/390: unlink d5/d22/f57 0 2024-03-18T14:51:03.681 INFO:tasks.workunit.client.1.smithi100.stdout:7/391: chown d5/d14/d16/d2e/d7b 2194 1 2024-03-18T14:51:03.681 INFO:tasks.workunit.client.0.smithi012.stdout:3/674: dwrite dc/d3c/d59/d86/d9a/f7e [0,4194304] 0 2024-03-18T14:51:03.683 INFO:tasks.workunit.client.0.smithi012.stdout:7/700: creat dd/d13/ff0 x:0 0 0 2024-03-18T14:51:03.685 INFO:tasks.workunit.client.1.smithi100.stdout:1/413: mkdir d8/d1e/d11/d27/d3f/d76 0 2024-03-18T14:51:03.690 INFO:tasks.workunit.client.1.smithi100.stdout:2/380: truncate d3/da/f30 2625892 0 2024-03-18T14:51:03.693 INFO:tasks.workunit.client.1.smithi100.stdout:8/430: creat d4/f90 x:0 0 0 2024-03-18T14:51:03.693 INFO:tasks.workunit.client.1.smithi100.stdout:1/414: write d8/d2a/f44 [3386979,33432] 0 2024-03-18T14:51:03.695 INFO:tasks.workunit.client.1.smithi100.stdout:3/441: symlink d3/d16/d13/d1f/d3f/d4f/d60/l9e 0 2024-03-18T14:51:03.695 INFO:tasks.workunit.client.0.smithi012.stdout:3/675: dwrite dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:51:03.698 INFO:tasks.workunit.client.0.smithi012.stdout:7/701: mkdir dd/d13/d23/d2f/d53/dc6/df1 0 2024-03-18T14:51:03.703 INFO:tasks.workunit.client.1.smithi100.stdout:4/390: truncate d9/d17/f23 2149852 0 2024-03-18T14:51:03.708 INFO:tasks.workunit.client.1.smithi100.stdout:6/375: mkdir d6/d7/df/d72/d48/d76 0 2024-03-18T14:51:03.709 INFO:tasks.workunit.client.1.smithi100.stdout:6/376: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f22 [2561633,3007] 0 2024-03-18T14:51:03.709 INFO:tasks.workunit.client.1.smithi100.stdout:3/442: dwrite d3/fa [0,4194304] 0 2024-03-18T14:51:03.711 INFO:tasks.workunit.client.0.smithi012.stdout:3/676: rename dc/d3c/d59/d86/d9a/d64/f79 to dc/d30/d3a/fdf 0 2024-03-18T14:51:03.716 INFO:tasks.workunit.client.0.smithi012.stdout:7/702: rename dd/d13/ff0 to dd/d13/d23/d2f/d53/dc6/df1/ff2 0 2024-03-18T14:51:03.716 INFO:tasks.workunit.client.0.smithi012.stdout:7/703: write dd/d13/d23/d33/d48/f5c [1815970,125247] 0 2024-03-18T14:51:03.716 INFO:tasks.workunit.client.0.smithi012.stdout:7/704: dread - dd/d13/d23/d2f/d7a/dab/dbc/fc3 zero size 2024-03-18T14:51:03.716 INFO:tasks.workunit.client.0.smithi012.stdout:7/705: creat dd/d13/d23/d76/d6f/db3/ff3 x:0 0 0 2024-03-18T14:51:03.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/377: dread d6/d12/f3b [0,4194304] 0 2024-03-18T14:51:03.718 INFO:tasks.workunit.client.0.smithi012.stdout:3/677: mknod dc/d3c/d4c/ce0 0 2024-03-18T14:51:03.728 INFO:tasks.workunit.client.0.smithi012.stdout:7/706: getdents dd/d13/d23/d76 0 2024-03-18T14:51:03.729 INFO:tasks.workunit.client.1.smithi100.stdout:9/388: sync 2024-03-18T14:51:03.729 INFO:tasks.workunit.client.1.smithi100.stdout:9/389: creat da/d46/d4d/d67/f75 x:0 0 0 2024-03-18T14:51:03.729 INFO:tasks.workunit.client.1.smithi100.stdout:9/390: dread - da/dd/f60 zero size 2024-03-18T14:51:03.729 INFO:tasks.workunit.client.1.smithi100.stdout:9/391: fdatasync da/d2c/d2f/f30 0 2024-03-18T14:51:03.732 INFO:tasks.workunit.client.0.smithi012.stdout:3/678: rename dc/ca3 to dc/d30/d3a/db7/ce1 0 2024-03-18T14:51:03.732 INFO:tasks.workunit.client.0.smithi012.stdout:3/679: stat dc/d3c 0 2024-03-18T14:51:03.733 INFO:tasks.workunit.client.0.smithi012.stdout:5/741: dwrite d3/d1e/d33/f4b [0,4194304] 0 2024-03-18T14:51:03.733 INFO:tasks.workunit.client.0.smithi012.stdout:5/742: chown d3/d29/d36/d9b/l2a 166 1 2024-03-18T14:51:03.736 INFO:tasks.workunit.client.0.smithi012.stdout:7/707: creat dd/d13/d23/d2f/d53/dc6/df1/ff4 x:0 0 0 2024-03-18T14:51:03.736 INFO:tasks.workunit.client.0.smithi012.stdout:7/708: creat dd/d13/d23/d2f/d53/ff5 x:0 0 0 2024-03-18T14:51:03.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/348: rmdir d29/d2d 39 2024-03-18T14:51:03.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/349: truncate d29/d2d/d32/d38/f5d 701878 0 2024-03-18T14:51:03.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/350: chown d29/d2d/c55 21261 1 2024-03-18T14:51:03.745 INFO:tasks.workunit.client.0.smithi012.stdout:5/743: rename d3/fc to d3/d15/d42/dc6/d73/d6b/ff4 0 2024-03-18T14:51:03.747 INFO:tasks.workunit.client.1.smithi100.stdout:1/415: link d8/d30/f35 d8/d1e/d11/f77 0 2024-03-18T14:51:03.747 INFO:tasks.workunit.client.1.smithi100.stdout:1/416: fsync d8/d1e/d11/d2d/f6e 0 2024-03-18T14:51:03.747 INFO:tasks.workunit.client.1.smithi100.stdout:1/417: creat d8/d1e/d11/d27/d3f/f78 x:0 0 0 2024-03-18T14:51:03.747 INFO:tasks.workunit.client.1.smithi100.stdout:1/418: dread - d8/d1e/d11/f60 zero size 2024-03-18T14:51:03.751 INFO:tasks.workunit.client.0.smithi012.stdout:7/709: creat dd/d13/d23/d2f/d7a/dab/dbc/ff6 x:0 0 0 2024-03-18T14:51:03.751 INFO:tasks.workunit.client.0.smithi012.stdout:7/710: creat dd/d13/d23/d2f/d53/ff7 x:0 0 0 2024-03-18T14:51:03.752 INFO:tasks.workunit.client.0.smithi012.stdout:3/680: dwrite dc/d3c/d59/d86/d9a/d54/f58 [0,4194304] 0 2024-03-18T14:51:03.752 INFO:tasks.workunit.client.0.smithi012.stdout:3/681: stat dc/fcb 0 2024-03-18T14:51:03.752 INFO:tasks.workunit.client.0.smithi012.stdout:3/682: fdatasync dc/d3c/f6c 0 2024-03-18T14:51:03.752 INFO:tasks.workunit.client.0.smithi012.stdout:7/711: read dd/d13/d23/d2f/d53/fa5 [338568,103908] 0 2024-03-18T14:51:03.754 INFO:tasks.workunit.client.0.smithi012.stdout:5/744: mknod d3/d29/cf5 0 2024-03-18T14:51:03.756 INFO:tasks.workunit.client.1.smithi100.stdout:3/443: rename d3/d16/f36 to d3/dd/d49/f9f 0 2024-03-18T14:51:03.759 INFO:tasks.workunit.client.1.smithi100.stdout:6/378: rmdir d6/d7/df/d26/d4e/d66 39 2024-03-18T14:51:03.759 INFO:tasks.workunit.client.1.smithi100.stdout:6/379: chown f4 30 1 2024-03-18T14:51:03.759 INFO:tasks.workunit.client.1.smithi100.stdout:6/380: readlink d6/d7/df/d26/d4e/d66/d36/l49 0 2024-03-18T14:51:03.761 INFO:tasks.workunit.client.0.smithi012.stdout:6/655: rmdir d17/d38/d51/d66 39 2024-03-18T14:51:03.761 INFO:tasks.workunit.client.0.smithi012.stdout:6/656: fdatasync d17/d19/d27/f68 0 2024-03-18T14:51:03.764 INFO:tasks.workunit.client.1.smithi100.stdout:7/392: symlink d5/d14/d16/d2c/l8e 0 2024-03-18T14:51:03.767 INFO:tasks.workunit.client.0.smithi012.stdout:6/657: dread d17/d38/fa2 [0,4194304] 0 2024-03-18T14:51:03.767 INFO:tasks.workunit.client.0.smithi012.stdout:6/658: stat d17/f35 0 2024-03-18T14:51:03.769 INFO:tasks.workunit.client.1.smithi100.stdout:1/419: dwrite d8/d30/f35 [0,4194304] 0 2024-03-18T14:51:03.770 INFO:tasks.workunit.client.0.smithi012.stdout:3/683: unlink dc/d30/d35/c33 0 2024-03-18T14:51:03.770 INFO:tasks.workunit.client.0.smithi012.stdout:6/659: dread d17/d26/d36/f7a [0,4194304] 0 2024-03-18T14:51:03.772 INFO:tasks.workunit.client.0.smithi012.stdout:7/712: mkdir dd/d13/d23/d76/d6f/dd9/ddd/df8 0 2024-03-18T14:51:03.772 INFO:tasks.workunit.client.0.smithi012.stdout:7/713: write dd/d13/f57 [1039398,78027] 0 2024-03-18T14:51:03.775 INFO:tasks.workunit.client.0.smithi012.stdout:5/745: creat d3/dad/ff6 x:0 0 0 2024-03-18T14:51:03.776 INFO:tasks.workunit.client.1.smithi100.stdout:8/431: getdents d4/dc/dd/de/d21/d23/d47 0 2024-03-18T14:51:03.779 INFO:tasks.workunit.client.0.smithi012.stdout:6/660: read d17/f75 [1705722,28120] 0 2024-03-18T14:51:03.779 INFO:tasks.workunit.client.0.smithi012.stdout:6/661: fsync d17/d18/f45 0 2024-03-18T14:51:03.779 INFO:tasks.workunit.client.1.smithi100.stdout:6/381: dwrite d6/fe [0,4194304] 0 2024-03-18T14:51:03.781 INFO:tasks.workunit.client.0.smithi012.stdout:3/684: mknod dc/d3c/d59/db2/ce2 0 2024-03-18T14:51:03.781 INFO:tasks.workunit.client.1.smithi100.stdout:4/391: getdents d9/d27/d62 0 2024-03-18T14:51:03.782 INFO:tasks.workunit.client.1.smithi100.stdout:4/392: fsync d9/d3a/f3c 0 2024-03-18T14:51:03.783 INFO:tasks.workunit.client.0.smithi012.stdout:7/714: link dd/d13/d23/fce dd/d13/d23/d2f/d53/dc6/ff9 0 2024-03-18T14:51:03.786 INFO:tasks.workunit.client.0.smithi012.stdout:5/746: mkdir d3/d15/d5f/df7 0 2024-03-18T14:51:03.792 INFO:tasks.workunit.client.1.smithi100.stdout:7/393: truncate d5/fd 4051373 0 2024-03-18T14:51:03.792 INFO:tasks.workunit.client.0.smithi012.stdout:3/685: getdents dc/d3c/d59/d86/d9a/d54 0 2024-03-18T14:51:03.792 INFO:tasks.workunit.client.0.smithi012.stdout:3/686: write dc/fa5 [293892,9579] 0 2024-03-18T14:51:03.793 INFO:tasks.workunit.client.0.smithi012.stdout:2/757: sync 2024-03-18T14:51:03.793 INFO:tasks.workunit.client.0.smithi012.stdout:0/765: sync 2024-03-18T14:51:03.795 INFO:tasks.workunit.client.0.smithi012.stdout:5/747: link d3/d29/d36/d9b/f49 d3/d29/d35/dcc/ff8 0 2024-03-18T14:51:03.795 INFO:tasks.workunit.client.0.smithi012.stdout:5/748: truncate d3/d29/d35/dcc/feb 481991 0 2024-03-18T14:51:03.801 INFO:tasks.workunit.client.0.smithi012.stdout:3/687: mkdir dc/d3c/d59/d86/d9a/dd7/de3 0 2024-03-18T14:51:03.801 INFO:tasks.workunit.client.0.smithi012.stdout:3/688: chown dc/d3c/lbf 345421 1 2024-03-18T14:51:03.804 INFO:tasks.workunit.client.1.smithi100.stdout:1/420: symlink d8/d1e/d11/d2d/l79 0 2024-03-18T14:51:03.805 INFO:tasks.workunit.client.0.smithi012.stdout:2/758: symlink d0/d6b/lef 0 2024-03-18T14:51:03.806 INFO:tasks.workunit.client.1.smithi100.stdout:9/392: getdents da/db 0 2024-03-18T14:51:03.807 INFO:tasks.workunit.client.0.smithi012.stdout:6/662: dwrite d17/f35 [0,4194304] 0 2024-03-18T14:51:03.808 INFO:tasks.workunit.client.0.smithi012.stdout:5/749: mknod d3/d29/d9d/dba/cf9 0 2024-03-18T14:51:03.808 INFO:tasks.workunit.client.0.smithi012.stdout:5/750: read d3/d1e/f93 [2530804,94213] 0 2024-03-18T14:51:03.808 INFO:tasks.workunit.client.1.smithi100.stdout:8/432: symlink d4/dc/dd/l91 0 2024-03-18T14:51:03.809 INFO:tasks.workunit.client.0.smithi012.stdout:3/689: mkdir dc/d30/d3a/da0/de4 0 2024-03-18T14:51:03.811 INFO:tasks.workunit.client.1.smithi100.stdout:6/382: rename d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/c24 to d6/d7/df/d72/d48/c77 0 2024-03-18T14:51:03.811 INFO:tasks.workunit.client.1.smithi100.stdout:6/383: truncate d6/d7/f8 958188 0 2024-03-18T14:51:03.812 INFO:tasks.workunit.client.0.smithi012.stdout:0/766: truncate d1/f4f 3204046 0 2024-03-18T14:51:03.812 INFO:tasks.workunit.client.0.smithi012.stdout:0/767: write d1/d16/d31/f35 [4870439,2469] 0 2024-03-18T14:51:03.813 INFO:tasks.workunit.client.1.smithi100.stdout:3/444: link d3/d16/c2f d3/ca0 0 2024-03-18T14:51:03.814 INFO:tasks.workunit.client.0.smithi012.stdout:7/715: dwrite dd/d13/d23/d60/f6c [0,4194304] 0 2024-03-18T14:51:03.816 INFO:tasks.workunit.client.0.smithi012.stdout:6/663: mknod d17/d26/d36/d4d/cc5 0 2024-03-18T14:51:03.816 INFO:tasks.workunit.client.0.smithi012.stdout:2/759: write d0/f6e [2841130,10721] 0 2024-03-18T14:51:03.818 INFO:tasks.workunit.client.0.smithi012.stdout:6/664: read d17/d19/f3c [418442,63302] 0 2024-03-18T14:51:03.818 INFO:tasks.workunit.client.0.smithi012.stdout:3/690: dread dc/f14 [0,4194304] 0 2024-03-18T14:51:03.818 INFO:tasks.workunit.client.0.smithi012.stdout:3/691: write dc/d30/d35/f75 [586429,42323] 0 2024-03-18T14:51:03.818 INFO:tasks.workunit.client.0.smithi012.stdout:3/692: stat dc/d3c/d59/d86 0 2024-03-18T14:51:03.819 INFO:tasks.workunit.client.1.smithi100.stdout:7/394: dwrite d5/d30/f3c [0,4194304] 0 2024-03-18T14:51:03.822 INFO:tasks.workunit.client.0.smithi012.stdout:5/751: mkdir d3/d84/dfa 0 2024-03-18T14:51:03.823 INFO:tasks.workunit.client.1.smithi100.stdout:9/393: dread da/f16 [0,4194304] 0 2024-03-18T14:51:03.825 INFO:tasks.workunit.client.1.smithi100.stdout:3/445: dread d3/dd/f28 [0,4194304] 0 2024-03-18T14:51:03.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/433: truncate d4/dc/dd/de/d19/f1e 6210070 0 2024-03-18T14:51:03.827 INFO:tasks.workunit.client.1.smithi100.stdout:8/434: stat d4/dc/dd/f44 0 2024-03-18T14:51:03.830 INFO:tasks.workunit.client.0.smithi012.stdout:6/665: dread ff [0,4194304] 0 2024-03-18T14:51:03.830 INFO:tasks.workunit.client.0.smithi012.stdout:6/666: chown d17/d26/c76 759095 1 2024-03-18T14:51:03.830 INFO:tasks.workunit.client.1.smithi100.stdout:4/393: rename d9/d13/f1e to d9/d27/d42/f72 0 2024-03-18T14:51:03.833 INFO:tasks.workunit.client.0.smithi012.stdout:0/768: unlink d1/d20/d23/d3b/ldc 0 2024-03-18T14:51:03.835 INFO:tasks.workunit.client.1.smithi100.stdout:6/384: mkdir d6/d12/d78 0 2024-03-18T14:51:03.839 INFO:tasks.workunit.client.0.smithi012.stdout:7/716: mkdir dd/d13/d23/d76/d6f/dd9/ddd/df8/dfa 0 2024-03-18T14:51:03.839 INFO:tasks.workunit.client.0.smithi012.stdout:6/667: write d17/d33/f88 [1291250,116098] 0 2024-03-18T14:51:03.839 INFO:tasks.workunit.client.0.smithi012.stdout:2/760: dwrite d0/d6b/dbf/d57/f64 [0,4194304] 0 2024-03-18T14:51:03.840 INFO:tasks.workunit.client.0.smithi012.stdout:2/761: dread - d0/d6b/d9f/fc5 zero size 2024-03-18T14:51:03.844 INFO:tasks.workunit.client.0.smithi012.stdout:2/762: dread d0/d6b/dbf/d33/d99/fce [0,4194304] 0 2024-03-18T14:51:03.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/395: link d5/d22/c7e d5/d14/d16/d2c/d86/d85/c8f 0 2024-03-18T14:51:03.849 INFO:tasks.workunit.client.1.smithi100.stdout:9/394: symlink da/l76 0 2024-03-18T14:51:03.849 INFO:tasks.workunit.client.1.smithi100.stdout:9/395: write da/d25/f35 [3822810,104039] 0 2024-03-18T14:51:03.850 INFO:tasks.workunit.client.0.smithi012.stdout:3/693: truncate dc/d3c/d59/fc8 61027 0 2024-03-18T14:51:03.853 INFO:tasks.workunit.client.1.smithi100.stdout:8/435: rename d4/dc/dd/de/d21/d23/d47/c8b to d4/dc/d1f/d59/d6c/c92 0 2024-03-18T14:51:03.855 INFO:tasks.workunit.client.0.smithi012.stdout:5/752: mkdir d3/d15/d42/dc6/d73/dd3/dfb 0 2024-03-18T14:51:03.857 INFO:tasks.workunit.client.1.smithi100.stdout:4/394: creat d9/d27/d62/f73 x:0 0 0 2024-03-18T14:51:03.857 INFO:tasks.workunit.client.1.smithi100.stdout:4/395: dread - d9/d27/f3d zero size 2024-03-18T14:51:03.857 INFO:tasks.workunit.client.1.smithi100.stdout:4/396: write d9/f1f [32203,119476] 0 2024-03-18T14:51:03.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/769: unlink d1/d16/d19/c3d 0 2024-03-18T14:51:03.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/770: creat d1/d20/d23/d3b/dc1/ffd x:0 0 0 2024-03-18T14:51:03.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/771: write d1/f9c [4543559,26758] 0 2024-03-18T14:51:03.859 INFO:tasks.workunit.client.0.smithi012.stdout:0/772: readlink d1/d20/d23/d82/ld9 0 2024-03-18T14:51:03.861 INFO:tasks.workunit.client.1.smithi100.stdout:6/385: unlink d6/d7/df/d26/l4b 0 2024-03-18T14:51:03.863 INFO:tasks.workunit.client.0.smithi012.stdout:7/717: rename dd/d13/d23/d76/d6f/dd9/fdf to dd/d13/d23/d33/d48/ffb 0 2024-03-18T14:51:03.863 INFO:tasks.workunit.client.0.smithi012.stdout:5/753: dread d3/d15/d42/dc6/d73/d6b/fcd [0,4194304] 0 2024-03-18T14:51:03.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/490: sync 2024-03-18T14:51:03.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/491: write dd/f18 [209262,84018] 0 2024-03-18T14:51:03.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/492: chown dd/d20/f27 11 1 2024-03-18T14:51:03.886 INFO:tasks.workunit.client.0.smithi012.stdout:6/668: mkdir d17/d26/d36/d4d/dc6 0 2024-03-18T14:51:03.888 INFO:tasks.workunit.client.1.smithi100.stdout:9/396: creat da/d2c/f77 x:0 0 0 2024-03-18T14:51:03.890 INFO:tasks.workunit.client.1.smithi100.stdout:8/436: creat d4/dc/dd/de/f93 x:0 0 0 2024-03-18T14:51:03.892 INFO:tasks.workunit.client.1.smithi100.stdout:0/493: dwrite dd/d2b/d95/d79/fa4 [0,4194304] 0 2024-03-18T14:51:03.893 INFO:tasks.workunit.client.0.smithi012.stdout:5/754: dwrite d3/d29/d36/d9b/f49 [0,4194304] 0 2024-03-18T14:51:03.893 INFO:tasks.workunit.client.1.smithi100.stdout:4/397: creat d9/d13/d39/f74 x:0 0 0 2024-03-18T14:51:03.894 INFO:tasks.workunit.client.0.smithi012.stdout:3/694: symlink dc/d3c/d72/le5 0 2024-03-18T14:51:03.896 INFO:tasks.workunit.client.1.smithi100.stdout:6/386: rename d6/d7/f59 to d6/d7/df/d26/d4e/d66/d36/d4f/d71/f79 0 2024-03-18T14:51:03.897 INFO:tasks.workunit.client.0.smithi012.stdout:6/669: dread d17/d19/f20 [0,4194304] 0 2024-03-18T14:51:03.899 INFO:tasks.workunit.client.0.smithi012.stdout:6/670: dread d17/d18/f45 [0,4194304] 0 2024-03-18T14:51:03.907 INFO:tasks.workunit.client.1.smithi100.stdout:2/381: sync 2024-03-18T14:51:03.914 INFO:tasks.workunit.client.0.smithi012.stdout:0/773: unlink d1/d16/d19/dd0/le8 0 2024-03-18T14:51:03.915 INFO:tasks.workunit.client.0.smithi012.stdout:0/774: stat d1/d20/d23/d3b/d33/d51/fa8 0 2024-03-18T14:51:03.916 INFO:tasks.workunit.client.1.smithi100.stdout:7/396: rmdir d5/d22/d24 39 2024-03-18T14:51:03.919 INFO:tasks.workunit.client.0.smithi012.stdout:2/763: rename d0/d3c/d40/d47/d85/cd1 to d0/d6b/dbf/d33/d71/d90/dea/cf0 0 2024-03-18T14:51:03.921 INFO:tasks.workunit.client.1.smithi100.stdout:9/397: creat da/dd/d29/d71/d72/f78 x:0 0 0 2024-03-18T14:51:03.921 INFO:tasks.workunit.client.1.smithi100.stdout:9/398: write da/d25/f3f [1885360,94340] 0 2024-03-18T14:51:03.922 INFO:tasks.workunit.client.0.smithi012.stdout:7/718: mknod dd/d13/d23/d76/d6f/db3/cfc 0 2024-03-18T14:51:03.922 INFO:tasks.workunit.client.0.smithi012.stdout:7/719: fsync dd/d13/d23/d60/f62 0 2024-03-18T14:51:03.922 INFO:tasks.workunit.client.0.smithi012.stdout:7/720: chown dd/d13/d23/d2f/d7a/db6/c81 32085 1 2024-03-18T14:51:03.923 INFO:tasks.workunit.client.1.smithi100.stdout:8/437: creat d4/dc/d1f/d59/d6c/d7c/f94 x:0 0 0 2024-03-18T14:51:03.923 INFO:tasks.workunit.client.1.smithi100.stdout:8/438: stat d4/dc/dd/de/f93 0 2024-03-18T14:51:03.924 INFO:tasks.workunit.client.1.smithi100.stdout:5/351: sync 2024-03-18T14:51:03.924 INFO:tasks.workunit.client.1.smithi100.stdout:5/352: stat d29/d2d/d39/d65/f66 0 2024-03-18T14:51:03.926 INFO:tasks.workunit.client.1.smithi100.stdout:0/494: truncate dd/f11 1828845 0 2024-03-18T14:51:03.927 INFO:tasks.workunit.client.1.smithi100.stdout:0/495: write dd/d20/f2d [136095,57598] 0 2024-03-18T14:51:03.928 INFO:tasks.workunit.client.1.smithi100.stdout:4/398: symlink d9/d13/d39/d4c/l75 0 2024-03-18T14:51:03.930 INFO:tasks.workunit.client.0.smithi012.stdout:0/775: mknod d1/d20/d23/d3b/d33/dea/cfe 0 2024-03-18T14:51:03.930 INFO:tasks.workunit.client.0.smithi012.stdout:0/776: chown d1/d16/d6e 140 1 2024-03-18T14:51:03.930 INFO:tasks.workunit.client.1.smithi100.stdout:7/397: mkdir d5/d90 0 2024-03-18T14:51:03.931 INFO:tasks.workunit.client.1.smithi100.stdout:7/398: truncate d5/d22/d24/d6e/f84 263233 0 2024-03-18T14:51:03.932 INFO:tasks.workunit.client.1.smithi100.stdout:9/399: mknod da/d2c/d2f/c79 0 2024-03-18T14:51:03.932 INFO:tasks.workunit.client.1.smithi100.stdout:9/400: fdatasync da/dd/f60 0 2024-03-18T14:51:03.933 INFO:tasks.workunit.client.0.smithi012.stdout:2/764: link d0/d6b/l70 d0/da2/lf1 0 2024-03-18T14:51:03.934 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:03 smithi012 ceph-mon[29182]: pgmap v6: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.0.smithi012.stdout:6/671: dwrite d17/d19/d27/d58/d91/fc3 [0,4194304] 0 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.0.smithi012.stdout:6/672: chown d17/d18/f94 0 1 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.1.smithi100.stdout:6/387: dwrite d6/d7/df/d26/d4e/d66/f68 [0,4194304] 0 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.1.smithi100.stdout:6/388: truncate d6/d7/df/d26/d4e/d66/d36/f6e 1006723 0 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.1.smithi100.stdout:2/382: dwrite d3/da/de/d20/f50 [0,4194304] 0 2024-03-18T14:51:03.935 INFO:tasks.workunit.client.1.smithi100.stdout:2/383: readlink d3/d6/l1f 0 2024-03-18T14:51:03.936 INFO:tasks.workunit.client.1.smithi100.stdout:8/439: symlink d4/dc/dd/de/d19/d2c/l95 0 2024-03-18T14:51:03.936 INFO:tasks.workunit.client.0.smithi012.stdout:7/721: mkdir dd/d13/d23/d60/d9c/dfd 0 2024-03-18T14:51:03.939 INFO:tasks.workunit.client.1.smithi100.stdout:5/353: creat d29/d2d/d39/d61/f69 x:0 0 0 2024-03-18T14:51:03.939 INFO:tasks.workunit.client.1.smithi100.stdout:5/354: truncate d29/d2d/d32/f5c 365290 0 2024-03-18T14:51:03.939 INFO:tasks.workunit.client.0.smithi012.stdout:5/755: getdents d3/d15/d42/dc6/d73/d6b/d8f 0 2024-03-18T14:51:03.942 INFO:tasks.workunit.client.1.smithi100.stdout:0/496: creat dd/d2b/d95/d79/d7d/da9/db0/fb6 x:0 0 0 2024-03-18T14:51:03.944 INFO:tasks.workunit.client.0.smithi012.stdout:3/695: mkdir dc/de6 0 2024-03-18T14:51:03.944 INFO:tasks.workunit.client.0.smithi012.stdout:3/696: chown dc/d30/d35/f2e 0 1 2024-03-18T14:51:03.946 INFO:tasks.workunit.client.1.smithi100.stdout:4/399: mkdir d9/d13/d39/d76 0 2024-03-18T14:51:03.946 INFO:tasks.workunit.client.1.smithi100.stdout:4/400: chown d9/d17/l49 20 1 2024-03-18T14:51:03.948 INFO:tasks.workunit.client.0.smithi012.stdout:0/777: symlink d1/d20/d4e/d54/d67/lff 0 2024-03-18T14:51:03.948 INFO:tasks.workunit.client.0.smithi012.stdout:0/778: stat d1/d16/d6e/lc4 0 2024-03-18T14:51:03.949 INFO:tasks.workunit.client.0.smithi012.stdout:7/722: dread dd/d13/d23/d2f/d7a/dab/fbe [0,4194304] 0 2024-03-18T14:51:03.949 INFO:tasks.workunit.client.0.smithi012.stdout:7/723: dread - dd/d13/d23/fdc zero size 2024-03-18T14:51:03.949 INFO:tasks.workunit.client.0.smithi012.stdout:7/724: read - dd/d13/d23/d2f/d53/dc6/df1/ff4 zero size 2024-03-18T14:51:03.949 INFO:tasks.workunit.client.1.smithi100.stdout:7/399: creat d5/d22/d24/f91 x:0 0 0 2024-03-18T14:51:03.955 INFO:tasks.workunit.client.0.smithi012.stdout:6/673: getdents d17/d38/d51/da8 0 2024-03-18T14:51:03.956 INFO:tasks.workunit.client.1.smithi100.stdout:6/389: mkdir d6/d7/df/d72/d65/d7a 0 2024-03-18T14:51:03.956 INFO:tasks.workunit.client.0.smithi012.stdout:1/665: sync 2024-03-18T14:51:03.957 INFO:tasks.workunit.client.1.smithi100.stdout:9/401: dwrite da/dd/f10 [0,4194304] 0 2024-03-18T14:51:03.957 INFO:tasks.workunit.client.1.smithi100.stdout:9/402: readlink l8 0 2024-03-18T14:51:03.957 INFO:tasks.workunit.client.1.smithi100.stdout:2/384: dwrite d3/d13/d40/d53/f66 [0,4194304] 0 2024-03-18T14:51:03.962 INFO:tasks.workunit.client.0.smithi012.stdout:5/756: truncate d3/d29/f7a 793458 0 2024-03-18T14:51:03.962 INFO:tasks.workunit.client.0.smithi012.stdout:3/697: mkdir dc/d30/d3a/da0/dc5/dc7/de7 0 2024-03-18T14:51:03.962 INFO:tasks.workunit.client.0.smithi012.stdout:3/698: fdatasync dc/d3c/d4c/d5f/f67 0 2024-03-18T14:51:03.962 INFO:tasks.workunit.client.0.smithi012.stdout:3/699: stat dc/f32 0 2024-03-18T14:51:03.967 INFO:tasks.workunit.client.0.smithi012.stdout:0/779: creat d1/d20/d23/d6f/d93/f100 x:0 0 0 2024-03-18T14:51:03.967 INFO:tasks.workunit.client.0.smithi012.stdout:1/666: read d8/db/d10/d16/d54/fa6 [1269818,53819] 0 2024-03-18T14:51:03.970 INFO:tasks.workunit.client.0.smithi012.stdout:7/725: getdents dd/d13/db7 0 2024-03-18T14:51:03.970 INFO:tasks.workunit.client.0.smithi012.stdout:7/726: stat dd/d13/d23/d76/d6f/db3/dc7/fde 0 2024-03-18T14:51:03.971 INFO:tasks.workunit.client.0.smithi012.stdout:7/727: dread dd/d13/d23/d2f/d7a/db6/f9b [0,4194304] 0 2024-03-18T14:51:03.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/765: rmdir d0/d6b/d9f 39 2024-03-18T14:51:03.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/766: dread - d0/d6b/fe7 zero size 2024-03-18T14:51:03.973 INFO:tasks.workunit.client.0.smithi012.stdout:2/767: fdatasync d0/d6c/fdc 0 2024-03-18T14:51:03.975 INFO:tasks.workunit.client.1.smithi100.stdout:0/497: symlink dd/d2b/d95/d79/lb7 0 2024-03-18T14:51:03.975 INFO:tasks.workunit.client.1.smithi100.stdout:0/498: write dd/d20/f5f [4919320,104412] 0 2024-03-18T14:51:03.975 INFO:tasks.workunit.client.1.smithi100.stdout:0/499: write dd/f18 [2865994,104698] 0 2024-03-18T14:51:03.975 INFO:tasks.workunit.client.1.smithi100.stdout:0/500: write dd/d20/f97 [95444,61435] 0 2024-03-18T14:51:03.975 INFO:tasks.workunit.client.1.smithi100.stdout:0/501: chown dd/d31/d91/d56/d62/d6e/c80 144162 1 2024-03-18T14:51:03.976 INFO:tasks.workunit.client.1.smithi100.stdout:2/385: dwrite d3/da/f30 [0,4194304] 0 2024-03-18T14:51:03.976 INFO:tasks.workunit.client.1.smithi100.stdout:2/386: dread - d3/d13/d40/f83 zero size 2024-03-18T14:51:03.979 INFO:tasks.workunit.client.0.smithi012.stdout:5/757: link d3/d29/d35/fea d3/dad/df2/ffc 0 2024-03-18T14:51:03.982 INFO:tasks.workunit.client.1.smithi100.stdout:7/400: creat d5/d14/d16/d44/f92 x:0 0 0 2024-03-18T14:51:03.983 INFO:tasks.workunit.client.1.smithi100.stdout:4/401: dread d9/d13/f33 [0,4194304] 0 2024-03-18T14:51:03.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/768: getdents d0/d6b/dbf/d33/d77 0 2024-03-18T14:51:03.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/769: stat d0/da2/f5c 0 2024-03-18T14:51:03.986 INFO:tasks.workunit.client.1.smithi100.stdout:6/390: symlink d6/d7/df/d26/l7b 0 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/391: write d6/d7/df/d26/f45 [1119979,24587] 0 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/392: readlink d6/d7/df/d72/d48/l69 0 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/393: creat d6/d7/df/f7c x:0 0 0 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/394: dread - d6/d7/df/d26/d4e/d66/d36/f54 zero size 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/395: fsync d6/d7/df/d26/d4e/d66/d36/d4f/d71/f13 0 2024-03-18T14:51:03.987 INFO:tasks.workunit.client.0.smithi012.stdout:7/728: dread dd/d13/d23/d60/f6c [0,4194304] 0 2024-03-18T14:51:03.992 INFO:tasks.workunit.client.0.smithi012.stdout:1/667: dwrite d8/db/d10/d16/da1/f23 [4194304,4194304] 0 2024-03-18T14:51:03.993 INFO:tasks.workunit.client.0.smithi012.stdout:0/780: dwrite d1/d20/d23/d3b/d33/d51/fa8 [0,4194304] 0 2024-03-18T14:51:03.994 INFO:tasks.workunit.client.0.smithi012.stdout:6/674: rmdir d17/d38/d51 39 2024-03-18T14:51:03.994 INFO:tasks.workunit.client.0.smithi012.stdout:6/675: fdatasync d17/d19/d27/d6e/fc0 0 2024-03-18T14:51:03.994 INFO:tasks.workunit.client.0.smithi012.stdout:6/676: creat d17/d38/fc7 x:0 0 0 2024-03-18T14:51:03.998 INFO:tasks.workunit.client.1.smithi100.stdout:8/440: mkdir d4/d96 0 2024-03-18T14:51:03.999 INFO:tasks.workunit.client.0.smithi012.stdout:4/642: sync 2024-03-18T14:51:04.000 INFO:tasks.workunit.client.0.smithi012.stdout:5/758: link d3/d1e/d33/l69 d3/d15/d42/dc6/d73/d6b/df3/lfd 0 2024-03-18T14:51:04.000 INFO:tasks.workunit.client.0.smithi012.stdout:1/668: dread d8/db/d10/d16/d54/fa6 [0,4194304] 0 2024-03-18T14:51:04.000 INFO:tasks.workunit.client.0.smithi012.stdout:1/669: fdatasync d8/db/d10/d16/d54/f93 0 2024-03-18T14:51:04.001 INFO:tasks.workunit.client.0.smithi012.stdout:4/643: dread d6/d19/d6c/f7a [0,4194304] 0 2024-03-18T14:51:04.003 INFO:tasks.workunit.client.1.smithi100.stdout:1/421: sync 2024-03-18T14:51:04.003 INFO:tasks.workunit.client.1.smithi100.stdout:1/422: read - d8/d30/f40 zero size 2024-03-18T14:51:04.003 INFO:tasks.workunit.client.1.smithi100.stdout:1/423: read d8/d1e/d11/d27/f64 [401133,20236] 0 2024-03-18T14:51:04.003 INFO:tasks.workunit.client.0.smithi012.stdout:8/915: sync 2024-03-18T14:51:04.003 INFO:tasks.workunit.client.0.smithi012.stdout:9/778: sync 2024-03-18T14:51:04.005 INFO:tasks.workunit.client.1.smithi100.stdout:7/401: symlink d5/d14/d16/d2e/d62/l93 0 2024-03-18T14:51:04.005 INFO:tasks.workunit.client.1.smithi100.stdout:7/402: chown d5/d14/f1e 34596 1 2024-03-18T14:51:04.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/403: write d5/d22/d24/d6e/f84 [88506,2803] 0 2024-03-18T14:51:04.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/404: truncate d5/d22/d24/f91 917293 0 2024-03-18T14:51:04.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/405: chown d5/d22/d3b/c6d 0 1 2024-03-18T14:51:04.006 INFO:tasks.workunit.client.1.smithi100.stdout:7/406: chown d5/d14/d16/d2e/d62 2347802 1 2024-03-18T14:51:04.006 INFO:tasks.workunit.client.0.smithi012.stdout:2/770: truncate d0/d3c/fb3 84325 0 2024-03-18T14:51:04.007 INFO:tasks.workunit.client.1.smithi100.stdout:1/424: dread d8/f29 [0,4194304] 0 2024-03-18T14:51:04.010 INFO:tasks.workunit.client.1.smithi100.stdout:4/402: link d9/d13/l30 d9/d17/l77 0 2024-03-18T14:51:04.013 INFO:tasks.workunit.client.1.smithi100.stdout:3/446: sync 2024-03-18T14:51:04.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/396: mknod d6/d7/df/d72/d65/d7a/c7d 0 2024-03-18T14:51:04.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/397: chown d6/d7/df/d72/d48/f4a 1799626 1 2024-03-18T14:51:04.018 INFO:tasks.workunit.client.0.smithi012.stdout:3/700: rmdir dc/d30/d3a 39 2024-03-18T14:51:04.018 INFO:tasks.workunit.client.0.smithi012.stdout:3/701: readlink dc/l1d 0 2024-03-18T14:51:04.022 INFO:tasks.workunit.client.1.smithi100.stdout:2/387: dwrite d3/d6/d5f/f6d [0,4194304] 0 2024-03-18T14:51:04.023 INFO:tasks.workunit.client.1.smithi100.stdout:2/388: read d3/da/f43 [148569,83791] 0 2024-03-18T14:51:04.024 INFO:tasks.workunit.client.0.smithi012.stdout:4/644: write d6/f2e [1539662,127620] 0 2024-03-18T14:51:04.024 INFO:tasks.workunit.client.0.smithi012.stdout:4/645: read d6/d1a/f29 [1856413,123397] 0 2024-03-18T14:51:04.024 INFO:tasks.workunit.client.0.smithi012.stdout:4/646: dread - d6/da/d89/d97/fa9 zero size 2024-03-18T14:51:04.027 INFO:tasks.workunit.client.0.smithi012.stdout:1/670: symlink d8/d1e/d27/d44/ld9 0 2024-03-18T14:51:04.027 INFO:tasks.workunit.client.0.smithi012.stdout:7/729: dwrite dd/d13/d23/d76/d6f/db3/ff3 [0,4194304] 0 2024-03-18T14:51:04.027 INFO:tasks.workunit.client.0.smithi012.stdout:7/730: dread - dd/d13/d23/d76/d6f/db3/fd0 zero size 2024-03-18T14:51:04.030 INFO:tasks.workunit.client.1.smithi100.stdout:8/441: dwrite d4/dc/d31/f3d [0,4194304] 0 2024-03-18T14:51:04.031 INFO:tasks.workunit.client.0.smithi012.stdout:9/779: rename d3/de6/dbf/d77/d52/d65/f86 to d3/de6/dbf/d77/d52/fff 0 2024-03-18T14:51:04.031 INFO:tasks.workunit.client.0.smithi012.stdout:9/780: truncate d3/de6/dbf/d77/d52/fff 364276 0 2024-03-18T14:51:04.032 INFO:tasks.workunit.client.1.smithi100.stdout:7/407: mknod d5/d30/c94 0 2024-03-18T14:51:04.033 INFO:tasks.workunit.client.1.smithi100.stdout:6/398: dread d6/f2f [0,4194304] 0 2024-03-18T14:51:04.033 INFO:tasks.workunit.client.0.smithi012.stdout:8/916: creat d0/d5a/d19/d63/dbe/f125 x:0 0 0 2024-03-18T14:51:04.035 INFO:tasks.workunit.client.1.smithi100.stdout:4/403: truncate d9/f6d 2803537 0 2024-03-18T14:51:04.036 INFO:tasks.workunit.client.1.smithi100.stdout:1/425: dwrite f5 [0,4194304] 0 2024-03-18T14:51:04.037 INFO:tasks.workunit.client.0.smithi012.stdout:0/781: dwrite d1/d20/d23/d3b/d33/d51/fa8 [0,4194304] 0 2024-03-18T14:51:04.037 INFO:tasks.workunit.client.0.smithi012.stdout:2/771: unlink d0/d6b/dbf/d57/d61/c92 0 2024-03-18T14:51:04.037 INFO:tasks.workunit.client.1.smithi100.stdout:3/447: dwrite d3/dd/f27 [0,4194304] 0 2024-03-18T14:51:04.039 INFO:tasks.workunit.client.0.smithi012.stdout:3/702: mknod dc/d3c/d59/d86/d9a/d54/ce8 0 2024-03-18T14:51:04.043 INFO:tasks.workunit.client.0.smithi012.stdout:4/647: mkdir d6/d19/d6c/d96/dda 0 2024-03-18T14:51:04.045 INFO:tasks.workunit.client.1.smithi100.stdout:2/389: getdents d3/d1a/d6c 0 2024-03-18T14:51:04.048 INFO:tasks.workunit.client.0.smithi012.stdout:6/677: rmdir d17/d19/d27/d6e 39 2024-03-18T14:51:04.048 INFO:tasks.workunit.client.0.smithi012.stdout:6/678: dread - d17/d38/d51/fa9 zero size 2024-03-18T14:51:04.052 INFO:tasks.workunit.client.1.smithi100.stdout:4/404: dwrite d9/d27/d42/f50 [0,4194304] 0 2024-03-18T14:51:04.052 INFO:tasks.workunit.client.1.smithi100.stdout:4/405: write d9/f1f [637002,66910] 0 2024-03-18T14:51:04.052 INFO:tasks.workunit.client.1.smithi100.stdout:4/406: chown d9/d17/l49 806528 1 2024-03-18T14:51:04.053 INFO:tasks.workunit.client.0.smithi012.stdout:1/671: creat d8/db/d10/d31/d43/d5e/d65/fda x:0 0 0 2024-03-18T14:51:04.053 INFO:tasks.workunit.client.1.smithi100.stdout:8/442: mkdir d4/dc/d97 0 2024-03-18T14:51:04.055 INFO:tasks.workunit.client.1.smithi100.stdout:7/408: creat d5/d14/d16/d2e/d7b/f95 x:0 0 0 2024-03-18T14:51:04.057 INFO:tasks.workunit.client.1.smithi100.stdout:6/399: unlink d6/d7/df/d26/d4e/d66/d36/d4f/d71/f52 0 2024-03-18T14:51:04.058 INFO:tasks.workunit.client.0.smithi012.stdout:7/731: link dd/d13/d23/d76/d6f/db3/cfc dd/d13/d23/d2f/de1/cfe 0 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/781: mknod d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/c100 0 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/782: chown d3/l8a 3416 1 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/783: stat d3/f3b 0 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/784: write d3/f2e [525542,51491] 0 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/785: chown d3/de6/dbf/d77/d52/d65/fc4 103 1 2024-03-18T14:51:04.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/786: write d3/f12 [3128775,27358] 0 2024-03-18T14:51:04.061 INFO:tasks.workunit.client.1.smithi100.stdout:1/426: unlink d8/d1e/c53 0 2024-03-18T14:51:04.062 INFO:tasks.workunit.client.0.smithi012.stdout:8/917: mkdir d0/d5a/d126 0 2024-03-18T14:51:04.065 INFO:tasks.workunit.client.0.smithi012.stdout:4/648: dwrite d6/d19/d6c/f88 [0,4194304] 0 2024-03-18T14:51:04.066 INFO:tasks.workunit.client.0.smithi012.stdout:0/782: mkdir d1/d20/d4e/d54/d8a/d101 0 2024-03-18T14:51:04.069 INFO:tasks.workunit.client.0.smithi012.stdout:2/772: truncate d0/d10/f32 2931381 0 2024-03-18T14:51:04.070 INFO:tasks.workunit.client.0.smithi012.stdout:2/773: chown d0/d6b/dbf/d33/d80 15546351 1 2024-03-18T14:51:04.071 INFO:tasks.workunit.client.1.smithi100.stdout:3/448: creat d3/fa1 x:0 0 0 2024-03-18T14:51:04.071 INFO:tasks.workunit.client.1.smithi100.stdout:3/449: creat d3/dd/d49/fa2 x:0 0 0 2024-03-18T14:51:04.071 INFO:tasks.workunit.client.1.smithi100.stdout:3/450: creat d3/d16/d13/d17/d7d/fa3 x:0 0 0 2024-03-18T14:51:04.071 INFO:tasks.workunit.client.0.smithi012.stdout:3/703: creat dc/d30/d3a/db7/dd3/fe9 x:0 0 0 2024-03-18T14:51:04.071 INFO:tasks.workunit.client.0.smithi012.stdout:3/704: truncate dc/d30/dab/fd0 893660 0 2024-03-18T14:51:04.072 INFO:tasks.workunit.client.1.smithi100.stdout:2/390: symlink d3/d1a/d76/l8a 0 2024-03-18T14:51:04.074 INFO:tasks.workunit.client.0.smithi012.stdout:6/679: rename d17/d19/d27/d6e/c99 to d17/d19/d27/db1/cc8 0 2024-03-18T14:51:04.076 INFO:tasks.workunit.client.0.smithi012.stdout:1/672: link d8/db/d10/d16/d54/fa6 d8/db/d10/d16/da1/d2e/d9c/d9d/fdb 0 2024-03-18T14:51:04.082 INFO:tasks.workunit.client.0.smithi012.stdout:3/705: write dc/d3c/d4c/f94 [1008858,40390] 0 2024-03-18T14:51:04.082 INFO:tasks.workunit.client.0.smithi012.stdout:3/706: chown dc/d3c/d72/f7b 28952727 1 2024-03-18T14:51:04.082 INFO:tasks.workunit.client.0.smithi012.stdout:3/707: chown dc/d30/d35/c39 56449 1 2024-03-18T14:51:04.082 INFO:tasks.workunit.client.0.smithi012.stdout:7/732: creat dd/d13/d23/d33/fff x:0 0 0 2024-03-18T14:51:04.082 INFO:tasks.workunit.client.0.smithi012.stdout:7/733: creat dd/d13/d23/d76/d6b/f100 x:0 0 0 2024-03-18T14:51:04.083 INFO:tasks.workunit.client.0.smithi012.stdout:7/734: dread - dd/d13/d23/d2f/fb9 zero size 2024-03-18T14:51:04.084 INFO:tasks.workunit.client.1.smithi100.stdout:8/443: truncate d4/dc/d1f/f32 598427 0 2024-03-18T14:51:04.084 INFO:tasks.workunit.client.1.smithi100.stdout:8/444: truncate d4/f90 110032 0 2024-03-18T14:51:04.086 INFO:tasks.workunit.client.1.smithi100.stdout:7/409: mknod d5/d14/d16/d2e/d62/d66/c96 0 2024-03-18T14:51:04.086 INFO:tasks.workunit.client.1.smithi100.stdout:7/410: creat d5/d14/d16/d2e/d62/f97 x:0 0 0 2024-03-18T14:51:04.087 INFO:tasks.workunit.client.0.smithi012.stdout:9/787: mknod d3/de6/dbf/d66/d69/d7f/d8c/d8b/c101 0 2024-03-18T14:51:04.091 INFO:tasks.workunit.client.0.smithi012.stdout:8/918: creat d0/d5a/d5e/ddf/d50/d114/f127 x:0 0 0 2024-03-18T14:51:04.093 INFO:tasks.workunit.client.1.smithi100.stdout:6/400: getdents d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27 0 2024-03-18T14:51:04.093 INFO:tasks.workunit.client.1.smithi100.stdout:6/401: read - d6/d7/df/f7c zero size 2024-03-18T14:51:04.093 INFO:tasks.workunit.client.1.smithi100.stdout:6/402: truncate d6/d12/f5c 1098822 0 2024-03-18T14:51:04.096 INFO:tasks.workunit.client.1.smithi100.stdout:1/427: rename d8/d1e/d11/d2d/l34 to d8/d1e/d11/d6d/l7a 0 2024-03-18T14:51:04.097 INFO:tasks.workunit.client.0.smithi012.stdout:4/649: creat d6/d1a/d30/d46/d6f/d7c/fdb x:0 0 0 2024-03-18T14:51:04.100 INFO:tasks.workunit.client.1.smithi100.stdout:2/391: dwrite d3/da/de/d48/f5d [0,4194304] 0 2024-03-18T14:51:04.100 INFO:tasks.workunit.client.1.smithi100.stdout:2/392: write d3/f4 [4092700,1279] 0 2024-03-18T14:51:04.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/673: dwrite d8/db/d10/d16/da1/d2e/d9c/fb1 [4194304,4194304] 0 2024-03-18T14:51:04.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/674: dread - d8/db/d10/d16/d6b/d7e/fad zero size 2024-03-18T14:51:04.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/675: write d8/d1e/d27/f38 [3702274,60218] 0 2024-03-18T14:51:04.103 INFO:tasks.workunit.client.0.smithi012.stdout:2/774: symlink d0/d6b/dbf/d33/d99/daa/lf2 0 2024-03-18T14:51:04.105 INFO:tasks.workunit.client.1.smithi100.stdout:5/355: sync 2024-03-18T14:51:04.110 INFO:tasks.workunit.client.0.smithi012.stdout:6/680: unlink d17/d38/fa2 0 2024-03-18T14:51:04.111 INFO:tasks.workunit.client.1.smithi100.stdout:6/403: unlink d6/d12/f3b 0 2024-03-18T14:51:04.113 INFO:tasks.workunit.client.0.smithi012.stdout:1/676: dread d8/db/d10/d16/f66 [0,4194304] 0 2024-03-18T14:51:04.113 INFO:tasks.workunit.client.0.smithi012.stdout:1/677: stat d8/d1e/f63 0 2024-03-18T14:51:04.113 INFO:tasks.workunit.client.0.smithi012.stdout:1/678: stat d8/d1e/f7c 0 2024-03-18T14:51:04.113 INFO:tasks.workunit.client.0.smithi012.stdout:1/679: fsync d8/db/d10/d31/d81/fb3 0 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/680: chown d8/db/l42 773974 1 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.0.smithi012.stdout:8/919: dwrite d0/d47/d59/dc9/d87/f121 [0,4194304] 0 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.0.smithi012.stdout:8/920: write d0/d5a/d5e/dd7/d84/d54/d62/dd0/f117 [659510,90744] 0 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.0.smithi012.stdout:8/921: fsync d0/d47/f4f 0 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.0.smithi012.stdout:8/922: readlink d0/l2 0 2024-03-18T14:51:04.114 INFO:tasks.workunit.client.1.smithi100.stdout:1/428: mknod d8/d2a/d37/d50/d6f/d71/c7b 0 2024-03-18T14:51:04.117 INFO:tasks.workunit.client.1.smithi100.stdout:3/451: unlink d3/dd/d49/fa2 0 2024-03-18T14:51:04.119 INFO:tasks.workunit.client.0.smithi012.stdout:3/708: truncate dc/d30/f81 3832771 0 2024-03-18T14:51:04.119 INFO:tasks.workunit.client.0.smithi012.stdout:3/709: creat dc/d91/fea x:0 0 0 2024-03-18T14:51:04.119 INFO:tasks.workunit.client.0.smithi012.stdout:3/710: fsync dc/d3c/d4c/f94 0 2024-03-18T14:51:04.119 INFO:tasks.workunit.client.0.smithi012.stdout:3/711: truncate dc/d3c/d59/d86/d9a/f7e 4442740 0 2024-03-18T14:51:04.120 INFO:tasks.workunit.client.1.smithi100.stdout:7/411: truncate d5/fd 1529090 0 2024-03-18T14:51:04.123 INFO:tasks.workunit.client.0.smithi012.stdout:1/681: write d8/db/d10/d31/d81/fae [3506276,19095] 0 2024-03-18T14:51:04.123 INFO:tasks.workunit.client.1.smithi100.stdout:1/429: truncate d8/d1e/f28 748876 0 2024-03-18T14:51:04.125 INFO:tasks.workunit.client.0.smithi012.stdout:7/735: link dd/d13/d23/d2f/d53/c88 dd/d13/db7/c101 0 2024-03-18T14:51:04.126 INFO:tasks.workunit.client.0.smithi012.stdout:7/736: dread - dd/d13/d23/d2f/d53/d8d/fef zero size 2024-03-18T14:51:04.126 INFO:tasks.workunit.client.0.smithi012.stdout:7/737: chown dd/d13/d23/d33/d37 1850012 1 2024-03-18T14:51:04.128 INFO:tasks.workunit.client.0.smithi012.stdout:9/788: rename d3/d49/l7a to d3/de6/dbf/d77/d52/d99/da0/l102 0 2024-03-18T14:51:04.129 INFO:tasks.workunit.client.1.smithi100.stdout:7/412: mknod d5/d14/d16/d2e/c98 0 2024-03-18T14:51:04.129 INFO:tasks.workunit.client.1.smithi100.stdout:7/413: truncate d5/d22/d24/d6e/d7a/f89 1596442 0 2024-03-18T14:51:04.129 INFO:tasks.workunit.client.1.smithi100.stdout:3/452: dread d3/d16/d13/d1f/d3f/d4f/f51 [0,4194304] 0 2024-03-18T14:51:04.130 INFO:tasks.workunit.client.0.smithi012.stdout:4/650: mkdir d6/da/d89/da1/dbb/ddc 0 2024-03-18T14:51:04.132 INFO:tasks.workunit.client.1.smithi100.stdout:1/430: rename d8/d1e/d11/d27/d3e/l4c to d8/l7c 0 2024-03-18T14:51:04.134 INFO:tasks.workunit.client.1.smithi100.stdout:7/414: symlink d5/d14/d16/d2c/d86/l99 0 2024-03-18T14:51:04.134 INFO:tasks.workunit.client.1.smithi100.stdout:7/415: fsync d5/d14/d16/d2e/d62/f8c 0 2024-03-18T14:51:04.135 INFO:tasks.workunit.client.1.smithi100.stdout:7/416: readlink d5/d14/d16/l68 0 2024-03-18T14:51:04.137 INFO:tasks.workunit.client.0.smithi012.stdout:9/789: dread d3/de6/f31 [0,4194304] 0 2024-03-18T14:51:04.139 INFO:tasks.workunit.client.0.smithi012.stdout:9/790: dread d3/de6/dbf/f4d [0,4194304] 0 2024-03-18T14:51:04.139 INFO:tasks.workunit.client.1.smithi100.stdout:5/356: dwrite d29/f50 [0,4194304] 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:5/357: dread - d29/f57 zero size 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:6/404: dwrite d6/d12/f43 [0,4194304] 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:6/405: creat d6/d12/d2c/f7e x:0 0 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:6/406: write d6/d7/df/f1e [892777,106942] 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:2/393: dwrite d3/d13/f38 [0,4194304] 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:3/453: dread d3/d16/d13/d1f/d3f/d4f/d60/f6b [0,4194304] 0 2024-03-18T14:51:04.140 INFO:tasks.workunit.client.1.smithi100.stdout:2/394: truncate d3/da/de/f28 2866115 0 2024-03-18T14:51:04.144 INFO:tasks.workunit.client.1.smithi100.stdout:1/431: write d8/d1e/d11/f21 [460822,70790] 0 2024-03-18T14:51:04.144 INFO:tasks.workunit.client.1.smithi100.stdout:1/432: fsync d8/f52 0 2024-03-18T14:51:04.144 INFO:tasks.workunit.client.1.smithi100.stdout:1/433: readlink d8/d1e/d11/d6d/l7a 0 2024-03-18T14:51:04.145 INFO:tasks.workunit.client.1.smithi100.stdout:6/407: write d6/d7/df/d72/f33 [956501,112916] 0 2024-03-18T14:51:04.145 INFO:tasks.workunit.client.1.smithi100.stdout:6/408: write d6/d12/f3a [9013209,67814] 0 2024-03-18T14:51:04.146 INFO:tasks.workunit.client.0.smithi012.stdout:9/791: write d3/de6/dbf/d77/d52/f6d [2929649,73881] 0 2024-03-18T14:51:04.146 INFO:tasks.workunit.client.0.smithi012.stdout:9/792: stat d3/de6/dbf/d66/l37 0 2024-03-18T14:51:04.146 INFO:tasks.workunit.client.0.smithi012.stdout:9/793: fsync d3/f4 0 2024-03-18T14:51:04.147 INFO:tasks.workunit.client.0.smithi012.stdout:2/775: creat d0/d6b/dbf/ff3 x:0 0 0 2024-03-18T14:51:04.149 INFO:tasks.workunit.client.1.smithi100.stdout:7/417: mknod d5/d14/d16/d63/c9a 0 2024-03-18T14:51:04.156 INFO:tasks.workunit.client.0.smithi012.stdout:8/923: mknod d0/d5a/d5e/dd7/d84/d2a/d3e/d119/c128 0 2024-03-18T14:51:04.158 INFO:tasks.workunit.client.0.smithi012.stdout:8/924: dread d0/d5a/d19/d57/f69 [0,4194304] 0 2024-03-18T14:51:04.164 INFO:tasks.workunit.client.0.smithi012.stdout:3/712: mkdir dc/d30/d3a/da0/dc5/deb 0 2024-03-18T14:51:04.169 INFO:tasks.workunit.client.0.smithi012.stdout:1/682: creat d8/db/d10/d31/d43/d5e/d65/d84/dc1/fdc x:0 0 0 2024-03-18T14:51:04.171 INFO:tasks.workunit.client.1.smithi100.stdout:1/434: dwrite d8/d2a/d37/f5d [0,4194304] 0 2024-03-18T14:51:04.171 INFO:tasks.workunit.client.1.smithi100.stdout:6/409: dwrite d6/d7/df/d72/d48/f4a [4194304,4194304] 0 2024-03-18T14:51:04.172 INFO:tasks.workunit.client.0.smithi012.stdout:7/738: rename dd/f5a to dd/d13/d23/d76/f102 0 2024-03-18T14:51:04.175 INFO:tasks.workunit.client.0.smithi012.stdout:4/651: creat d6/d19/da5/fdd x:0 0 0 2024-03-18T14:51:04.175 INFO:tasks.workunit.client.0.smithi012.stdout:4/652: chown d6/d1a/d30/d46/d6f/d7c/fdb 1034501 1 2024-03-18T14:51:04.176 INFO:tasks.workunit.client.1.smithi100.stdout:3/454: truncate d3/ff 105047 0 2024-03-18T14:51:04.176 INFO:tasks.workunit.client.0.smithi012.stdout:3/713: write dc/d3c/d59/d86/d9a/d54/f58 [3669661,34401] 0 2024-03-18T14:51:04.178 INFO:tasks.workunit.client.0.smithi012.stdout:9/794: symlink d3/de6/dbf/d77/d52/d95/deb/l103 0 2024-03-18T14:51:04.179 INFO:tasks.workunit.client.0.smithi012.stdout:9/795: dread d3/de6/dbf/d66/d21/d35/fb9 [0,4194304] 0 2024-03-18T14:51:04.179 INFO:tasks.workunit.client.0.smithi012.stdout:9/796: write d3/f6 [2101724,85374] 0 2024-03-18T14:51:04.179 INFO:tasks.workunit.client.0.smithi012.stdout:9/797: write d3/de6/dbf/d66/d6e/dc6/ff2 [1041563,117437] 0 2024-03-18T14:51:04.180 INFO:tasks.workunit.client.1.smithi100.stdout:2/395: rename d3/d6/d29/f3d to d3/d1a/d6c/f8b 0 2024-03-18T14:51:04.181 INFO:tasks.workunit.client.1.smithi100.stdout:2/396: dread d3/da/f33 [0,4194304] 0 2024-03-18T14:51:04.182 INFO:tasks.workunit.client.0.smithi012.stdout:8/925: symlink d0/d5a/d5e/dd7/d84/d54/da4/l129 0 2024-03-18T14:51:04.183 INFO:tasks.workunit.client.0.smithi012.stdout:8/926: creat d0/d47/f12a x:0 0 0 2024-03-18T14:51:04.187 INFO:tasks.workunit.client.0.smithi012.stdout:1/683: link d8/db/d10/d16/da1/db9/ccd d8/db/d10/d31/d43/d5e/d65/d84/cdd 0 2024-03-18T14:51:04.188 INFO:tasks.workunit.client.1.smithi100.stdout:6/410: dread d6/d7/df/f55 [0,4194304] 0 2024-03-18T14:51:04.188 INFO:tasks.workunit.client.1.smithi100.stdout:6/411: stat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f70 0 2024-03-18T14:51:04.193 INFO:tasks.workunit.client.0.smithi012.stdout:2/776: rename d0/d6b/dbf/d31/l78 to d0/d6b/dbf/d33/d80/de1/lf4 0 2024-03-18T14:51:04.195 INFO:tasks.workunit.client.0.smithi012.stdout:7/739: mkdir dd/d13/d103 0 2024-03-18T14:51:04.198 INFO:tasks.workunit.client.1.smithi100.stdout:1/435: unlink d8/d30/f42 0 2024-03-18T14:51:04.199 INFO:tasks.workunit.client.1.smithi100.stdout:1/436: creat d8/d2a/d37/f7d x:0 0 0 2024-03-18T14:51:04.199 INFO:tasks.workunit.client.0.smithi012.stdout:3/714: dwrite dc/d3c/d59/d86/fa8 [0,4194304] 0 2024-03-18T14:51:04.201 INFO:tasks.workunit.client.0.smithi012.stdout:4/653: creat d6/d1a/d30/d46/fde x:0 0 0 2024-03-18T14:51:04.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/654: fdatasync d6/d1a/d30/d46/d6f/d8b/fb0 0 2024-03-18T14:51:04.202 INFO:tasks.workunit.client.0.smithi012.stdout:4/655: truncate d6/da/d58/fbd 1017708 0 2024-03-18T14:51:04.202 INFO:tasks.workunit.client.1.smithi100.stdout:7/418: dread d5/d14/d16/d2c/f5b [0,4194304] 0 2024-03-18T14:51:04.205 INFO:tasks.workunit.client.1.smithi100.stdout:5/358: write d29/f50 [4854805,42868] 0 2024-03-18T14:51:04.207 INFO:tasks.workunit.client.1.smithi100.stdout:6/412: dwrite d6/d7/df/d26/f45 [0,4194304] 0 2024-03-18T14:51:04.207 INFO:tasks.workunit.client.1.smithi100.stdout:6/413: chown d6/d12/d2c 22709728 1 2024-03-18T14:51:04.207 INFO:tasks.workunit.client.0.smithi012.stdout:9/798: symlink d3/de6/dc1/l104 0 2024-03-18T14:51:04.210 INFO:tasks.workunit.client.0.smithi012.stdout:8/927: getdents d0/d5a 0 2024-03-18T14:51:04.211 INFO:tasks.workunit.client.1.smithi100.stdout:9/403: sync 2024-03-18T14:51:04.214 INFO:tasks.workunit.client.0.smithi012.stdout:2/777: mkdir d0/d6b/dbf/d33/d77/df5 0 2024-03-18T14:51:04.214 INFO:tasks.workunit.client.0.smithi012.stdout:2/778: fdatasync d0/d6b/fc2 0 2024-03-18T14:51:04.214 INFO:tasks.workunit.client.0.smithi012.stdout:2/779: truncate d0/d6b/dbf/d33/f37 556039 0 2024-03-18T14:51:04.214 INFO:tasks.workunit.client.0.smithi012.stdout:2/780: truncate d0/d6b/dbf/d57/fd7 229148 0 2024-03-18T14:51:04.216 INFO:tasks.workunit.client.0.smithi012.stdout:7/740: mknod dd/d13/d23/d76/d6f/db3/c104 0 2024-03-18T14:51:04.216 INFO:tasks.workunit.client.0.smithi012.stdout:7/741: write dd/d13/d23/d2f/d7a/db6/f9b [455644,123637] 0 2024-03-18T14:51:04.216 INFO:tasks.workunit.client.1.smithi100.stdout:5/359: dread f15 [0,4194304] 0 2024-03-18T14:51:04.216 INFO:tasks.workunit.client.1.smithi100.stdout:5/360: chown d29/d4a/c4d 2 1 2024-03-18T14:51:04.219 INFO:tasks.workunit.client.1.smithi100.stdout:3/455: symlink d3/d16/d13/d17/la4 0 2024-03-18T14:51:04.220 INFO:tasks.workunit.client.0.smithi012.stdout:9/799: write d3/d49/f57 [2437050,110633] 0 2024-03-18T14:51:04.222 INFO:tasks.workunit.client.1.smithi100.stdout:2/397: rmdir d3/d1a/d76 39 2024-03-18T14:51:04.223 INFO:tasks.workunit.client.0.smithi012.stdout:1/684: dwrite d8/db/d10/d16/d6b/d7e/fd6 [0,4194304] 0 2024-03-18T14:51:04.223 INFO:tasks.workunit.client.1.smithi100.stdout:0/502: sync 2024-03-18T14:51:04.223 INFO:tasks.workunit.client.1.smithi100.stdout:0/503: write dd/d20/f27 [828998,82955] 0 2024-03-18T14:51:04.223 INFO:tasks.workunit.client.1.smithi100.stdout:0/504: readlink dd/d2b/l7a 0 2024-03-18T14:51:04.225 INFO:tasks.workunit.client.0.smithi012.stdout:4/656: dwrite d6/d19/fad [0,4194304] 0 2024-03-18T14:51:04.228 INFO:tasks.workunit.client.1.smithi100.stdout:7/419: rmdir d5/d22/d3b 39 2024-03-18T14:51:04.235 INFO:tasks.workunit.client.0.smithi012.stdout:3/715: mknod dc/d30/cec 0 2024-03-18T14:51:04.237 INFO:tasks.workunit.client.1.smithi100.stdout:9/404: dwrite da/dd/f28 [0,4194304] 0 2024-03-18T14:51:04.238 INFO:tasks.workunit.client.1.smithi100.stdout:9/405: rename da/d25 to da/d25/d7a 22 2024-03-18T14:51:04.241 INFO:tasks.workunit.client.0.smithi012.stdout:8/928: symlink d0/d5a/d5e/ddf/l12b 0 2024-03-18T14:51:04.241 INFO:tasks.workunit.client.0.smithi012.stdout:8/929: truncate d0/d15/f94 1731490 0 2024-03-18T14:51:04.241 INFO:tasks.workunit.client.0.smithi012.stdout:8/930: write d0/d47/d59/fca [78035,39199] 0 2024-03-18T14:51:04.244 INFO:tasks.workunit.client.1.smithi100.stdout:6/414: truncate f4 6639800 0 2024-03-18T14:51:04.244 INFO:tasks.workunit.client.1.smithi100.stdout:6/415: readlink d6/d7/df/d26/d4e/d66/l51 0 2024-03-18T14:51:04.244 INFO:tasks.workunit.client.1.smithi100.stdout:6/416: stat d6/d7/df/d26/d4e/d66/d36 0 2024-03-18T14:51:04.245 INFO:tasks.workunit.client.0.smithi012.stdout:2/781: mkdir d0/d6b/dbf/d57/df6 0 2024-03-18T14:51:04.251 INFO:tasks.workunit.client.1.smithi100.stdout:5/361: creat d29/d2d/d39/d65/f6a x:0 0 0 2024-03-18T14:51:04.252 INFO:tasks.workunit.client.0.smithi012.stdout:7/742: truncate dd/d13/d23/d76/f3e 1005939 0 2024-03-18T14:51:04.253 INFO:tasks.workunit.client.1.smithi100.stdout:2/398: truncate d3/d6/d29/f35 1980358 0 2024-03-18T14:51:04.257 INFO:tasks.workunit.client.1.smithi100.stdout:0/505: mkdir dd/d31/d91/db8 0 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/407: sync 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:8/445: sync 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/408: write d9/d13/d25/d51/f5d [535311,98244] 0 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/409: chown d9/l15 38651868 1 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/410: creat d9/d13/d5a/f78 x:0 0 0 2024-03-18T14:51:04.259 INFO:tasks.workunit.client.1.smithi100.stdout:4/411: chown d9/d13/d5a/f65 981948 1 2024-03-18T14:51:04.263 INFO:tasks.workunit.client.1.smithi100.stdout:7/420: mkdir d5/d22/d3b/d9b 0 2024-03-18T14:51:04.263 INFO:tasks.workunit.client.1.smithi100.stdout:7/421: creat d5/f9c x:0 0 0 2024-03-18T14:51:04.264 INFO:tasks.workunit.client.1.smithi100.stdout:6/417: mkdir d6/d7/df/d26/d4e/d66/d36/d4f/d71/d7f 0 2024-03-18T14:51:04.267 INFO:tasks.workunit.client.1.smithi100.stdout:6/418: dread d6/d12/d2c/f4c [0,4194304] 0 2024-03-18T14:51:04.268 INFO:tasks.workunit.client.1.smithi100.stdout:5/362: rename d29/f2a to d29/d2d/d39/d61/f6b 0 2024-03-18T14:51:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:1/685: dwrite d8/db/d10/fb0 [0,4194304] 0 2024-03-18T14:51:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:1/686: creat d8/db/d10/d31/d43/d5e/d65/d84/fde x:0 0 0 2024-03-18T14:51:04.270 INFO:tasks.workunit.client.1.smithi100.stdout:2/399: mkdir d3/d8c 0 2024-03-18T14:51:04.273 INFO:tasks.workunit.client.1.smithi100.stdout:3/456: dwrite d3/d16/d13/d17/f33 [0,4194304] 0 2024-03-18T14:51:04.273 INFO:tasks.workunit.client.1.smithi100.stdout:3/457: read - d3/d16/d13/d17/f96 zero size 2024-03-18T14:51:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/657: dwrite d6/d19/d39/f41 [8388608,4194304] 0 2024-03-18T14:51:04.275 INFO:tasks.workunit.client.1.smithi100.stdout:0/506: symlink dd/d31/d91/db8/lb9 0 2024-03-18T14:51:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:3/716: mknod dc/d3c/d59/d86/d9a/dd7/ced 0 2024-03-18T14:51:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:3/717: fsync dc/d30/d35/f70 0 2024-03-18T14:51:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:3/718: dread - dc/d3c/d59/d86/fdc zero size 2024-03-18T14:51:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:3/719: fsync dc/d3c/d59/d86/d9a/f9b 0 2024-03-18T14:51:04.277 INFO:tasks.workunit.client.0.smithi012.stdout:3/720: write dc/d30/d35/f75 [867579,99127] 0 2024-03-18T14:51:04.278 INFO:tasks.workunit.client.0.smithi012.stdout:2/782: mknod d0/d6b/dbf/d33/d71/de5/ded/cf7 0 2024-03-18T14:51:04.278 INFO:tasks.workunit.client.0.smithi012.stdout:2/783: dread - d0/d6b/dbf/d33/d80/fe9 zero size 2024-03-18T14:51:04.281 INFO:tasks.workunit.client.1.smithi100.stdout:4/412: mkdir d9/d13/d39/d79 0 2024-03-18T14:51:04.281 INFO:tasks.workunit.client.0.smithi012.stdout:9/800: dwrite d3/f4 [0,4194304] 0 2024-03-18T14:51:04.281 INFO:tasks.workunit.client.0.smithi012.stdout:7/743: dwrite dd/d13/d23/d33/fff [0,4194304] 0 2024-03-18T14:51:04.282 INFO:tasks.workunit.client.1.smithi100.stdout:7/422: dwrite d5/d14/d16/f1d [0,4194304] 0 2024-03-18T14:51:04.282 INFO:tasks.workunit.client.1.smithi100.stdout:9/406: dwrite da/d51/f52 [0,4194304] 0 2024-03-18T14:51:04.286 INFO:tasks.workunit.client.1.smithi100.stdout:5/363: symlink d29/d2d/d32/d38/l6c 0 2024-03-18T14:51:04.287 INFO:tasks.workunit.client.0.smithi012.stdout:9/801: dread d3/de6/dbf/d66/d21/f46 [0,4194304] 0 2024-03-18T14:51:04.288 INFO:tasks.workunit.client.0.smithi012.stdout:7/744: write dd/d13/d23/d60/f6c [3280446,41384] 0 2024-03-18T14:51:04.289 INFO:tasks.workunit.client.0.smithi012.stdout:7/745: dread dd/d13/d23/d76/fb2 [0,4194304] 0 2024-03-18T14:51:04.289 INFO:tasks.workunit.client.1.smithi100.stdout:8/446: rename d4/dc/f2d to d4/dc/d1f/d59/d6c/f98 0 2024-03-18T14:51:04.291 INFO:tasks.workunit.client.0.smithi012.stdout:1/687: truncate d8/db/d10/d31/d43/d5e/f62 1532674 0 2024-03-18T14:51:04.291 INFO:tasks.workunit.client.1.smithi100.stdout:5/364: dread d29/d2d/d39/f4e [0,4194304] 0 2024-03-18T14:51:04.292 INFO:tasks.workunit.client.1.smithi100.stdout:2/400: symlink d3/d6/d29/d36/d49/l8d 0 2024-03-18T14:51:04.293 INFO:tasks.workunit.client.0.smithi012.stdout:4/658: symlink d6/d27/ldf 0 2024-03-18T14:51:04.294 INFO:tasks.workunit.client.0.smithi012.stdout:3/721: dwrite f3 [0,4194304] 0 2024-03-18T14:51:04.294 INFO:tasks.workunit.client.0.smithi012.stdout:3/722: write f6 [5151360,47529] 0 2024-03-18T14:51:04.294 INFO:tasks.workunit.client.0.smithi012.stdout:3/723: fdatasync dc/d3c/d59/fbc 0 2024-03-18T14:51:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:03 smithi100 ceph-mon[37807]: pgmap v6: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail 2024-03-18T14:51:04.298 INFO:tasks.workunit.client.0.smithi012.stdout:2/784: mknod d0/d6b/dbf/d33/d80/de1/cf8 0 2024-03-18T14:51:04.298 INFO:tasks.workunit.client.0.smithi012.stdout:2/785: write d0/d6b/dbf/d57/fd7 [1090931,42903] 0 2024-03-18T14:51:04.300 INFO:tasks.workunit.client.1.smithi100.stdout:0/507: truncate dd/d31/d91/d65/f74 230313 0 2024-03-18T14:51:04.304 INFO:tasks.workunit.client.1.smithi100.stdout:5/365: dread f18 [0,4194304] 0 2024-03-18T14:51:04.307 INFO:tasks.workunit.client.1.smithi100.stdout:7/423: mknod d5/d22/d24/d6e/d7a/c9d 0 2024-03-18T14:51:04.309 INFO:tasks.workunit.client.1.smithi100.stdout:2/401: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-18T14:51:04.309 INFO:tasks.workunit.client.1.smithi100.stdout:2/402: creat d3/da/de/d48/f8e x:0 0 0 2024-03-18T14:51:04.314 INFO:tasks.workunit.client.1.smithi100.stdout:9/407: unlink da/d2c/d2f/c79 0 2024-03-18T14:51:04.318 INFO:tasks.workunit.client.1.smithi100.stdout:5/366: dwrite d29/d2d/d32/f5c [0,4194304] 0 2024-03-18T14:51:04.320 INFO:tasks.workunit.client.0.smithi012.stdout:9/802: mknod d3/de6/dbf/d66/d69/d7f/c105 0 2024-03-18T14:51:04.321 INFO:tasks.workunit.client.1.smithi100.stdout:6/419: rmdir d6/d7/df/d26/d4e/d66/d36 39 2024-03-18T14:51:04.324 INFO:tasks.workunit.client.1.smithi100.stdout:8/447: link c2 d4/dc/dd/de/d21/c99 0 2024-03-18T14:51:04.327 INFO:tasks.workunit.client.0.smithi012.stdout:7/746: rmdir dd/d13/d23/d33/d48 39 2024-03-18T14:51:04.329 INFO:tasks.workunit.client.0.smithi012.stdout:1/688: symlink d8/db/d10/d31/ldf 0 2024-03-18T14:51:04.330 INFO:tasks.workunit.client.1.smithi100.stdout:3/458: mkdir d3/da5 0 2024-03-18T14:51:04.330 INFO:tasks.workunit.client.0.smithi012.stdout:4/659: mknod d6/da/d89/d94/da2/ce0 0 2024-03-18T14:51:04.332 INFO:tasks.workunit.client.1.smithi100.stdout:0/508: symlink dd/d2b/lba 0 2024-03-18T14:51:04.337 INFO:tasks.workunit.client.1.smithi100.stdout:6/420: dwrite d6/d12/f1f [4194304,4194304] 0 2024-03-18T14:51:04.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/724: unlink dc/d30/f98 0 2024-03-18T14:51:04.338 INFO:tasks.workunit.client.0.smithi012.stdout:3/725: dread - dc/d3c/d4c/fc6 zero size 2024-03-18T14:51:04.339 INFO:tasks.workunit.client.0.smithi012.stdout:3/726: dread dc/d91/fbb [0,4194304] 0 2024-03-18T14:51:04.339 INFO:tasks.workunit.client.0.smithi012.stdout:3/727: chown dc/fcb 9429532 1 2024-03-18T14:51:04.339 INFO:tasks.workunit.client.0.smithi012.stdout:3/728: fdatasync dc/f14 0 2024-03-18T14:51:04.339 INFO:tasks.workunit.client.0.smithi012.stdout:3/729: fsync dc/f22 0 2024-03-18T14:51:04.345 INFO:tasks.workunit.client.0.smithi012.stdout:2/786: symlink d0/d6c/lf9 0 2024-03-18T14:51:04.345 INFO:tasks.workunit.client.0.smithi012.stdout:2/787: fsync d0/d3c/d40/d47/d85/fba 0 2024-03-18T14:51:04.345 INFO:tasks.workunit.client.0.smithi012.stdout:2/788: write d0/d6b/dbf/f73 [2864230,97424] 0 2024-03-18T14:51:04.348 INFO:tasks.workunit.client.0.smithi012.stdout:9/803: link d3/de6/dbf/d77/d52/d65/cc8 d3/de6/dbf/d77/d52/d95/da3/c106 0 2024-03-18T14:51:04.348 INFO:tasks.workunit.client.0.smithi012.stdout:9/804: readlink d3/de6/dbf/d77/d52/lf7 0 2024-03-18T14:51:04.349 INFO:tasks.workunit.client.0.smithi012.stdout:7/747: getdents dd/d13/d23/d76/d6f/dd9/ddd 0 2024-03-18T14:51:04.351 INFO:tasks.workunit.client.0.smithi012.stdout:1/689: unlink d8/db/d10/d16/d6b/d7e/c90 0 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/789: creat d0/d3c/d40/d47/ffa x:0 0 0 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/790: readlink d0/l1d 0 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/791: dread - d0/d3c/d40/d47/d85/fdb zero size 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/792: chown d0/d6b/dbf/d4f/fe6 10555 1 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/793: chown d0/d6b/dbf/d33/d71/d90/l93 31 1 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:2/794: creat d0/d3c/ffb x:0 0 0 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:0/783: sync 2024-03-18T14:51:04.354 INFO:tasks.workunit.client.0.smithi012.stdout:5/759: sync 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:1/690: mknod d8/db/d10/d56/dcc/ce0 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:1/691: truncate d8/db/d10/d16/d6b/d7e/fad 241166 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/730: dwrite dc/fcb [0,4194304] 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/731: fsync dc/d30/dab/fb8 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/732: write dc/d30/f3f [1246878,25938] 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/733: fsync dc/d30/f99 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:3/734: write dc/d3c/d59/d86/d9a/fd5 [978497,119294] 0 2024-03-18T14:51:04.358 INFO:tasks.workunit.client.0.smithi012.stdout:4/660: dwrite d6/d1a/d30/d46/d6f/f71 [0,4194304] 0 2024-03-18T14:51:04.363 INFO:tasks.workunit.client.0.smithi012.stdout:7/748: dread dd/d13/d23/f27 [0,4194304] 0 2024-03-18T14:51:04.363 INFO:tasks.workunit.client.0.smithi012.stdout:2/795: getdents d0/d6b/dbf/d33/dac/dd0 0 2024-03-18T14:51:04.364 INFO:tasks.workunit.client.0.smithi012.stdout:9/805: dread d3/de6/dbf/d66/d69/d7f/d8c/d8b/f9f [0,4194304] 0 2024-03-18T14:51:04.365 INFO:tasks.workunit.client.0.smithi012.stdout:2/796: write d0/d6b/dbf/d57/d61/f6d [1380,36923] 0 2024-03-18T14:51:04.365 INFO:tasks.workunit.client.1.smithi100.stdout:4/413: rmdir d9 39 2024-03-18T14:51:04.365 INFO:tasks.workunit.client.1.smithi100.stdout:4/414: read d9/d17/f24 [2631208,122374] 0 2024-03-18T14:51:04.366 INFO:tasks.workunit.client.0.smithi012.stdout:0/784: chown d1/d20/d23/d3b/f2c 321871 1 2024-03-18T14:51:04.366 INFO:tasks.workunit.client.0.smithi012.stdout:0/785: dread - d1/d20/d23/d6f/d93/f100 zero size 2024-03-18T14:51:04.367 INFO:tasks.workunit.client.1.smithi100.stdout:7/424: rmdir d5/d14/d16 39 2024-03-18T14:51:04.372 INFO:tasks.workunit.client.0.smithi012.stdout:3/735: getdents dc/d30/dab 0 2024-03-18T14:51:04.376 INFO:tasks.workunit.client.1.smithi100.stdout:2/403: rename d3/d6/d29/d36/d4a/f56 to d3/d13/d40/d53/f8f 0 2024-03-18T14:51:04.378 INFO:tasks.workunit.client.0.smithi012.stdout:1/692: dwrite d8/db/f25 [0,4194304] 0 2024-03-18T14:51:04.378 INFO:tasks.workunit.client.0.smithi012.stdout:1/693: fsync d8/db/d10/d31/d45/f6d 0 2024-03-18T14:51:04.380 INFO:tasks.workunit.client.1.smithi100.stdout:9/408: truncate da/dd/d3c/f59 3856513 0 2024-03-18T14:51:04.381 INFO:tasks.workunit.client.1.smithi100.stdout:2/404: dread d3/d1a/f2d [0,4194304] 0 2024-03-18T14:51:04.381 INFO:tasks.workunit.client.1.smithi100.stdout:2/405: readlink d3/d13/l14 0 2024-03-18T14:51:04.382 INFO:tasks.workunit.client.1.smithi100.stdout:2/406: fsync d3/d6/f2f 0 2024-03-18T14:51:04.382 INFO:tasks.workunit.client.0.smithi012.stdout:4/661: creat d6/d19/d6c/fe1 x:0 0 0 2024-03-18T14:51:04.386 INFO:tasks.workunit.client.0.smithi012.stdout:7/749: getdents dd/d13/d23/d2f/d7a/db6 0 2024-03-18T14:51:04.388 INFO:tasks.workunit.client.1.smithi100.stdout:2/407: dread d3/d6/f44 [0,4194304] 0 2024-03-18T14:51:04.388 INFO:tasks.workunit.client.1.smithi100.stdout:5/367: creat d29/d2d/d39/d60/f6d x:0 0 0 2024-03-18T14:51:04.389 INFO:tasks.workunit.client.1.smithi100.stdout:5/368: dread f25 [0,4194304] 0 2024-03-18T14:51:04.391 INFO:tasks.workunit.client.1.smithi100.stdout:8/448: link d4/dc/dd/de/d21/d23/d25/l3e d4/dc/d1f/d59/d6c/d7c/l9a 0 2024-03-18T14:51:04.391 INFO:tasks.workunit.client.1.smithi100.stdout:8/449: write d4/dc/d1f/f32 [1441842,4825] 0 2024-03-18T14:51:04.391 INFO:tasks.workunit.client.1.smithi100.stdout:8/450: fsync d4/dc/f8a 0 2024-03-18T14:51:04.392 INFO:tasks.workunit.client.1.smithi100.stdout:1/437: sync 2024-03-18T14:51:04.393 INFO:tasks.workunit.client.1.smithi100.stdout:1/438: chown d8/d1e/l57 9 1 2024-03-18T14:51:04.393 INFO:tasks.workunit.client.0.smithi012.stdout:2/797: symlink d0/d6b/dbf/d33/d99/daa/lfc 0 2024-03-18T14:51:04.394 INFO:tasks.workunit.client.1.smithi100.stdout:0/509: unlink dd/d2b/d95/d79/lb7 0 2024-03-18T14:51:04.397 INFO:tasks.workunit.client.1.smithi100.stdout:6/421: link d6/f6c d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 0 2024-03-18T14:51:04.401 INFO:tasks.workunit.client.1.smithi100.stdout:4/415: creat d9/d13/d39/d4c/f7a x:0 0 0 2024-03-18T14:51:04.404 INFO:tasks.workunit.client.1.smithi100.stdout:7/425: mknod d5/d22/d3b/d47/c9e 0 2024-03-18T14:51:04.407 INFO:tasks.workunit.client.0.smithi012.stdout:7/750: dwrite dd/d13/d23/d2f/f9f [0,4194304] 0 2024-03-18T14:51:04.408 INFO:tasks.workunit.client.1.smithi100.stdout:3/459: rename d3/dd/d21/d3e/f75 to d3/dd/d21/fa6 0 2024-03-18T14:51:04.408 INFO:tasks.workunit.client.1.smithi100.stdout:3/460: dread - d3/d16/d13/d17/d7d/f82 zero size 2024-03-18T14:51:04.410 INFO:tasks.workunit.client.1.smithi100.stdout:2/408: creat d3/da/de/d20/f90 x:0 0 0 2024-03-18T14:51:04.411 INFO:tasks.workunit.client.1.smithi100.stdout:0/510: dread dd/d2b/f4c [4194304,4194304] 0 2024-03-18T14:51:04.411 INFO:tasks.workunit.client.1.smithi100.stdout:0/511: chown dd/d31/d91/l50 83 1 2024-03-18T14:51:04.413 INFO:tasks.workunit.client.1.smithi100.stdout:4/416: mkdir d9/d27/d62/d7b 0 2024-03-18T14:51:04.414 INFO:tasks.workunit.client.1.smithi100.stdout:3/461: dread d3/dd/f24 [0,4194304] 0 2024-03-18T14:51:04.414 INFO:tasks.workunit.client.1.smithi100.stdout:3/462: truncate d3/d16/d13/d17/f2b 1095238 0 2024-03-18T14:51:04.414 INFO:tasks.workunit.client.0.smithi012.stdout:1/694: symlink d8/dbf/le1 0 2024-03-18T14:51:04.415 INFO:tasks.workunit.client.1.smithi100.stdout:7/426: mkdir d5/d22/d3b/d9f 0 2024-03-18T14:51:04.417 INFO:tasks.workunit.client.1.smithi100.stdout:7/427: dread d5/d22/d3b/d47/f49 [0,4194304] 0 2024-03-18T14:51:04.417 INFO:tasks.workunit.client.1.smithi100.stdout:9/409: rename da/f1d to da/d51/d55/f7b 0 2024-03-18T14:51:04.418 INFO:tasks.workunit.client.1.smithi100.stdout:7/428: dread d5/d22/d3b/d47/f49 [0,4194304] 0 2024-03-18T14:51:04.419 INFO:tasks.workunit.client.0.smithi012.stdout:9/806: link d3/de6/dbf/d66/c42 d3/de6/c107 0 2024-03-18T14:51:04.419 INFO:tasks.workunit.client.1.smithi100.stdout:0/512: symlink dd/d2b/d95/d79/lbb 0 2024-03-18T14:51:04.419 INFO:tasks.workunit.client.1.smithi100.stdout:0/513: dread - dd/d2b/d95/d79/d7d/da9/db0/fb6 zero size 2024-03-18T14:51:04.420 INFO:tasks.workunit.client.0.smithi012.stdout:2/798: symlink d0/d6b/dbf/d33/d77/df5/lfd 0 2024-03-18T14:51:04.420 INFO:tasks.workunit.client.0.smithi012.stdout:2/799: read d0/d6b/dbf/f27 [2554026,104877] 0 2024-03-18T14:51:04.420 INFO:tasks.workunit.client.1.smithi100.stdout:8/451: getdents d4/dc/d31 0 2024-03-18T14:51:04.421 INFO:tasks.workunit.client.1.smithi100.stdout:4/417: getdents d9/d13/d5a 0 2024-03-18T14:51:04.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/422: dwrite d6/d12/f43 [0,4194304] 0 2024-03-18T14:51:04.422 INFO:tasks.workunit.client.0.smithi012.stdout:7/751: mkdir dd/d13/d23/d105 0 2024-03-18T14:51:04.422 INFO:tasks.workunit.client.0.smithi012.stdout:7/752: chown dd/d13/d23/d2f/d7a 0 1 2024-03-18T14:51:04.423 INFO:tasks.workunit.client.1.smithi100.stdout:5/369: dwrite f23 [0,4194304] 0 2024-03-18T14:51:04.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/463: creat d3/dd/d21/fa7 x:0 0 0 2024-03-18T14:51:04.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/464: truncate d3/dd/f2a 1786259 0 2024-03-18T14:51:04.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/465: dread - d3/d16/d13/f9c zero size 2024-03-18T14:51:04.425 INFO:tasks.workunit.client.0.smithi012.stdout:1/695: creat d8/d1e/d27/fe2 x:0 0 0 2024-03-18T14:51:04.428 INFO:tasks.workunit.client.1.smithi100.stdout:1/439: rename d8/d1e/d11/f21 to d8/f7e 0 2024-03-18T14:51:04.431 INFO:tasks.workunit.client.0.smithi012.stdout:1/696: dread d8/db/d10/f34 [0,4194304] 0 2024-03-18T14:51:04.433 INFO:tasks.workunit.client.1.smithi100.stdout:7/429: mkdir d5/d22/d3b/d9f/da0 0 2024-03-18T14:51:04.433 INFO:tasks.workunit.client.1.smithi100.stdout:7/430: write d5/d30/d4d/d5a/f6f [4741816,37113] 0 2024-03-18T14:51:04.433 INFO:tasks.workunit.client.1.smithi100.stdout:7/431: chown d5/d14/d16/d2c/d86/f60 2277 1 2024-03-18T14:51:04.434 INFO:tasks.workunit.client.1.smithi100.stdout:0/514: truncate dd/d2b/f40 2040038 0 2024-03-18T14:51:04.437 INFO:tasks.workunit.client.1.smithi100.stdout:5/370: dread d29/f37 [0,4194304] 0 2024-03-18T14:51:04.437 INFO:tasks.workunit.client.1.smithi100.stdout:8/452: truncate d4/dc/dd/de/d19/f2f 1425897 0 2024-03-18T14:51:04.438 INFO:tasks.workunit.client.0.smithi012.stdout:4/662: rmdir d6/da/d89 39 2024-03-18T14:51:04.441 INFO:tasks.workunit.client.1.smithi100.stdout:4/418: getdents d9/d13 0 2024-03-18T14:51:04.443 INFO:tasks.workunit.client.0.smithi012.stdout:1/697: dread d8/db/d10/d16/da1/f23 [0,4194304] 0 2024-03-18T14:51:04.443 INFO:tasks.workunit.client.0.smithi012.stdout:1/698: chown d8/db/d10/d16/fa8 2699388 1 2024-03-18T14:51:04.443 INFO:tasks.workunit.client.0.smithi012.stdout:1/699: chown d8/db/d10/d16/da1/l1f 47889147 1 2024-03-18T14:51:04.443 INFO:tasks.workunit.client.0.smithi012.stdout:9/807: creat d3/de6/dbf/d66/d69/d7f/d8c/d8b/f108 x:0 0 0 2024-03-18T14:51:04.448 INFO:tasks.workunit.client.0.smithi012.stdout:4/663: dread d6/da/d58/f59 [0,4194304] 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:4/664: creat d6/d1a/d92/fe2 x:0 0 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:4/665: write d6/d19/d39/f50 [8505257,65244] 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:4/666: chown d6/d1a/c55 865414098 1 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:7/753: dwrite dd/d13/d23/d76/d6f/f91 [0,4194304] 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:7/754: fsync dd/d13/f72 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:7/755: write dd/d13/d23/d33/d48/f5c [4706886,6232] 0 2024-03-18T14:51:04.449 INFO:tasks.workunit.client.0.smithi012.stdout:2/800: mkdir d0/d6b/dbf/d33/d71/d90/dea/dfe 0 2024-03-18T14:51:04.451 INFO:tasks.workunit.client.1.smithi100.stdout:6/423: symlink d6/d7/df/d26/l81 0 2024-03-18T14:51:04.451 INFO:tasks.workunit.client.1.smithi100.stdout:6/424: truncate d6/f2f 4308975 0 2024-03-18T14:51:04.452 INFO:tasks.workunit.client.0.smithi012.stdout:3/736: rmdir dc/d3c/d59/d86/d9a/d64/d93 0 2024-03-18T14:51:04.454 INFO:tasks.workunit.client.0.smithi012.stdout:1/700: mknod d8/db/d10/d16/d6b/ce3 0 2024-03-18T14:51:04.455 INFO:tasks.workunit.client.1.smithi100.stdout:5/371: dwrite f7 [0,4194304] 0 2024-03-18T14:51:04.456 INFO:tasks.workunit.client.1.smithi100.stdout:5/372: chown d29/l3a 72414 1 2024-03-18T14:51:04.456 INFO:tasks.workunit.client.1.smithi100.stdout:3/466: mknod d3/d16/d13/d1f/ca8 0 2024-03-18T14:51:04.459 INFO:tasks.workunit.client.1.smithi100.stdout:9/410: rename da/l1b to da/d51/d55/l7c 0 2024-03-18T14:51:04.460 INFO:tasks.workunit.client.0.smithi012.stdout:4/667: symlink d6/d19/d6c/le3 0 2024-03-18T14:51:04.461 INFO:tasks.workunit.client.1.smithi100.stdout:1/440: creat d8/d1e/d11/d6d/f7f x:0 0 0 2024-03-18T14:51:04.462 INFO:tasks.workunit.client.1.smithi100.stdout:1/441: stat d8/d30/d61 0 2024-03-18T14:51:04.472 INFO:tasks.workunit.client.0.smithi012.stdout:7/756: getdents dd/d13/d23/d60/d9c 0 2024-03-18T14:51:04.475 INFO:tasks.workunit.client.0.smithi012.stdout:1/701: creat d8/db/d10/d16/da1/fe4 x:0 0 0 2024-03-18T14:51:04.478 INFO:tasks.workunit.client.1.smithi100.stdout:7/432: mkdir d5/d90/da1 0 2024-03-18T14:51:04.478 INFO:tasks.workunit.client.1.smithi100.stdout:7/433: dread - d5/d14/d16/d2e/d7b/f65 zero size 2024-03-18T14:51:04.478 INFO:tasks.workunit.client.1.smithi100.stdout:7/434: creat d5/fa2 x:0 0 0 2024-03-18T14:51:04.480 INFO:tasks.workunit.client.0.smithi012.stdout:1/702: symlink d8/db/d10/d31/d81/dc5/le5 0 2024-03-18T14:51:04.480 INFO:tasks.workunit.client.0.smithi012.stdout:1/703: chown d8/db/d10/d16/da1/d2e/fb4 1959733482 1 2024-03-18T14:51:04.484 INFO:tasks.workunit.client.1.smithi100.stdout:4/419: unlink d9/d27/f6b 0 2024-03-18T14:51:04.484 INFO:tasks.workunit.client.1.smithi100.stdout:4/420: chown d9/d13/d25/d51/f61 16148780 1 2024-03-18T14:51:04.484 INFO:tasks.workunit.client.1.smithi100.stdout:4/421: fdatasync d9/d3a/f3c 0 2024-03-18T14:51:04.485 INFO:tasks.workunit.client.0.smithi012.stdout:1/704: symlink d8/db/d10/d31/d43/d5e/d65/d84/dc0/le6 0 2024-03-18T14:51:04.485 INFO:tasks.workunit.client.0.smithi012.stdout:1/705: fdatasync d8/f40 0 2024-03-18T14:51:04.487 INFO:tasks.workunit.client.1.smithi100.stdout:6/425: unlink d6/d7/df/d26/c20 0 2024-03-18T14:51:04.488 INFO:tasks.workunit.client.0.smithi012.stdout:7/757: dread dd/d13/f75 [0,4194304] 0 2024-03-18T14:51:04.488 INFO:tasks.workunit.client.0.smithi012.stdout:7/758: write dd/d13/d23/d2f/d53/fa5 [437930,105812] 0 2024-03-18T14:51:04.490 INFO:tasks.workunit.client.0.smithi012.stdout:6/681: sync 2024-03-18T14:51:04.490 INFO:tasks.workunit.client.0.smithi012.stdout:2/801: dwrite d0/d6b/fc2 [0,4194304] 0 2024-03-18T14:51:04.490 INFO:tasks.workunit.client.0.smithi012.stdout:3/737: dwrite dc/d30/f99 [0,4194304] 0 2024-03-18T14:51:04.490 INFO:tasks.workunit.client.0.smithi012.stdout:3/738: truncate dc/d3c/d4c/d57/fb9 625270 0 2024-03-18T14:51:04.491 INFO:tasks.workunit.client.1.smithi100.stdout:5/373: truncate f10 3790901 0 2024-03-18T14:51:04.492 INFO:tasks.workunit.client.0.smithi012.stdout:1/706: creat d8/d1e/d27/d44/fe7 x:0 0 0 2024-03-18T14:51:04.492 INFO:tasks.workunit.client.0.smithi012.stdout:4/668: dwrite d6/d27/f56 [0,4194304] 0 2024-03-18T14:51:04.495 INFO:tasks.workunit.client.1.smithi100.stdout:3/467: mkdir d3/d16/d13/d1f/d3f/d4f/d60/da9 0 2024-03-18T14:51:04.500 INFO:tasks.workunit.client.0.smithi012.stdout:7/759: creat dd/f106 x:0 0 0 2024-03-18T14:51:04.501 INFO:tasks.workunit.client.1.smithi100.stdout:0/515: rename dd/d31/d91/d7b/c94 to dd/d31/d91/d56/d62/cbc 0 2024-03-18T14:51:04.501 INFO:tasks.workunit.client.1.smithi100.stdout:0/516: chown dd/fe 176 1 2024-03-18T14:51:04.502 INFO:tasks.workunit.client.0.smithi012.stdout:1/707: dread d8/d1e/f63 [0,4194304] 0 2024-03-18T14:51:04.502 INFO:tasks.workunit.client.0.smithi012.stdout:1/708: chown d8/db/d10/d31/c55 153840 1 2024-03-18T14:51:04.503 INFO:tasks.workunit.client.0.smithi012.stdout:6/682: write d17/d19/d27/d2f/f62 [1404496,130877] 0 2024-03-18T14:51:04.503 INFO:tasks.workunit.client.0.smithi012.stdout:6/683: write fb [2136428,96906] 0 2024-03-18T14:51:04.504 INFO:tasks.workunit.client.1.smithi100.stdout:9/411: creat da/d2c/d2f/f7d x:0 0 0 2024-03-18T14:51:04.505 INFO:tasks.workunit.client.1.smithi100.stdout:9/412: fsync da/d51/d55/f7b 0 2024-03-18T14:51:04.507 INFO:tasks.workunit.client.0.smithi012.stdout:2/802: rename d0/d6b/dbf/d57/d61/cee to d0/d6b/dbf/d57/d61/cff 0 2024-03-18T14:51:04.507 INFO:tasks.workunit.client.0.smithi012.stdout:2/803: creat d0/d6b/dbf/d4f/f100 x:0 0 0 2024-03-18T14:51:04.508 INFO:tasks.workunit.client.1.smithi100.stdout:1/442: read - d8/d1e/d11/d27/d3f/f78 zero size 2024-03-18T14:51:04.511 INFO:tasks.workunit.client.0.smithi012.stdout:3/739: dwrite dc/fcb [0,4194304] 0 2024-03-18T14:51:04.513 INFO:tasks.workunit.client.1.smithi100.stdout:3/468: dwrite d3/ff [0,4194304] 0 2024-03-18T14:51:04.517 INFO:tasks.workunit.client.0.smithi012.stdout:4/669: creat d6/d1a/d98/daf/db5/fe4 x:0 0 0 2024-03-18T14:51:04.526 INFO:tasks.workunit.client.1.smithi100.stdout:8/453: rmdir d4/dc/dd/de/d21 39 2024-03-18T14:51:04.529 INFO:tasks.workunit.client.1.smithi100.stdout:4/422: link d9/d17/l77 d9/d13/l7c 0 2024-03-18T14:51:04.531 INFO:tasks.workunit.client.1.smithi100.stdout:9/413: dwrite da/f4f [0,4194304] 0 2024-03-18T14:51:04.532 INFO:tasks.workunit.client.1.smithi100.stdout:1/443: dwrite d8/d30/f40 [0,4194304] 0 2024-03-18T14:51:04.533 INFO:tasks.workunit.client.1.smithi100.stdout:1/444: fsync d8/d2a/f47 0 2024-03-18T14:51:04.533 INFO:tasks.workunit.client.1.smithi100.stdout:1/445: chown d8/d1e/d11/d27/f4d 42596487 1 2024-03-18T14:51:04.533 INFO:tasks.workunit.client.0.smithi012.stdout:7/760: rename dd/d13/d23/d2f/f3d to dd/d13/d23/d33/dd5/f107 0 2024-03-18T14:51:04.534 INFO:tasks.workunit.client.1.smithi100.stdout:6/426: creat d6/d7/df/d26/d4e/f82 x:0 0 0 2024-03-18T14:51:04.535 INFO:tasks.workunit.client.1.smithi100.stdout:6/427: chown d6/d7/df/d26/d4e/d66/d36/c62 28502469 1 2024-03-18T14:51:04.537 INFO:tasks.workunit.client.0.smithi012.stdout:1/709: mkdir d8/d1e/d27/d44/dc2/de8 0 2024-03-18T14:51:04.538 INFO:tasks.workunit.client.1.smithi100.stdout:8/454: dread d4/dc/d31/f3d [0,4194304] 0 2024-03-18T14:51:04.538 INFO:tasks.workunit.client.1.smithi100.stdout:8/455: chown d4/f6 13927057 1 2024-03-18T14:51:04.541 INFO:tasks.workunit.client.1.smithi100.stdout:5/374: rename d29/d2d/f5e to d29/d2d/d32/d38/f6e 0 2024-03-18T14:51:04.543 INFO:tasks.workunit.client.0.smithi012.stdout:6/684: mknod d17/d38/d5b/cc9 0 2024-03-18T14:51:04.543 INFO:tasks.workunit.client.0.smithi012.stdout:6/685: creat d17/d19/d27/d58/d91/fca x:0 0 0 2024-03-18T14:51:04.543 INFO:tasks.workunit.client.0.smithi012.stdout:6/686: fdatasync d17/d19/d27/d6e/fc0 0 2024-03-18T14:51:04.545 INFO:tasks.workunit.client.1.smithi100.stdout:0/517: symlink dd/d31/d91/d56/d59/lbd 0 2024-03-18T14:51:04.546 INFO:tasks.workunit.client.1.smithi100.stdout:0/518: write dd/fb1 [657459,17038] 0 2024-03-18T14:51:04.549 INFO:tasks.workunit.client.0.smithi012.stdout:1/710: dread d8/d1e/d27/f74 [0,4194304] 0 2024-03-18T14:51:04.556 INFO:tasks.workunit.client.1.smithi100.stdout:1/446: dwrite d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:04.556 INFO:tasks.workunit.client.0.smithi012.stdout:4/670: read d6/d1a/d30/d46/f48 [206073,15607] 0 2024-03-18T14:51:04.556 INFO:tasks.workunit.client.0.smithi012.stdout:4/671: truncate d6/db6/fd9 913054 0 2024-03-18T14:51:04.556 INFO:tasks.workunit.client.0.smithi012.stdout:4/672: chown d6/d19/f51 89 1 2024-03-18T14:51:04.557 INFO:tasks.workunit.client.0.smithi012.stdout:4/673: dread d6/da/f91 [0,4194304] 0 2024-03-18T14:51:04.558 INFO:tasks.workunit.client.0.smithi012.stdout:4/674: write d6/f2e [4296738,86056] 0 2024-03-18T14:51:04.558 INFO:tasks.workunit.client.1.smithi100.stdout:3/469: mknod d3/d16/d13/d17/caa 0 2024-03-18T14:51:04.558 INFO:tasks.workunit.client.1.smithi100.stdout:8/456: dwrite d4/dc/f8a [0,4194304] 0 2024-03-18T14:51:04.560 INFO:tasks.workunit.client.0.smithi012.stdout:2/804: rename d0/da2/f5c to d0/d6b/dbf/f101 0 2024-03-18T14:51:04.562 INFO:tasks.workunit.client.1.smithi100.stdout:7/435: symlink d5/d14/d16/d2c/la3 0 2024-03-18T14:51:04.565 INFO:tasks.workunit.client.0.smithi012.stdout:7/761: rename dd/d13/d23/d76/d6b/fb0 to dd/d13/d23/d76/d6b/f108 0 2024-03-18T14:51:04.572 INFO:tasks.workunit.client.1.smithi100.stdout:4/423: unlink d9/d13/d5a/f65 0 2024-03-18T14:51:04.572 INFO:tasks.workunit.client.1.smithi100.stdout:4/424: write d9/d27/f3d [649999,11396] 0 2024-03-18T14:51:04.573 INFO:tasks.workunit.client.1.smithi100.stdout:4/425: chown d9/d3a 422022339 1 2024-03-18T14:51:04.574 INFO:tasks.workunit.client.1.smithi100.stdout:9/414: truncate da/d25/f3f 438230 0 2024-03-18T14:51:04.578 INFO:tasks.workunit.client.0.smithi012.stdout:6/687: mknod d17/d19/d27/ccb 0 2024-03-18T14:51:04.578 INFO:tasks.workunit.client.0.smithi012.stdout:6/688: fsync d17/d19/d27/f54 0 2024-03-18T14:51:04.579 INFO:tasks.workunit.client.1.smithi100.stdout:6/428: mknod d6/d7/df/d26/d4e/c83 0 2024-03-18T14:51:04.580 INFO:tasks.workunit.client.0.smithi012.stdout:3/740: getdents dc/d30/d35 0 2024-03-18T14:51:04.581 INFO:tasks.workunit.client.1.smithi100.stdout:2/409: sync 2024-03-18T14:51:04.587 INFO:tasks.workunit.client.1.smithi100.stdout:4/426: read d9/d27/d42/f50 [2876449,42623] 0 2024-03-18T14:51:04.588 INFO:tasks.workunit.client.1.smithi100.stdout:4/427: write d9/d3a/f3c [495712,93512] 0 2024-03-18T14:51:04.588 INFO:tasks.workunit.client.0.smithi012.stdout:1/711: dwrite d8/db/d10/d31/d81/f8c [0,4194304] 0 2024-03-18T14:51:04.588 INFO:tasks.workunit.client.0.smithi012.stdout:2/805: dwrite d0/d6b/dbf/d33/d71/fc1 [0,4194304] 0 2024-03-18T14:51:04.588 INFO:tasks.workunit.client.0.smithi012.stdout:2/806: chown d0/da2/cad 6484315 1 2024-03-18T14:51:04.589 INFO:tasks.workunit.client.0.smithi012.stdout:7/762: rmdir dd/d13/db7 39 2024-03-18T14:51:04.589 INFO:tasks.workunit.client.1.smithi100.stdout:3/470: dwrite d3/f29 [4194304,4194304] 0 2024-03-18T14:51:04.589 INFO:tasks.workunit.client.1.smithi100.stdout:3/471: write d3/dd/d21/d3e/f32 [8897936,34158] 0 2024-03-18T14:51:04.589 INFO:tasks.workunit.client.1.smithi100.stdout:3/472: dread - d3/d16/d13/d1f/f90 zero size 2024-03-18T14:51:04.590 INFO:tasks.workunit.client.1.smithi100.stdout:3/473: read - d3/dd/d21/fa7 zero size 2024-03-18T14:51:04.590 INFO:tasks.workunit.client.1.smithi100.stdout:3/474: dread - d3/d16/d13/d17/f61 zero size 2024-03-18T14:51:04.592 INFO:tasks.workunit.client.0.smithi012.stdout:6/689: mkdir d17/d38/d51/dcc 0 2024-03-18T14:51:04.592 INFO:tasks.workunit.client.0.smithi012.stdout:6/690: read - d17/d19/d27/d6e/fc0 zero size 2024-03-18T14:51:04.592 INFO:tasks.workunit.client.0.smithi012.stdout:6/691: read d17/f75 [917598,112562] 0 2024-03-18T14:51:04.593 INFO:tasks.workunit.client.1.smithi100.stdout:1/447: mknod d8/d1e/d11/d6d/d59/c80 0 2024-03-18T14:51:04.594 INFO:tasks.workunit.client.1.smithi100.stdout:1/448: stat d8/d1e/d11/d2d 0 2024-03-18T14:51:04.594 INFO:tasks.workunit.client.1.smithi100.stdout:1/449: chown d8/d30/d61 181352 1 2024-03-18T14:51:04.594 INFO:tasks.workunit.client.1.smithi100.stdout:1/450: write f7 [4479414,59609] 0 2024-03-18T14:51:04.595 INFO:tasks.workunit.client.0.smithi012.stdout:3/741: creat dc/d3c/d72/fee x:0 0 0 2024-03-18T14:51:04.595 INFO:tasks.workunit.client.0.smithi012.stdout:3/742: chown dc/d3c/f4d 167766 1 2024-03-18T14:51:04.598 INFO:tasks.workunit.client.0.smithi012.stdout:4/675: getdents d6/d1a/d30 0 2024-03-18T14:51:04.598 INFO:tasks.workunit.client.0.smithi012.stdout:4/676: fdatasync d6/f25 0 2024-03-18T14:51:04.598 INFO:tasks.workunit.client.0.smithi012.stdout:4/677: chown d6/d19/c45 163627 1 2024-03-18T14:51:04.598 INFO:tasks.workunit.client.0.smithi012.stdout:4/678: chown d6/d27/f67 49236 1 2024-03-18T14:51:04.602 INFO:tasks.workunit.client.0.smithi012.stdout:8/931: sync 2024-03-18T14:51:04.604 INFO:tasks.workunit.client.1.smithi100.stdout:8/457: creat d4/dc/d31/d66/f9b x:0 0 0 2024-03-18T14:51:04.604 INFO:tasks.workunit.client.1.smithi100.stdout:8/458: creat d4/dc/d1f/f9c x:0 0 0 2024-03-18T14:51:04.606 INFO:tasks.workunit.client.1.smithi100.stdout:6/429: creat d6/d7/df/d26/f84 x:0 0 0 2024-03-18T14:51:04.607 INFO:tasks.workunit.client.0.smithi012.stdout:2/807: dread d0/d3c/d40/f5a [0,4194304] 0 2024-03-18T14:51:04.608 INFO:tasks.workunit.client.0.smithi012.stdout:3/743: dread dc/d30/d3a/fdf [0,4194304] 0 2024-03-18T14:51:04.608 INFO:tasks.workunit.client.0.smithi012.stdout:3/744: write dc/d91/fbb [617775,72536] 0 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.0.smithi012.stdout:3/745: stat dc/d30/d3a/l42 0 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.0.smithi012.stdout:3/746: dread - dc/d30/fd6 zero size 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.0.smithi012.stdout:1/712: dwrite d8/d1e/d27/f3c [0,4194304] 0 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.0.smithi012.stdout:1/713: creat d8/db/d10/d31/d43/d5e/d65/d70/dc7/fe9 x:0 0 0 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.0.smithi012.stdout:1/714: readlink d8/db/d10/d31/l78 0 2024-03-18T14:51:04.609 INFO:tasks.workunit.client.1.smithi100.stdout:2/410: mknod d3/d6/d5f/c91 0 2024-03-18T14:51:04.611 INFO:tasks.workunit.client.0.smithi012.stdout:7/763: rename dd/d13/d23/d76/d6f/dd9/ddd/df8 to dd/d13/d23/d76/d109 0 2024-03-18T14:51:04.613 INFO:tasks.workunit.client.1.smithi100.stdout:2/411: dread d3/fc [0,4194304] 0 2024-03-18T14:51:04.613 INFO:tasks.workunit.client.1.smithi100.stdout:5/375: rename f14 to d29/d2d/f6f 0 2024-03-18T14:51:04.615 INFO:tasks.workunit.client.0.smithi012.stdout:6/692: mkdir d17/d26/d36/d4d/d82/dcd 0 2024-03-18T14:51:04.618 INFO:tasks.workunit.client.0.smithi012.stdout:4/679: mkdir d6/da/d89/da1/dbb/ddc/de5 0 2024-03-18T14:51:04.621 INFO:tasks.workunit.client.1.smithi100.stdout:4/428: creat d9/d27/d62/f7d x:0 0 0 2024-03-18T14:51:04.625 INFO:tasks.workunit.client.0.smithi012.stdout:3/747: mknod dc/d3c/d59/db2/cef 0 2024-03-18T14:51:04.626 INFO:tasks.workunit.client.1.smithi100.stdout:3/475: mkdir d3/d16/d13/d17/dab 0 2024-03-18T14:51:04.629 INFO:tasks.workunit.client.0.smithi012.stdout:7/764: link dd/d13/d23/d76/d6b/fe8 dd/d13/d23/d76/f10a 0 2024-03-18T14:51:04.629 INFO:tasks.workunit.client.0.smithi012.stdout:7/765: write dd/d13/d23/fdc [985287,7088] 0 2024-03-18T14:51:04.629 INFO:tasks.workunit.client.0.smithi012.stdout:7/766: readlink dd/l86 0 2024-03-18T14:51:04.629 INFO:tasks.workunit.client.0.smithi012.stdout:7/767: chown dd/d13/d23/d2f/d53/d8d 77889764 1 2024-03-18T14:51:04.630 INFO:tasks.workunit.client.0.smithi012.stdout:6/693: link c10 d17/d38/d51/dcc/cce 0 2024-03-18T14:51:04.631 INFO:tasks.workunit.client.0.smithi012.stdout:4/680: mknod d6/da/ce6 0 2024-03-18T14:51:04.631 INFO:tasks.workunit.client.0.smithi012.stdout:4/681: chown d6/d1a/d30/c52 97550372 1 2024-03-18T14:51:04.632 INFO:tasks.workunit.client.0.smithi012.stdout:7/768: creat dd/d13/d23/d60/d9c/f10b x:0 0 0 2024-03-18T14:51:04.634 INFO:tasks.workunit.client.1.smithi100.stdout:0/519: rmdir dd 39 2024-03-18T14:51:04.634 INFO:tasks.workunit.client.1.smithi100.stdout:0/520: chown dd/d31/d91/c86 2 1 2024-03-18T14:51:04.634 INFO:tasks.workunit.client.1.smithi100.stdout:0/521: write dd/fb1 [598345,83329] 0 2024-03-18T14:51:04.635 INFO:tasks.workunit.client.0.smithi012.stdout:6/694: creat d17/d38/d51/d66/fcf x:0 0 0 2024-03-18T14:51:04.635 INFO:tasks.workunit.client.0.smithi012.stdout:6/695: write d17/d38/f5a [1935202,19908] 0 2024-03-18T14:51:04.636 INFO:tasks.workunit.client.0.smithi012.stdout:6/696: fsync d17/d18/db8/f55 0 2024-03-18T14:51:04.637 INFO:tasks.workunit.client.1.smithi100.stdout:1/451: mkdir d8/d2a/d37/d50/d6f/d81 0 2024-03-18T14:51:04.640 INFO:tasks.workunit.client.1.smithi100.stdout:2/412: dwrite d3/da/de/ff [0,4194304] 0 2024-03-18T14:51:04.640 INFO:tasks.workunit.client.1.smithi100.stdout:2/413: creat d3/d6/d29/f92 x:0 0 0 2024-03-18T14:51:04.640 INFO:tasks.workunit.client.1.smithi100.stdout:8/459: creat d4/dc/dd/de/d21/d23/d25/f9d x:0 0 0 2024-03-18T14:51:04.641 INFO:tasks.workunit.client.0.smithi012.stdout:6/697: mkdir d17/d38/d51/dcc/dd0 0 2024-03-18T14:51:04.643 INFO:tasks.workunit.client.0.smithi012.stdout:7/769: dread dd/d13/d23/d33/f78 [4194304,4194304] 0 2024-03-18T14:51:04.643 INFO:tasks.workunit.client.0.smithi012.stdout:7/770: chown dd/d13/d23/d76/d6b/f100 2575608 1 2024-03-18T14:51:04.643 INFO:tasks.workunit.client.1.smithi100.stdout:7/436: getdents d5/d14/d16/d44 0 2024-03-18T14:51:04.644 INFO:tasks.workunit.client.0.smithi012.stdout:2/808: dwrite d0/d6b/dbf/d33/fd5 [0,4194304] 0 2024-03-18T14:51:04.644 INFO:tasks.workunit.client.0.smithi012.stdout:3/748: dread dc/f43 [0,4194304] 0 2024-03-18T14:51:04.645 INFO:tasks.workunit.client.0.smithi012.stdout:8/932: dwrite d0/fe2 [4194304,4194304] 0 2024-03-18T14:51:04.645 INFO:tasks.workunit.client.0.smithi012.stdout:8/933: dread - d0/d15/fec zero size 2024-03-18T14:51:04.645 INFO:tasks.workunit.client.0.smithi012.stdout:8/934: write d0/d5a/d5e/dd7/d84/d54/d62/daf/f115 [736879,104618] 0 2024-03-18T14:51:04.645 INFO:tasks.workunit.client.0.smithi012.stdout:8/935: stat d0/d5a/d5e/dd7/d84/d2a/d3e/d119/c128 0 2024-03-18T14:51:04.652 INFO:tasks.workunit.client.0.smithi012.stdout:6/698: link d17/d19/d27/d2f/c95 d17/d26/d36/d4d/d82/d97/cd1 0 2024-03-18T14:51:04.656 INFO:tasks.workunit.client.0.smithi012.stdout:3/749: dread dc/d30/f51 [0,4194304] 0 2024-03-18T14:51:04.659 INFO:tasks.workunit.client.1.smithi100.stdout:9/415: rename da/d46/d4d/d67/d74 to da/d51/d55/d7e 0 2024-03-18T14:51:04.659 INFO:tasks.workunit.client.1.smithi100.stdout:9/416: read da/d46/f63 [419519,62366] 0 2024-03-18T14:51:04.660 INFO:tasks.workunit.client.0.smithi012.stdout:2/809: mknod d0/c102 0 2024-03-18T14:51:04.662 INFO:tasks.workunit.client.1.smithi100.stdout:4/429: symlink d9/d27/d62/d7b/l7e 0 2024-03-18T14:51:04.664 INFO:tasks.workunit.client.1.smithi100.stdout:3/476: mknod d3/d16/d13/d1f/d3f/cac 0 2024-03-18T14:51:04.664 INFO:tasks.workunit.client.1.smithi100.stdout:3/477: chown d3/d16/d13/d17/d88 76261973 1 2024-03-18T14:51:04.665 INFO:tasks.workunit.client.0.smithi012.stdout:6/699: dread d17/d18/f9e [0,4194304] 0 2024-03-18T14:51:04.668 INFO:tasks.workunit.client.1.smithi100.stdout:0/522: creat dd/d2b/d95/d79/d7d/da9/db0/fbe x:0 0 0 2024-03-18T14:51:04.669 INFO:tasks.workunit.client.0.smithi012.stdout:7/771: dwrite dd/d13/d23/d2f/f9f [0,4194304] 0 2024-03-18T14:51:04.669 INFO:tasks.workunit.client.0.smithi012.stdout:4/682: dwrite d6/da/d89/d97/dc5/dd3/fcd [4194304,4194304] 0 2024-03-18T14:51:04.671 INFO:tasks.workunit.client.1.smithi100.stdout:1/452: mkdir d8/d82 0 2024-03-18T14:51:04.674 INFO:tasks.workunit.client.0.smithi012.stdout:3/750: mknod dc/d30/d3a/da0/dc5/dc7/de7/cf0 0 2024-03-18T14:51:04.677 INFO:tasks.workunit.client.1.smithi100.stdout:2/414: symlink d3/d1a/d6c/l93 0 2024-03-18T14:51:04.680 INFO:tasks.workunit.client.0.smithi012.stdout:7/772: link dd/d13/d23/d33/dcc/fe3 dd/d13/d23/d33/d48/f10c 0 2024-03-18T14:51:04.680 INFO:tasks.workunit.client.0.smithi012.stdout:7/773: fdatasync dd/d13/d23/d60/f6c 0 2024-03-18T14:51:04.680 INFO:tasks.workunit.client.0.smithi012.stdout:7/774: chown dd/d13/d23/d2f/f32 54540 1 2024-03-18T14:51:04.680 INFO:tasks.workunit.client.0.smithi012.stdout:7/775: truncate dd/d13/d23/d76/d6b/f100 298655 0 2024-03-18T14:51:04.682 INFO:tasks.workunit.client.1.smithi100.stdout:8/460: link d4/d3c/c5e d4/dc/d1f/d59/c9e 0 2024-03-18T14:51:04.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/683: unlink d6/d19/d39/f41 0 2024-03-18T14:51:04.688 INFO:tasks.workunit.client.1.smithi100.stdout:7/437: mknod d5/d14/d16/d2e/d62/d66/d4f/ca4 0 2024-03-18T14:51:04.688 INFO:tasks.workunit.client.1.smithi100.stdout:7/438: chown d5/d14/l2a 7 1 2024-03-18T14:51:04.691 INFO:tasks.workunit.client.1.smithi100.stdout:6/430: creat d6/d7/df/d72/f85 x:0 0 0 2024-03-18T14:51:04.693 INFO:tasks.workunit.client.0.smithi012.stdout:7/776: symlink dd/d13/d23/d76/d6f/dd9/l10d 0 2024-03-18T14:51:04.693 INFO:tasks.workunit.client.0.smithi012.stdout:7/777: truncate dd/d13/d23/d2f/d53/dc6/df1/ff4 1002404 0 2024-03-18T14:51:04.695 INFO:tasks.workunit.client.0.smithi012.stdout:4/684: mkdir d6/d1a/d30/d46/d6f/d8b/de7 0 2024-03-18T14:51:04.698 INFO:tasks.workunit.client.1.smithi100.stdout:5/376: rename d29/d2d/d39/d65/f66 to d29/d2d/d39/d60/f70 0 2024-03-18T14:51:04.699 INFO:tasks.workunit.client.1.smithi100.stdout:9/417: symlink da/dd/d29/d71/l7f 0 2024-03-18T14:51:04.701 INFO:tasks.workunit.client.0.smithi012.stdout:4/685: getdents d6/d1a/d98 0 2024-03-18T14:51:04.703 INFO:tasks.workunit.client.1.smithi100.stdout:4/430: creat d9/f7f x:0 0 0 2024-03-18T14:51:04.703 INFO:tasks.workunit.client.0.smithi012.stdout:2/810: dwrite d0/d6b/dbf/f27 [0,4194304] 0 2024-03-18T14:51:04.703 INFO:tasks.workunit.client.0.smithi012.stdout:2/811: readlink d0/d6b/dbf/d33/d80/de1/lf4 0 2024-03-18T14:51:04.705 INFO:tasks.workunit.client.1.smithi100.stdout:2/415: dwrite d3/d6/f67 [0,4194304] 0 2024-03-18T14:51:04.705 INFO:tasks.workunit.client.1.smithi100.stdout:0/523: dwrite dd/d31/d91/d56/d71/fa3 [0,4194304] 0 2024-03-18T14:51:04.706 INFO:tasks.workunit.client.0.smithi012.stdout:3/751: dwrite dc/f12 [0,4194304] 0 2024-03-18T14:51:04.706 INFO:tasks.workunit.client.0.smithi012.stdout:3/752: stat dc/l76 0 2024-03-18T14:51:04.706 INFO:tasks.workunit.client.0.smithi012.stdout:7/778: dread dd/d13/d23/d76/f102 [0,4194304] 0 2024-03-18T14:51:04.707 INFO:tasks.workunit.client.0.smithi012.stdout:4/686: rmdir d6/d1a/d98/daf/db5 39 2024-03-18T14:51:04.709 INFO:tasks.workunit.client.1.smithi100.stdout:1/453: mkdir d8/d1e/d11/d27/d83 0 2024-03-18T14:51:04.713 INFO:tasks.workunit.client.1.smithi100.stdout:0/524: write dd/f1d [3241991,110201] 0 2024-03-18T14:51:04.713 INFO:tasks.workunit.client.0.smithi012.stdout:2/812: dread d0/d3c/d40/d47/f49 [0,4194304] 0 2024-03-18T14:51:04.714 INFO:tasks.workunit.client.0.smithi012.stdout:2/813: truncate d0/d3c/d40/f5a 6172721 0 2024-03-18T14:51:04.714 INFO:tasks.workunit.client.0.smithi012.stdout:3/753: rename dc/d3c/d4c/c61 to dc/d3c/d72/cf1 0 2024-03-18T14:51:04.715 INFO:tasks.workunit.client.1.smithi100.stdout:7/439: symlink d5/d22/d24/d39/la5 0 2024-03-18T14:51:04.715 INFO:tasks.workunit.client.1.smithi100.stdout:7/440: dread - d5/d14/d16/d44/f92 zero size 2024-03-18T14:51:04.715 INFO:tasks.workunit.client.1.smithi100.stdout:7/441: creat d5/d14/d16/d2e/d7b/fa6 x:0 0 0 2024-03-18T14:51:04.715 INFO:tasks.workunit.client.1.smithi100.stdout:7/442: write d5/d14/d16/d2e/f7d [1011527,89788] 0 2024-03-18T14:51:04.716 INFO:tasks.workunit.client.0.smithi012.stdout:7/779: getdents dd/d13/d23/d33/dd5 0 2024-03-18T14:51:04.718 INFO:tasks.workunit.client.1.smithi100.stdout:6/431: stat d6/d7/df/d72/d48/l69 0 2024-03-18T14:51:04.718 INFO:tasks.workunit.client.1.smithi100.stdout:6/432: fsync d6/d7/f8 0 2024-03-18T14:51:04.720 INFO:tasks.workunit.client.0.smithi012.stdout:5/760: sync 2024-03-18T14:51:04.720 INFO:tasks.workunit.client.0.smithi012.stdout:0/786: sync 2024-03-18T14:51:04.720 INFO:tasks.workunit.client.0.smithi012.stdout:0/787: chown d1/d16/d79/f8f 1997823 1 2024-03-18T14:51:04.720 INFO:tasks.workunit.client.0.smithi012.stdout:5/761: getdents d3/de0 0 2024-03-18T14:51:04.720 INFO:tasks.workunit.client.0.smithi012.stdout:5/762: write d3/d29/d36/fe4 [134860,81733] 0 2024-03-18T14:51:04.721 INFO:tasks.workunit.client.0.smithi012.stdout:4/687: unlink d6/d19/d6c/cca 0 2024-03-18T14:51:04.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/754: write dc/d3c/d4c/d57/fa1 [3310585,91837] 0 2024-03-18T14:51:04.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/755: fsync dc/f32 0 2024-03-18T14:51:04.722 INFO:tasks.workunit.client.0.smithi012.stdout:3/756: truncate dc/fa5 819600 0 2024-03-18T14:51:04.722 INFO:tasks.workunit.client.0.smithi012.stdout:4/688: dread d6/d1a/d30/d46/f48 [0,4194304] 0 2024-03-18T14:51:04.724 INFO:tasks.workunit.client.1.smithi100.stdout:8/461: rename d4/dc/dd/de/d21/d23/d47/d56/l87 to d4/dc/dd/de/l9f 0 2024-03-18T14:51:04.728 INFO:tasks.workunit.client.0.smithi012.stdout:2/814: unlink d0/d6b/dbf/f27 0 2024-03-18T14:51:04.731 INFO:tasks.workunit.client.0.smithi012.stdout:0/788: write d1/d20/d4e/d54/d8a/fb9 [965752,853] 0 2024-03-18T14:51:04.734 INFO:tasks.workunit.client.1.smithi100.stdout:5/377: mkdir d29/d2d/d39/d71 0 2024-03-18T14:51:04.737 INFO:tasks.workunit.client.1.smithi100.stdout:9/418: mknod da/dd/d3b/c80 0 2024-03-18T14:51:04.737 INFO:tasks.workunit.client.1.smithi100.stdout:9/419: fsync da/d2c/d2f/f31 0 2024-03-18T14:51:04.737 INFO:tasks.workunit.client.1.smithi100.stdout:9/420: chown da/dd/d3c 2 1 2024-03-18T14:51:04.737 INFO:tasks.workunit.client.1.smithi100.stdout:9/421: fdatasync da/d51/d55/f5c 0 2024-03-18T14:51:04.739 INFO:tasks.workunit.client.0.smithi012.stdout:7/780: rename dd/d13/d23/d33/f98 to dd/d13/d23/d76/d6b/f10e 0 2024-03-18T14:51:04.741 INFO:tasks.workunit.client.1.smithi100.stdout:4/431: mknod d9/d13/c80 0 2024-03-18T14:51:04.742 INFO:tasks.workunit.client.1.smithi100.stdout:6/433: dwrite d6/d12/f43 [0,4194304] 0 2024-03-18T14:51:04.742 INFO:tasks.workunit.client.1.smithi100.stdout:6/434: fdatasync d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b 0 2024-03-18T14:51:04.748 INFO:tasks.workunit.client.1.smithi100.stdout:2/416: rmdir d3/d1a/d6c 39 2024-03-18T14:51:04.748 INFO:tasks.workunit.client.1.smithi100.stdout:2/417: write d3/da/de/d20/f90 [813199,13759] 0 2024-03-18T14:51:04.749 INFO:tasks.workunit.client.1.smithi100.stdout:8/462: dwrite d4/dc/dd/de/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:04.749 INFO:tasks.workunit.client.0.smithi012.stdout:2/815: dwrite d0/d6b/dbf/d4f/f5f [0,4194304] 0 2024-03-18T14:51:04.751 INFO:tasks.workunit.client.0.smithi012.stdout:7/781: write dd/d13/d23/d2f/f9f [3518782,16816] 0 2024-03-18T14:51:04.751 INFO:tasks.workunit.client.0.smithi012.stdout:7/782: creat dd/d13/d23/d76/d6f/dd9/f10f x:0 0 0 2024-03-18T14:51:04.753 INFO:tasks.workunit.client.1.smithi100.stdout:3/478: getdents d3/d16/d13/d1f 0 2024-03-18T14:51:04.753 INFO:tasks.workunit.client.1.smithi100.stdout:3/479: creat d3/d16/d13/d17/d7d/fad x:0 0 0 2024-03-18T14:51:04.755 INFO:tasks.workunit.client.1.smithi100.stdout:1/454: mkdir d8/d2a/d84 0 2024-03-18T14:51:04.758 INFO:tasks.workunit.client.1.smithi100.stdout:0/525: link dd/c29 dd/d31/d91/d56/d62/d6b/cbf 0 2024-03-18T14:51:04.758 INFO:tasks.workunit.client.1.smithi100.stdout:0/526: fsync dd/d20/f97 0 2024-03-18T14:51:04.758 INFO:tasks.workunit.client.1.smithi100.stdout:0/527: read - dd/d2b/d95/d79/d7d/da9/db0/fb6 zero size 2024-03-18T14:51:04.759 INFO:tasks.workunit.client.0.smithi012.stdout:5/763: rmdir d3/d15/d42/dc6 39 2024-03-18T14:51:04.759 INFO:tasks.workunit.client.1.smithi100.stdout:0/528: write dd/f8f [153296,31810] 0 2024-03-18T14:51:04.762 INFO:tasks.workunit.client.1.smithi100.stdout:4/432: dwrite d9/d27/f4b [0,4194304] 0 2024-03-18T14:51:04.763 INFO:tasks.workunit.client.0.smithi012.stdout:3/757: symlink dc/d3c/d59/d86/d9a/d64/lf2 0 2024-03-18T14:51:04.763 INFO:tasks.workunit.client.0.smithi012.stdout:3/758: chown dc/d30/l3b 0 1 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.1.smithi100.stdout:7/443: rmdir d5/d22/d24/d6e/d7a 39 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.1.smithi100.stdout:5/378: symlink d29/d2d/d39/d71/l72 0 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.1.smithi100.stdout:5/379: dread - d29/d2d/d32/d38/f52 zero size 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.1.smithi100.stdout:5/380: chown d29/d4a/c4d 64 1 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.0.smithi012.stdout:4/689: truncate d6/d19/f85 743857 0 2024-03-18T14:51:04.764 INFO:tasks.workunit.client.0.smithi012.stdout:4/690: chown d6/f2e 7 1 2024-03-18T14:51:04.766 INFO:tasks.workunit.client.0.smithi012.stdout:7/783: read dd/d13/d23/d33/fa8 [2882579,68049] 0 2024-03-18T14:51:04.776 INFO:tasks.workunit.client.0.smithi012.stdout:0/789: rename d1/d16/c4c to d1/d20/d4e/d54/d67/dbc/c102 0 2024-03-18T14:51:04.776 INFO:tasks.workunit.client.0.smithi012.stdout:0/790: readlink d1/d20/d23/l9e 0 2024-03-18T14:51:04.776 INFO:tasks.workunit.client.0.smithi012.stdout:0/791: dread - d1/d20/d23/d6f/d93/f100 zero size 2024-03-18T14:51:04.776 INFO:tasks.workunit.client.1.smithi100.stdout:7/444: dread d5/d14/d16/d44/f76 [0,4194304] 0 2024-03-18T14:51:04.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/418: dwrite d3/d6/d29/d36/f4e [0,4194304] 0 2024-03-18T14:51:04.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/419: write d3/da/f3a [5071317,114171] 0 2024-03-18T14:51:04.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/420: fsync d3/d6/d5f/d63/f72 0 2024-03-18T14:51:04.781 INFO:tasks.workunit.client.1.smithi100.stdout:4/433: dwrite d9/d13/d39/d4c/f7a [0,4194304] 0 2024-03-18T14:51:04.782 INFO:tasks.workunit.client.1.smithi100.stdout:1/455: dwrite d8/f69 [0,4194304] 0 2024-03-18T14:51:04.792 INFO:tasks.workunit.client.0.smithi012.stdout:5/764: unlink d3/d29/l78 0 2024-03-18T14:51:04.797 INFO:tasks.workunit.client.0.smithi012.stdout:3/759: mkdir dc/d3c/d72/df3 0 2024-03-18T14:51:04.800 INFO:tasks.workunit.client.0.smithi012.stdout:4/691: creat d6/da/d89/da1/fe8 x:0 0 0 2024-03-18T14:51:04.801 INFO:tasks.workunit.client.1.smithi100.stdout:1/456: dwrite d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:04.801 INFO:tasks.workunit.client.1.smithi100.stdout:1/457: dread - d8/d1e/d11/d6d/f7f zero size 2024-03-18T14:51:04.801 INFO:tasks.workunit.client.1.smithi100.stdout:1/458: readlink d8/d1e/d11/l72 0 2024-03-18T14:51:04.802 INFO:tasks.workunit.client.1.smithi100.stdout:3/480: mkdir d3/dd/d21/dae 0 2024-03-18T14:51:04.804 INFO:tasks.workunit.client.0.smithi012.stdout:7/784: symlink dd/d13/d23/d2f/d53/dc6/df1/l110 0 2024-03-18T14:51:04.804 INFO:tasks.workunit.client.0.smithi012.stdout:7/785: readlink dd/d4f/lad 0 2024-03-18T14:51:04.807 INFO:tasks.workunit.client.1.smithi100.stdout:0/529: dwrite dd/f8f [0,4194304] 0 2024-03-18T14:51:04.807 INFO:tasks.workunit.client.1.smithi100.stdout:0/530: write dd/f1a [1079002,79195] 0 2024-03-18T14:51:04.807 INFO:tasks.workunit.client.0.smithi012.stdout:3/760: dread dc/d3c/d59/d86/fa8 [0,4194304] 0 2024-03-18T14:51:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:5/765: dwrite d3/d15/f90 [0,4194304] 0 2024-03-18T14:51:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:5/766: write d3/d29/d35/fd4 [782343,54806] 0 2024-03-18T14:51:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:5/767: stat d3/d15/d42/dc6/d73/d6b/df3 0 2024-03-18T14:51:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:5/768: fsync d3/d29/d36/fb3 0 2024-03-18T14:51:04.810 INFO:tasks.workunit.client.0.smithi012.stdout:5/769: stat d3/d29/d36/d9b/cd1 0 2024-03-18T14:51:04.813 INFO:tasks.workunit.client.0.smithi012.stdout:0/792: mknod d1/d20/d23/d3b/dc1/c103 0 2024-03-18T14:51:04.814 INFO:tasks.workunit.client.0.smithi012.stdout:0/793: chown d1/d20/d23/d26/cc2 12179 1 2024-03-18T14:51:04.814 INFO:tasks.workunit.client.0.smithi012.stdout:0/794: chown d1/d20/d23/d82 70 1 2024-03-18T14:51:04.814 INFO:tasks.workunit.client.0.smithi012.stdout:0/795: creat d1/d20/d23/d7f/f104 x:0 0 0 2024-03-18T14:51:04.816 INFO:tasks.workunit.client.0.smithi012.stdout:4/692: mkdir d6/d1a/d30/de9 0 2024-03-18T14:51:04.818 INFO:tasks.workunit.client.0.smithi012.stdout:7/786: rmdir dd/d13/d23/d2f 39 2024-03-18T14:51:04.818 INFO:tasks.workunit.client.0.smithi012.stdout:7/787: write dd/d13/d23/d33/f84 [1215008,16039] 0 2024-03-18T14:51:04.819 INFO:tasks.workunit.client.1.smithi100.stdout:5/381: symlink d29/d2d/d32/l73 0 2024-03-18T14:51:04.821 INFO:tasks.workunit.client.0.smithi012.stdout:2/816: rename d0/d6b/dbf/d57 to d0/d6b/dbf/d33/d77/df5/d103 0 2024-03-18T14:51:04.823 INFO:tasks.workunit.client.0.smithi012.stdout:3/761: dwrite dc/d30/fac [0,4194304] 0 2024-03-18T14:51:04.823 INFO:tasks.workunit.client.0.smithi012.stdout:3/762: write dc/f92 [298192,114794] 0 2024-03-18T14:51:04.830 INFO:tasks.workunit.client.1.smithi100.stdout:9/422: getdents da/d51/d55 0 2024-03-18T14:51:04.830 INFO:tasks.workunit.client.1.smithi100.stdout:9/423: fdatasync da/d51/d55/f7b 0 2024-03-18T14:51:04.831 INFO:tasks.workunit.client.1.smithi100.stdout:9/424: creat da/dd/d29/d71/d72/f81 x:0 0 0 2024-03-18T14:51:04.831 INFO:tasks.workunit.client.1.smithi100.stdout:9/425: truncate da/dd/d29/f5a 65857 0 2024-03-18T14:51:04.831 INFO:tasks.workunit.client.1.smithi100.stdout:9/426: truncate da/d51/f69 918359 0 2024-03-18T14:51:04.836 INFO:tasks.workunit.client.1.smithi100.stdout:7/445: rename d5/d14/d16/d2e/f7d to d5/d14/d16/d44/fa7 0 2024-03-18T14:51:04.836 INFO:tasks.workunit.client.1.smithi100.stdout:7/446: truncate d5/d14/d16/d2c/d86/f75 484316 0 2024-03-18T14:51:04.841 INFO:tasks.workunit.client.1.smithi100.stdout:8/463: rename d4/f3a to d4/fa0 0 2024-03-18T14:51:04.848 INFO:tasks.workunit.client.0.smithi012.stdout:3/763: dwrite dc/d3c/d4c/d5f/fb0 [0,4194304] 0 2024-03-18T14:51:04.849 INFO:tasks.workunit.client.1.smithi100.stdout:2/421: mknod d3/d13/d4f/c94 0 2024-03-18T14:51:04.853 INFO:tasks.workunit.client.0.smithi012.stdout:0/796: stat d1/d16/d19/c2b 0 2024-03-18T14:51:04.856 INFO:tasks.workunit.client.0.smithi012.stdout:4/693: mknod d6/da/d89/cea 0 2024-03-18T14:51:04.857 INFO:tasks.workunit.client.1.smithi100.stdout:4/434: truncate d9/f3b 339661 0 2024-03-18T14:51:04.860 INFO:tasks.workunit.client.0.smithi012.stdout:7/788: link dd/d13/d23/d2f/l3c dd/d13/d23/d60/d9c/dfd/l111 0 2024-03-18T14:51:04.860 INFO:tasks.workunit.client.0.smithi012.stdout:7/789: creat dd/d13/d23/d33/f112 x:0 0 0 2024-03-18T14:51:04.860 INFO:tasks.workunit.client.0.smithi012.stdout:7/790: write dd/d13/d23/d76/d6f/f91 [4315175,55306] 0 2024-03-18T14:51:04.860 INFO:tasks.workunit.client.0.smithi012.stdout:7/791: readlink dd/d13/d23/d33/lb1 0 2024-03-18T14:51:04.862 INFO:tasks.workunit.client.0.smithi012.stdout:2/817: symlink d0/d6b/dbf/d33/d71/d90/dea/l104 0 2024-03-18T14:51:04.862 INFO:tasks.workunit.client.0.smithi012.stdout:2/818: chown d0/d6b/dbf/d33/d80/c83 63836127 1 2024-03-18T14:51:04.862 INFO:tasks.workunit.client.0.smithi012.stdout:2/819: fdatasync d0/d6b/dbf/d33/d80/fc8 0 2024-03-18T14:51:04.862 INFO:tasks.workunit.client.0.smithi012.stdout:2/820: write d0/db4/fb9 [4623807,70192] 0 2024-03-18T14:51:04.864 INFO:tasks.workunit.client.1.smithi100.stdout:1/459: link d8/d2a/d37/f5f d8/d2a/d37/d50/f85 0 2024-03-18T14:51:04.865 INFO:tasks.workunit.client.0.smithi012.stdout:4/694: dread d6/f25 [0,4194304] 0 2024-03-18T14:51:04.868 INFO:tasks.workunit.client.1.smithi100.stdout:3/481: symlink d3/d16/d13/d17/d88/laf 0 2024-03-18T14:51:04.870 INFO:tasks.workunit.client.0.smithi012.stdout:3/764: dwrite dc/d3c/d59/fdd [0,4194304] 0 2024-03-18T14:51:04.872 INFO:tasks.workunit.client.1.smithi100.stdout:0/531: creat dd/d2b/fc0 x:0 0 0 2024-03-18T14:51:04.872 INFO:tasks.workunit.client.1.smithi100.stdout:0/532: chown dd/d20/ca1 737026375 1 2024-03-18T14:51:04.874 INFO:tasks.workunit.client.0.smithi012.stdout:5/770: rmdir d3/d29/d35 39 2024-03-18T14:51:04.874 INFO:tasks.workunit.client.0.smithi012.stdout:5/771: chown d3/db6/fca 1839196410 1 2024-03-18T14:51:04.876 INFO:tasks.workunit.client.0.smithi012.stdout:7/792: dwrite dd/d13/d23/d33/f78 [4194304,4194304] 0 2024-03-18T14:51:04.876 INFO:tasks.workunit.client.0.smithi012.stdout:9/808: sync 2024-03-18T14:51:04.880 INFO:tasks.workunit.client.1.smithi100.stdout:5/382: mkdir d29/d2d/d39/d74 0 2024-03-18T14:51:04.885 INFO:tasks.workunit.client.0.smithi012.stdout:0/797: creat d1/d20/d23/f105 x:0 0 0 2024-03-18T14:51:04.885 INFO:tasks.workunit.client.0.smithi012.stdout:0/798: creat d1/d20/d23/d7f/f106 x:0 0 0 2024-03-18T14:51:04.886 INFO:tasks.workunit.client.0.smithi012.stdout:0/799: stat d1/d20/d23/d26/ld7 0 2024-03-18T14:51:04.887 INFO:tasks.workunit.client.1.smithi100.stdout:0/533: dwrite dd/d31/d91/d65/f74 [0,4194304] 0 2024-03-18T14:51:04.891 INFO:tasks.workunit.client.1.smithi100.stdout:6/435: sync 2024-03-18T14:51:04.891 INFO:tasks.workunit.client.1.smithi100.stdout:6/436: dread - d6/d7/df/d26/f84 zero size 2024-03-18T14:51:04.894 INFO:tasks.workunit.client.0.smithi012.stdout:4/695: rename d6/d27/c4a to d6/d1a/d30/d46/d6f/d8b/de7/ceb 0 2024-03-18T14:51:04.894 INFO:tasks.workunit.client.0.smithi012.stdout:4/696: creat d6/d19/d39/fec x:0 0 0 2024-03-18T14:51:04.894 INFO:tasks.workunit.client.0.smithi012.stdout:4/697: write d6/fe [859363,77766] 0 2024-03-18T14:51:04.896 INFO:tasks.workunit.client.1.smithi100.stdout:4/435: mkdir d9/d81 0 2024-03-18T14:51:04.902 INFO:tasks.workunit.client.1.smithi100.stdout:1/460: link d8/d30/d61/f63 d8/d1e/f86 0 2024-03-18T14:51:04.902 INFO:tasks.workunit.client.1.smithi100.stdout:1/461: fdatasync d8/d2a/d37/f39 0 2024-03-18T14:51:04.903 INFO:tasks.workunit.client.1.smithi100.stdout:1/462: creat d8/d1e/d11/d6d/f87 x:0 0 0 2024-03-18T14:51:04.903 INFO:tasks.workunit.client.0.smithi012.stdout:3/765: mkdir dc/d3c/d59/d86/d9a/d64/df4 0 2024-03-18T14:51:04.904 INFO:tasks.workunit.client.0.smithi012.stdout:4/698: dread d6/d1a/d30/d46/d6f/f71 [0,4194304] 0 2024-03-18T14:51:04.904 INFO:tasks.workunit.client.0.smithi012.stdout:4/699: read - d6/da/d89/d97/f9a zero size 2024-03-18T14:51:04.904 INFO:tasks.workunit.client.0.smithi012.stdout:4/700: fsync d6/d1a/d64/fcf 0 2024-03-18T14:51:04.904 INFO:tasks.workunit.client.0.smithi012.stdout:4/701: truncate d6/d1a/f31 1455224 0 2024-03-18T14:51:04.904 INFO:tasks.workunit.client.1.smithi100.stdout:3/482: rmdir d3/d16/d13/d1f/d7c 39 2024-03-18T14:51:04.905 INFO:tasks.workunit.client.0.smithi012.stdout:5/772: mkdir d3/d15/d42/dc6/dfe 0 2024-03-18T14:51:04.906 INFO:tasks.workunit.client.0.smithi012.stdout:5/773: write d3/d15/d5f/fc0 [448420,62296] 0 2024-03-18T14:51:04.906 INFO:tasks.workunit.client.0.smithi012.stdout:5/774: getdents d3/d29/d36/d9b/ddf/def 0 2024-03-18T14:51:04.906 INFO:tasks.workunit.client.1.smithi100.stdout:5/383: unlink fa 0 2024-03-18T14:51:04.909 INFO:tasks.workunit.client.1.smithi100.stdout:7/447: getdents d5 0 2024-03-18T14:51:04.912 INFO:tasks.workunit.client.0.smithi012.stdout:7/793: mkdir dd/d13/d23/d2f/d53/dc6/df1/d113 0 2024-03-18T14:51:04.912 INFO:tasks.workunit.client.0.smithi012.stdout:7/794: chown dd/d13/d23/d60/d9c/f10b 1669134 1 2024-03-18T14:51:04.916 INFO:tasks.workunit.client.0.smithi012.stdout:4/702: dread d6/d19/d39/fc4 [0,4194304] 0 2024-03-18T14:51:04.916 INFO:tasks.workunit.client.0.smithi012.stdout:4/703: chown d6/da/ce6 114930 1 2024-03-18T14:51:04.916 INFO:tasks.workunit.client.0.smithi012.stdout:4/704: readlink d6/da/d58/l77 0 2024-03-18T14:51:04.916 INFO:tasks.workunit.client.1.smithi100.stdout:0/534: mkdir dd/d2b/d95/dc1 0 2024-03-18T14:51:04.917 INFO:tasks.workunit.client.0.smithi012.stdout:0/800: mkdir d1/d16/d19/d8d/d107 0 2024-03-18T14:51:04.918 INFO:tasks.workunit.client.0.smithi012.stdout:3/766: write dc/d30/d35/f65 [2685097,103917] 0 2024-03-18T14:51:04.919 INFO:tasks.workunit.client.1.smithi100.stdout:8/464: getdents d4/dc/dd/d48 0 2024-03-18T14:51:04.920 INFO:tasks.workunit.client.1.smithi100.stdout:6/437: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/f73 [0,4194304] 0 2024-03-18T14:51:04.921 INFO:tasks.workunit.client.0.smithi012.stdout:7/795: dread dd/d13/d23/d76/d6b/f108 [0,4194304] 0 2024-03-18T14:51:04.921 INFO:tasks.workunit.client.0.smithi012.stdout:2/821: getdents d0/d3c/d40/d47 0 2024-03-18T14:51:04.921 INFO:tasks.workunit.client.0.smithi012.stdout:5/775: dwrite d3/d29/d35/dcc/feb [0,4194304] 0 2024-03-18T14:51:04.922 INFO:tasks.workunit.client.0.smithi012.stdout:1/715: sync 2024-03-18T14:51:04.922 INFO:tasks.workunit.client.0.smithi012.stdout:5/776: write d3/d15/d42/dc6/d73/d6b/df3/fde [189250,90762] 0 2024-03-18T14:51:04.922 INFO:tasks.workunit.client.0.smithi012.stdout:1/716: chown d8/db/d10/d16/da1/fe4 508864706 1 2024-03-18T14:51:04.922 INFO:tasks.workunit.client.0.smithi012.stdout:1/717: read d8/d1e/d27/fb7 [151913,126744] 0 2024-03-18T14:51:04.922 INFO:tasks.workunit.client.0.smithi012.stdout:1/718: write d8/db/d10/d16/d54/fa6 [1358072,25089] 0 2024-03-18T14:51:04.925 INFO:tasks.workunit.client.1.smithi100.stdout:3/483: creat d3/d16/d13/d17/dab/fb0 x:0 0 0 2024-03-18T14:51:04.925 INFO:tasks.workunit.client.0.smithi012.stdout:1/719: dread d8/db/d10/f33 [0,4194304] 0 2024-03-18T14:51:04.925 INFO:tasks.workunit.client.0.smithi012.stdout:1/720: creat d8/db/d10/d16/da1/d2e/d9c/fea x:0 0 0 2024-03-18T14:51:04.926 INFO:tasks.workunit.client.0.smithi012.stdout:1/721: dread - d8/db/d10/d31/d43/d5e/d65/d84/fcf zero size 2024-03-18T14:51:04.927 INFO:tasks.workunit.client.1.smithi100.stdout:5/384: rename d29/d2d/d32/d38/l6c to d29/l75 0 2024-03-18T14:51:04.927 INFO:tasks.workunit.client.0.smithi012.stdout:9/809: symlink d3/de6/dbf/d77/da8/l109 0 2024-03-18T14:51:04.928 INFO:tasks.workunit.client.1.smithi100.stdout:7/448: creat d5/d22/d3b/d9f/da0/fa8 x:0 0 0 2024-03-18T14:51:04.929 INFO:tasks.workunit.client.0.smithi012.stdout:4/705: link d6/d1a/d30/d46/d6f/d8b/fb0 d6/d1a/d30/d46/d6f/d7c/fed 0 2024-03-18T14:51:04.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/535: symlink dd/d31/d91/d56/lc2 0 2024-03-18T14:51:04.930 INFO:tasks.workunit.client.1.smithi100.stdout:0/536: stat dd/d31/c8d 0 2024-03-18T14:51:04.931 INFO:tasks.workunit.client.0.smithi012.stdout:0/801: mknod d1/d20/c108 0 2024-03-18T14:51:04.931 INFO:tasks.workunit.client.0.smithi012.stdout:0/802: stat d1/d20/d23/d7f/fca 0 2024-03-18T14:51:04.932 INFO:tasks.workunit.client.0.smithi012.stdout:3/767: dread dc/f22 [0,4194304] 0 2024-03-18T14:51:04.933 INFO:tasks.workunit.client.0.smithi012.stdout:7/796: truncate dd/d13/d23/d76/d6f/db3/dc7/fd8 1061702 0 2024-03-18T14:51:04.936 INFO:tasks.workunit.client.1.smithi100.stdout:3/484: creat d3/d16/d13/d17/d88/fb1 x:0 0 0 2024-03-18T14:51:04.940 INFO:tasks.workunit.client.0.smithi012.stdout:0/803: dread d1/f9 [0,4194304] 0 2024-03-18T14:51:04.940 INFO:tasks.workunit.client.0.smithi012.stdout:0/804: readlink d1/d2d/la2 0 2024-03-18T14:51:04.940 INFO:tasks.workunit.client.1.smithi100.stdout:7/449: symlink d5/d22/d24/la9 0 2024-03-18T14:51:04.942 INFO:tasks.workunit.client.0.smithi012.stdout:9/810: dread d3/de6/dbf/d77/f56 [0,4194304] 0 2024-03-18T14:51:04.942 INFO:tasks.workunit.client.0.smithi012.stdout:9/811: read - d3/d9/d5c/fad zero size 2024-03-18T14:51:04.942 INFO:tasks.workunit.client.0.smithi012.stdout:0/805: dread d1/d16/d19/d8d/fad [0,4194304] 0 2024-03-18T14:51:04.945 INFO:tasks.workunit.client.1.smithi100.stdout:0/537: mknod dd/d31/d91/d56/cc3 0 2024-03-18T14:51:04.947 INFO:tasks.workunit.client.1.smithi100.stdout:8/465: rename d4/d3c/l51 to d4/la1 0 2024-03-18T14:51:04.950 INFO:tasks.workunit.client.1.smithi100.stdout:0/538: mknod dd/d31/d91/db8/cc4 0 2024-03-18T14:51:04.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/438: rename d6/d7/df/d26/c31 to d6/d7/df/d72/d65/d7a/c86 0 2024-03-18T14:51:04.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/439: stat d6/d7/df/d72/d65 0 2024-03-18T14:51:04.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/440: fdatasync d6/d12/d2c/f4c 0 2024-03-18T14:51:04.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/441: truncate d6/d7/f8 1702965 0 2024-03-18T14:51:04.958 INFO:tasks.workunit.client.1.smithi100.stdout:5/385: dwrite f23 [0,4194304] 0 2024-03-18T14:51:04.961 INFO:tasks.workunit.client.1.smithi100.stdout:0/539: creat dd/d31/d91/d56/d62/d6b/fc5 x:0 0 0 2024-03-18T14:51:04.964 INFO:tasks.workunit.client.1.smithi100.stdout:7/450: rename d5/d14/d16/d2e/d62/d66/d4f/c7c to d5/d22/d24/d6e/caa 0 2024-03-18T14:51:04.964 INFO:tasks.workunit.client.1.smithi100.stdout:7/451: truncate d5/d14/d16/d2c/d86/f7f 978816 0 2024-03-18T14:51:04.967 INFO:tasks.workunit.client.0.smithi012.stdout:5/777: symlink d3/d15/d42/dc6/d73/d6b/d8f/lff 0 2024-03-18T14:51:04.969 INFO:tasks.workunit.client.1.smithi100.stdout:6/442: link d6/d7/df/d26/d4e/d66/d36/d4f/d71/c29 d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d63/c87 0 2024-03-18T14:51:04.969 INFO:tasks.workunit.client.1.smithi100.stdout:6/443: dread - d6/d7/df/d26/f84 zero size 2024-03-18T14:51:04.969 INFO:tasks.workunit.client.1.smithi100.stdout:6/444: readlink d6/d7/df/d72/d48/l69 0 2024-03-18T14:51:04.972 INFO:tasks.workunit.client.1.smithi100.stdout:8/466: unlink d4/l24 0 2024-03-18T14:51:04.975 INFO:tasks.workunit.client.0.smithi012.stdout:1/722: creat d8/feb x:0 0 0 2024-03-18T14:51:04.977 INFO:tasks.workunit.client.0.smithi012.stdout:4/706: creat d6/d1a/d30/d46/d6f/d8b/fee x:0 0 0 2024-03-18T14:51:04.980 INFO:tasks.workunit.client.0.smithi012.stdout:4/707: write d6/d1a/f31 [521375,89619] 0 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.1.smithi100.stdout:0/540: creat dd/d2b/d95/d79/d7d/da9/dac/fc6 x:0 0 0 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.1.smithi100.stdout:0/541: fdatasync dd/d2b/f3d 0 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.1.smithi100.stdout:5/386: dwrite d29/d2d/d32/f35 [4194304,4194304] 0 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:8/936: sync 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:6/700: sync 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:6/701: dread - d17/d26/d36/d4d/d82/fb0 zero size 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:4/708: dread d6/db6/fd9 [0,4194304] 0 2024-03-18T14:51:04.982 INFO:tasks.workunit.client.0.smithi012.stdout:2/822: getdents d0/d6b/dbf/d33/d71 0 2024-03-18T14:51:04.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/823: dread d0/d6b/dbf/f24 [0,4194304] 0 2024-03-18T14:51:04.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/824: dread - d0/d6b/fe7 zero size 2024-03-18T14:51:04.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/825: write d0/d6b/d9f/fc5 [282640,58778] 0 2024-03-18T14:51:04.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/826: read d0/da2/d59/fa4 [463735,80026] 0 2024-03-18T14:51:04.986 INFO:tasks.workunit.client.0.smithi012.stdout:2/827: creat d0/d6b/dbf/d4f/f105 x:0 0 0 2024-03-18T14:51:04.987 INFO:tasks.workunit.client.0.smithi012.stdout:9/812: mknod d3/de6/dbf/d66/d21/d35/d48/dda/c10a 0 2024-03-18T14:51:04.989 INFO:tasks.workunit.client.0.smithi012.stdout:0/806: mkdir d1/d16/d6e/d109 0 2024-03-18T14:51:04.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/467: mkdir d4/dc/dd/de/d19/d2c/da2 0 2024-03-18T14:51:04.991 INFO:tasks.workunit.client.1.smithi100.stdout:8/468: dread - d4/dc/dd/de/d21/d23/f78 zero size 2024-03-18T14:51:04.993 INFO:tasks.workunit.client.0.smithi012.stdout:5/778: rename d3/d29/d36/d9b/f71 to d3/d15/d42/dc6/dfe/f100 0 2024-03-18T14:51:04.995 INFO:tasks.workunit.client.0.smithi012.stdout:2/828: dread d0/f97 [0,4194304] 0 2024-03-18T14:51:04.995 INFO:tasks.workunit.client.1.smithi100.stdout:0/542: unlink dd/d2b/d95/d79/lbb 0 2024-03-18T14:51:04.995 INFO:tasks.workunit.client.1.smithi100.stdout:0/543: fdatasync dd/d2b/f3d 0 2024-03-18T14:51:04.996 INFO:tasks.workunit.client.0.smithi012.stdout:7/797: rmdir dd/d13/d23/d2f/d7a/dab/dbc 39 2024-03-18T14:51:04.996 INFO:tasks.workunit.client.0.smithi012.stdout:7/798: creat dd/d13/d23/d76/d6f/db3/f114 x:0 0 0 2024-03-18T14:51:04.996 INFO:tasks.workunit.client.0.smithi012.stdout:7/799: creat dd/d13/d23/d33/dcc/f115 x:0 0 0 2024-03-18T14:51:04.999 INFO:tasks.workunit.client.1.smithi100.stdout:9/427: sync 2024-03-18T14:51:04.999 INFO:tasks.workunit.client.0.smithi012.stdout:8/937: symlink d0/d5a/d5e/dd7/d84/l12c 0 2024-03-18T14:51:05.000 INFO:tasks.workunit.client.1.smithi100.stdout:8/469: write d4/dc/dd/de/d19/d37/f65 [3976411,84653] 0 2024-03-18T14:51:05.000 INFO:tasks.workunit.client.1.smithi100.stdout:5/387: rename d29/d2d/d32/d38/f52 to d29/d2d/d32/d38/f76 0 2024-03-18T14:51:05.002 INFO:tasks.workunit.client.0.smithi012.stdout:4/709: rmdir d6/d1a/d30 39 2024-03-18T14:51:05.002 INFO:tasks.workunit.client.0.smithi012.stdout:4/710: dread - d6/d19/d39/fec zero size 2024-03-18T14:51:05.004 INFO:tasks.workunit.client.1.smithi100.stdout:7/452: getdents d5/d22 0 2024-03-18T14:51:05.004 INFO:tasks.workunit.client.1.smithi100.stdout:7/453: chown d5/d14/l2a 160 1 2024-03-18T14:51:05.007 INFO:tasks.workunit.client.1.smithi100.stdout:6/445: unlink d6/d7/df/f1e 0 2024-03-18T14:51:05.009 INFO:tasks.workunit.client.0.smithi012.stdout:9/813: mkdir d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/d10b 0 2024-03-18T14:51:05.013 INFO:tasks.workunit.client.1.smithi100.stdout:0/544: dwrite dd/f2e [0,4194304] 0 2024-03-18T14:51:05.014 INFO:tasks.workunit.client.0.smithi012.stdout:0/807: symlink d1/d20/d4e/d54/d8a/l10a 0 2024-03-18T14:51:05.014 INFO:tasks.workunit.client.1.smithi100.stdout:2/422: sync 2024-03-18T14:51:05.018 INFO:tasks.workunit.client.0.smithi012.stdout:1/723: rename d8/db/d10/lca to d8/db/d10/d31/d43/d5e/d65/d84/dc0/lec 0 2024-03-18T14:51:05.020 INFO:tasks.workunit.client.0.smithi012.stdout:5/779: creat d3/d84/dfa/f101 x:0 0 0 2024-03-18T14:51:05.021 INFO:tasks.workunit.client.1.smithi100.stdout:7/454: dread d5/d14/d16/d2e/d62/d66/f54 [4194304,4194304] 0 2024-03-18T14:51:05.023 INFO:tasks.workunit.client.1.smithi100.stdout:9/428: unlink da/d46/d4d/l47 0 2024-03-18T14:51:05.026 INFO:tasks.workunit.client.0.smithi012.stdout:0/808: dread d1/d20/f2e [0,4194304] 0 2024-03-18T14:51:05.029 INFO:tasks.workunit.client.0.smithi012.stdout:2/829: creat d0/d6b/f106 x:0 0 0 2024-03-18T14:51:05.029 INFO:tasks.workunit.client.0.smithi012.stdout:2/830: readlink d0/d6c/l76 0 2024-03-18T14:51:05.032 INFO:tasks.workunit.client.1.smithi100.stdout:8/470: unlink d4/dc/d1f/d59/d6c/f80 0 2024-03-18T14:51:05.032 INFO:tasks.workunit.client.1.smithi100.stdout:8/471: chown d4/dc/dd/de/d19/l4c 399 1 2024-03-18T14:51:05.034 INFO:tasks.workunit.client.1.smithi100.stdout:4/436: sync 2024-03-18T14:51:05.034 INFO:tasks.workunit.client.1.smithi100.stdout:1/463: sync 2024-03-18T14:51:05.034 INFO:tasks.workunit.client.1.smithi100.stdout:3/485: sync 2024-03-18T14:51:05.034 INFO:tasks.workunit.client.1.smithi100.stdout:3/486: fsync d3/dd/d21/f55 0 2024-03-18T14:51:05.035 INFO:tasks.workunit.client.1.smithi100.stdout:6/446: dwrite d6/d7/df/d26/f56 [0,4194304] 0 2024-03-18T14:51:05.035 INFO:tasks.workunit.client.0.smithi012.stdout:0/809: dread d1/d20/d7b/ff [0,4194304] 0 2024-03-18T14:51:05.035 INFO:tasks.workunit.client.1.smithi100.stdout:5/388: creat d29/d2d/d39/d71/f77 x:0 0 0 2024-03-18T14:51:05.036 INFO:tasks.workunit.client.0.smithi012.stdout:1/724: dwrite d8/d1e/d27/f38 [0,4194304] 0 2024-03-18T14:51:05.039 INFO:tasks.workunit.client.0.smithi012.stdout:7/800: symlink dd/d13/d23/d2f/d53/d8d/l116 0 2024-03-18T14:51:05.039 INFO:tasks.workunit.client.0.smithi012.stdout:7/801: fdatasync dd/d13/d23/d2f/d53/f5d 0 2024-03-18T14:51:05.042 INFO:tasks.workunit.client.0.smithi012.stdout:8/938: creat d0/d5a/d5e/dd7/d122/f12d x:0 0 0 2024-03-18T14:51:05.045 INFO:tasks.workunit.client.1.smithi100.stdout:0/545: rename dd/fe to dd/d2b/d95/d79/fc7 0 2024-03-18T14:51:05.046 INFO:tasks.workunit.client.0.smithi012.stdout:4/711: mknod d6/db6/cef 0 2024-03-18T14:51:05.046 INFO:tasks.workunit.client.0.smithi012.stdout:4/712: stat d6/d27/ldf 0 2024-03-18T14:51:05.048 INFO:tasks.workunit.client.1.smithi100.stdout:9/429: creat da/dd/d29/d71/f82 x:0 0 0 2024-03-18T14:51:05.049 INFO:tasks.workunit.client.0.smithi012.stdout:9/814: mknod d3/de6/dbf/d77/d52/d95/c10c 0 2024-03-18T14:51:05.049 INFO:tasks.workunit.client.0.smithi012.stdout:9/815: chown d3/d9/d5c/dbd 745202 1 2024-03-18T14:51:05.049 INFO:tasks.workunit.client.0.smithi012.stdout:9/816: fsync d3/d9/fd 0 2024-03-18T14:51:05.050 INFO:tasks.workunit.client.1.smithi100.stdout:8/472: creat d4/dc/d97/fa3 x:0 0 0 2024-03-18T14:51:05.052 INFO:tasks.workunit.client.0.smithi012.stdout:6/702: rename d17/d19/d27/d58/f7c to d17/d19/d98/fd2 0 2024-03-18T14:51:05.054 INFO:tasks.workunit.client.1.smithi100.stdout:1/464: getdents d8/d1e/d11 0 2024-03-18T14:51:05.056 INFO:tasks.workunit.client.0.smithi012.stdout:5/780: truncate d3/d29/d35/dcc/feb 3782261 0 2024-03-18T14:51:05.057 INFO:tasks.workunit.client.1.smithi100.stdout:4/437: getdents d9/d17 0 2024-03-18T14:51:05.058 INFO:tasks.workunit.client.0.smithi012.stdout:2/831: mkdir d0/d107 0 2024-03-18T14:51:05.060 INFO:tasks.workunit.client.0.smithi012.stdout:9/817: dread d3/de6/dbf/d77/d52/fc0 [0,4194304] 0 2024-03-18T14:51:05.061 INFO:tasks.workunit.client.1.smithi100.stdout:4/438: dread d9/f22 [0,4194304] 0 2024-03-18T14:51:05.062 INFO:tasks.workunit.client.1.smithi100.stdout:3/487: unlink d3/d16/c20 0 2024-03-18T14:51:05.062 INFO:tasks.workunit.client.1.smithi100.stdout:3/488: fsync d3/d16/d13/d17/f7a 0 2024-03-18T14:51:05.063 INFO:tasks.workunit.client.0.smithi012.stdout:0/810: link d1/d16/d79/ff5 d1/d16/d6e/f10b 0 2024-03-18T14:51:05.067 INFO:tasks.workunit.client.1.smithi100.stdout:4/439: dread d9/d27/f3d [0,4194304] 0 2024-03-18T14:51:05.067 INFO:tasks.workunit.client.1.smithi100.stdout:4/440: fdatasync d9/fe 0 2024-03-18T14:51:05.067 INFO:tasks.workunit.client.1.smithi100.stdout:4/441: chown d9/d27/d42/f72 1554 1 2024-03-18T14:51:05.067 INFO:tasks.workunit.client.1.smithi100.stdout:4/442: dread - d9/f7f zero size 2024-03-18T14:51:05.069 INFO:tasks.workunit.client.0.smithi012.stdout:1/725: mknod d8/d1e/d27/ced 0 2024-03-18T14:51:05.069 INFO:tasks.workunit.client.0.smithi012.stdout:1/726: chown d8/db/d10/d16/da1/l1f 22 1 2024-03-18T14:51:05.069 INFO:tasks.workunit.client.1.smithi100.stdout:3/489: write d3/dd/d21/d3e/f79 [6457145,14664] 0 2024-03-18T14:51:05.071 INFO:tasks.workunit.client.1.smithi100.stdout:6/447: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f88 x:0 0 0 2024-03-18T14:51:05.071 INFO:tasks.workunit.client.1.smithi100.stdout:6/448: creat d6/d12/f89 x:0 0 0 2024-03-18T14:51:05.072 INFO:tasks.workunit.client.0.smithi012.stdout:7/802: dwrite dd/f45 [0,4194304] 0 2024-03-18T14:51:05.072 INFO:tasks.workunit.client.0.smithi012.stdout:7/803: creat dd/f117 x:0 0 0 2024-03-18T14:51:05.073 INFO:tasks.workunit.client.0.smithi012.stdout:4/713: mkdir d6/d19/d6c/d96/df0 0 2024-03-18T14:51:05.074 INFO:tasks.workunit.client.1.smithi100.stdout:5/389: mknod d29/d2d/d39/d71/c78 0 2024-03-18T14:51:05.074 INFO:tasks.workunit.client.1.smithi100.stdout:5/390: write f2 [1222442,122348] 0 2024-03-18T14:51:05.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/391: chown l3 156118811 1 2024-03-18T14:51:05.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/392: dread - d29/d2d/d32/f68 zero size 2024-03-18T14:51:05.075 INFO:tasks.workunit.client.1.smithi100.stdout:1/465: dwrite d8/d2a/d37/f5d [4194304,4194304] 0 2024-03-18T14:51:05.075 INFO:tasks.workunit.client.1.smithi100.stdout:1/466: stat d8/d1e/d11/f1b 0 2024-03-18T14:51:05.076 INFO:tasks.workunit.client.0.smithi012.stdout:7/804: dread dd/d13/d23/fdc [0,4194304] 0 2024-03-18T14:51:05.080 INFO:tasks.workunit.client.1.smithi100.stdout:4/443: write d9/f11 [956817,5718] 0 2024-03-18T14:51:05.084 INFO:tasks.workunit.client.1.smithi100.stdout:0/546: creat dd/d31/d91/d65/fc8 x:0 0 0 2024-03-18T14:51:05.086 INFO:tasks.workunit.client.0.smithi012.stdout:6/703: unlink d17/d26/c46 0 2024-03-18T14:51:05.087 INFO:tasks.workunit.client.0.smithi012.stdout:6/704: chown d17/d26/d36/d4d/f7e 12803494 1 2024-03-18T14:51:05.087 INFO:tasks.workunit.client.0.smithi012.stdout:6/705: stat d17/d38/d5b/f80 0 2024-03-18T14:51:05.087 INFO:tasks.workunit.client.0.smithi012.stdout:6/706: write d17/d38/d5b/f80 [499638,114304] 0 2024-03-18T14:51:05.088 INFO:tasks.workunit.client.1.smithi100.stdout:9/430: truncate da/dd/f10 1286623 0 2024-03-18T14:51:05.089 INFO:tasks.workunit.client.1.smithi100.stdout:9/431: readlink da/dd/d29/d71/l7f 0 2024-03-18T14:51:05.092 INFO:tasks.workunit.client.0.smithi012.stdout:8/939: rename d0/d5a/d19/dad/l120 to d0/d5a/d5e/ddf/dfc/l12e 0 2024-03-18T14:51:05.095 INFO:tasks.workunit.client.0.smithi012.stdout:2/832: unlink d0/d6b/dbf/d31/ca6 0 2024-03-18T14:51:05.097 INFO:tasks.workunit.client.1.smithi100.stdout:6/449: mkdir d6/d7/df/d26/d4e/d66/d36/d4f/d8a 0 2024-03-18T14:51:05.097 INFO:tasks.workunit.client.1.smithi100.stdout:6/450: fdatasync d6/d7/df/d26/f1c 0 2024-03-18T14:51:05.099 INFO:tasks.workunit.client.0.smithi012.stdout:9/818: rmdir d3/de6/dbf/d77/d52/d95/deb 39 2024-03-18T14:51:05.102 INFO:tasks.workunit.client.1.smithi100.stdout:5/393: mknod d29/d2d/d39/d65/c79 0 2024-03-18T14:51:05.103 INFO:tasks.workunit.client.0.smithi012.stdout:0/811: mkdir d1/d20/d4e/d54/d67/d10c 0 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:3/490: dwrite d3/d16/d13/d1f/f97 [0,4194304] 0 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:1/467: truncate d8/d2a/f47 469097 0 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:1/468: read - d8/d2a/d37/d50/f85 zero size 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:1/469: creat d8/d1e/d11/d6d/f88 x:0 0 0 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:1/470: dread - d8/d2a/d37/d50/f85 zero size 2024-03-18T14:51:05.106 INFO:tasks.workunit.client.1.smithi100.stdout:1/471: dread - d8/d1e/d11/d27/d3f/f78 zero size 2024-03-18T14:51:05.107 INFO:tasks.workunit.client.1.smithi100.stdout:1/472: fsync d8/d1e/d11/d27/f4d 0 2024-03-18T14:51:05.110 INFO:tasks.workunit.client.0.smithi012.stdout:7/805: dwrite dd/d13/d23/f99 [0,4194304] 0 2024-03-18T14:51:05.110 INFO:tasks.workunit.client.0.smithi012.stdout:1/727: dwrite d8/db/d10/d31/d81/fb3 [0,4194304] 0 2024-03-18T14:51:05.112 INFO:tasks.workunit.client.0.smithi012.stdout:4/714: creat d6/d19/d6c/d96/df0/ff1 x:0 0 0 2024-03-18T14:51:05.112 INFO:tasks.workunit.client.0.smithi012.stdout:4/715: write d6/d1a/d30/fc8 [603023,116425] 0 2024-03-18T14:51:05.112 INFO:tasks.workunit.client.0.smithi012.stdout:8/940: dwrite d0/d5a/d19/d57/f76 [0,4194304] 0 2024-03-18T14:51:05.113 INFO:tasks.workunit.client.0.smithi012.stdout:8/941: fdatasync d0/d5a/d5e/dd7/d84/d54/d9f/f10f 0 2024-03-18T14:51:05.114 INFO:tasks.workunit.client.0.smithi012.stdout:6/707: rename d17/d26/d36/f4b to d17/d33/fd3 0 2024-03-18T14:51:05.114 INFO:tasks.workunit.client.1.smithi100.stdout:6/451: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f70 [0,4194304] 0 2024-03-18T14:51:05.115 INFO:tasks.workunit.client.1.smithi100.stdout:6/452: chown d6/d7/df/d26/c3e 12 1 2024-03-18T14:51:05.115 INFO:tasks.workunit.client.1.smithi100.stdout:4/444: mkdir d9/d27/d62/d7b/d82 0 2024-03-18T14:51:05.117 INFO:tasks.workunit.client.0.smithi012.stdout:9/819: creat d3/de6/dbf/d77/d52/d95/deb/f10d x:0 0 0 2024-03-18T14:51:05.123 INFO:tasks.workunit.client.1.smithi100.stdout:0/547: link f6 dd/fc9 0 2024-03-18T14:51:05.127 INFO:tasks.workunit.client.1.smithi100.stdout:9/432: symlink da/d25/l83 0 2024-03-18T14:51:05.133 INFO:tasks.workunit.client.1.smithi100.stdout:5/394: symlink d29/l7a 0 2024-03-18T14:51:05.142 INFO:tasks.workunit.client.0.smithi012.stdout:7/806: link dd/d13/d23/d2f/d7a/db6/lc5 dd/d13/d23/d76/d6f/l118 0 2024-03-18T14:51:05.142 INFO:tasks.workunit.client.0.smithi012.stdout:7/807: dread - dd/d13/d23/d2f/d53/ff7 zero size 2024-03-18T14:51:05.143 INFO:tasks.workunit.client.1.smithi100.stdout:3/491: symlink d3/d16/d13/d1f/d3f/d4f/d65/lb2 0 2024-03-18T14:51:05.144 INFO:tasks.workunit.client.0.smithi012.stdout:1/728: creat d8/d1e/d27/d44/dc2/fee x:0 0 0 2024-03-18T14:51:05.144 INFO:tasks.workunit.client.0.smithi012.stdout:1/729: dread - d8/db/d10/d31/d43/d5e/fbc zero size 2024-03-18T14:51:05.144 INFO:tasks.workunit.client.1.smithi100.stdout:4/445: dwrite d9/d27/d62/f7d [0,4194304] 0 2024-03-18T14:51:05.145 INFO:tasks.workunit.client.0.smithi012.stdout:4/716: dwrite d6/fe [0,4194304] 0 2024-03-18T14:51:05.145 INFO:tasks.workunit.client.0.smithi012.stdout:4/717: write d6/d1a/d92/f5d [633756,64569] 0 2024-03-18T14:51:05.145 INFO:tasks.workunit.client.1.smithi100.stdout:1/473: unlink l3 0 2024-03-18T14:51:05.146 INFO:tasks.workunit.client.1.smithi100.stdout:0/548: dwrite dd/d20/f28 [0,4194304] 0 2024-03-18T14:51:05.148 INFO:tasks.workunit.client.0.smithi012.stdout:9/820: dwrite d3/d9/fd5 [0,4194304] 0 2024-03-18T14:51:05.148 INFO:tasks.workunit.client.0.smithi012.stdout:9/821: chown d3/d9/fa2 20106 1 2024-03-18T14:51:05.148 INFO:tasks.workunit.client.0.smithi012.stdout:9/822: chown d3/de6/dbf/d77/d52/d99/da0/ddf 1 1 2024-03-18T14:51:05.148 INFO:tasks.workunit.client.1.smithi100.stdout:2/423: rename d3/d1a/d69 to d3/da/de/d20/d82/d95 0 2024-03-18T14:51:05.155 INFO:tasks.workunit.client.0.smithi012.stdout:8/942: rmdir d0/d5a/d5e/ddf 39 2024-03-18T14:51:05.155 INFO:tasks.workunit.client.0.smithi012.stdout:8/943: chown d0/d5a/d19/d63 1139936 1 2024-03-18T14:51:05.155 INFO:tasks.workunit.client.0.smithi012.stdout:8/944: readlink d0/d47/la0 0 2024-03-18T14:51:05.157 INFO:tasks.workunit.client.0.smithi012.stdout:7/808: dread dd/d13/d23/d33/fff [0,4194304] 0 2024-03-18T14:51:05.157 INFO:tasks.workunit.client.1.smithi100.stdout:6/453: readlink d6/d7/df/l57 0 2024-03-18T14:51:05.159 INFO:tasks.workunit.client.0.smithi012.stdout:6/708: creat d17/d26/d36/d4d/d82/d97/fd4 x:0 0 0 2024-03-18T14:51:05.159 INFO:tasks.workunit.client.0.smithi012.stdout:6/709: fsync d17/d26/d36/d4d/d82/f8b 0 2024-03-18T14:51:05.159 INFO:tasks.workunit.client.0.smithi012.stdout:6/710: chown f8 21 1 2024-03-18T14:51:05.162 INFO:tasks.workunit.client.0.smithi012.stdout:7/809: dread dd/d13/d23/d76/f102 [0,4194304] 0 2024-03-18T14:51:05.163 INFO:tasks.workunit.client.0.smithi012.stdout:0/812: rmdir d1/d20/d23/d7f/da7 39 2024-03-18T14:51:05.164 INFO:tasks.workunit.client.1.smithi100.stdout:9/433: creat da/dd/d3c/d6b/f84 x:0 0 0 2024-03-18T14:51:05.166 INFO:tasks.workunit.client.0.smithi012.stdout:1/730: symlink d8/db/d10/d16/d6b/d7e/lef 0 2024-03-18T14:51:05.166 INFO:tasks.workunit.client.1.smithi100.stdout:5/395: rmdir d29/d2d/d32/d38 39 2024-03-18T14:51:05.169 INFO:tasks.workunit.client.1.smithi100.stdout:0/549: dwrite dd/d2b/f89 [0,4194304] 0 2024-03-18T14:51:05.169 INFO:tasks.workunit.client.1.smithi100.stdout:0/550: fsync dd/d2b/f40 0 2024-03-18T14:51:05.169 INFO:tasks.workunit.client.1.smithi100.stdout:3/492: truncate d3/ff 264439 0 2024-03-18T14:51:05.177 INFO:tasks.workunit.client.0.smithi012.stdout:9/823: creat d3/de6/dbf/d66/d69/d7f/d8c/f10e x:0 0 0 2024-03-18T14:51:05.178 INFO:tasks.workunit.client.1.smithi100.stdout:4/446: symlink d9/d13/d39/d76/l83 0 2024-03-18T14:51:05.179 INFO:tasks.workunit.client.1.smithi100.stdout:0/551: dread dd/d31/d91/f66 [0,4194304] 0 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/493: dwrite d3/dd/f6a [0,4194304] 0 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/494: chown d3/d16/d13/d1f/d37 519642565 1 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/495: write d3/d16/d13/d1f/d3f/d4f/f68 [1441533,53122] 0 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/496: readlink d3/dd/l26 0 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/497: write d3/dd/f28 [4521011,60217] 0 2024-03-18T14:51:05.190 INFO:tasks.workunit.client.1.smithi100.stdout:3/498: chown d3/d16/d13/f39 68727 1 2024-03-18T14:51:05.191 INFO:tasks.workunit.client.1.smithi100.stdout:7/455: rename d5/d14/d16/d44/f76 to d5/d14/fab 0 2024-03-18T14:51:05.191 INFO:tasks.workunit.client.1.smithi100.stdout:7/456: chown d5/f52 368 1 2024-03-18T14:51:05.191 INFO:tasks.workunit.client.1.smithi100.stdout:7/457: dread - d5/d14/d16/d2c/f74 zero size 2024-03-18T14:51:05.191 INFO:tasks.workunit.client.1.smithi100.stdout:3/499: write d3/d16/d13/d1f/d37/f54 [1361770,769] 0 2024-03-18T14:51:05.191 INFO:tasks.workunit.client.1.smithi100.stdout:3/500: creat d3/d16/d13/d1f/d37/fb3 x:0 0 0 2024-03-18T14:51:05.192 INFO:tasks.workunit.client.1.smithi100.stdout:3/501: write d3/d16/d13/d1f/d3f/d4f/d60/f63 [250444,104999] 0 2024-03-18T14:51:05.192 INFO:tasks.workunit.client.1.smithi100.stdout:3/502: write d3/dd/d49/f70 [4518481,109939] 0 2024-03-18T14:51:05.193 INFO:tasks.workunit.client.1.smithi100.stdout:2/424: rmdir d3/d13 39 2024-03-18T14:51:05.193 INFO:tasks.workunit.client.1.smithi100.stdout:2/425: write d3/d6/d5f/f68 [30537,112107] 0 2024-03-18T14:51:05.193 INFO:tasks.workunit.client.1.smithi100.stdout:3/503: read d3/d16/d13/d1f/f6c [467086,52363] 0 2024-03-18T14:51:05.196 INFO:tasks.workunit.client.0.smithi012.stdout:6/711: fsync d17/d38/fc7 0 2024-03-18T14:51:05.196 INFO:tasks.workunit.client.0.smithi012.stdout:6/712: readlink d17/d26/l9c 0 2024-03-18T14:51:05.198 INFO:tasks.workunit.client.0.smithi012.stdout:7/810: mkdir dd/d119 0 2024-03-18T14:51:05.201 INFO:tasks.workunit.client.0.smithi012.stdout:1/731: creat d8/dbf/ff0 x:0 0 0 2024-03-18T14:51:05.202 INFO:tasks.workunit.client.0.smithi012.stdout:1/732: write d8/d1e/f63 [93102,54966] 0 2024-03-18T14:51:05.202 INFO:tasks.workunit.client.1.smithi100.stdout:5/396: symlink d29/d2d/d39/d65/l7b 0 2024-03-18T14:51:05.204 INFO:tasks.workunit.client.0.smithi012.stdout:9/824: symlink d3/de6/dbf/l10f 0 2024-03-18T14:51:05.204 INFO:tasks.workunit.client.0.smithi012.stdout:9/825: write d3/d9/fd [1103767,72820] 0 2024-03-18T14:51:05.207 INFO:tasks.workunit.client.0.smithi012.stdout:8/945: rmdir d0/d5a/d5e/dd7/d84/d54/d9f 39 2024-03-18T14:51:05.207 INFO:tasks.workunit.client.0.smithi012.stdout:8/946: fsync d0/d5a/d5e/dd7/d84/d54/fb2 0 2024-03-18T14:51:05.207 INFO:tasks.workunit.client.0.smithi012.stdout:9/826: write d3/de6/dbf/d66/f51 [1321142,119247] 0 2024-03-18T14:51:05.208 INFO:tasks.workunit.client.1.smithi100.stdout:2/426: dread d3/d13/d40/d53/f6f [0,4194304] 0 2024-03-18T14:51:05.211 INFO:tasks.workunit.client.0.smithi012.stdout:6/713: write d17/d19/f6d [573588,101103] 0 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/714: write d17/d19/d27/f54 [5163098,97227] 0 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/715: truncate d17/d19/d98/fd2 768358 0 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.0.smithi012.stdout:6/716: stat d17/d19/d27/d2f/l32 0 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.0.smithi012.stdout:3/768: sync 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.1.smithi100.stdout:0/552: dwrite dd/d31/d91/d56/d71/f90 [0,4194304] 0 2024-03-18T14:51:05.212 INFO:tasks.workunit.client.1.smithi100.stdout:0/553: chown dd/d20/f4d 1 1 2024-03-18T14:51:05.213 INFO:tasks.workunit.client.1.smithi100.stdout:8/473: rename d4/dc/d1f/f5c to d4/dc/dd/de/d21/d23/d47/fa4 0 2024-03-18T14:51:05.217 INFO:tasks.workunit.client.0.smithi012.stdout:7/811: link dd/d13/d23/d60/l70 dd/d13/d23/l11a 0 2024-03-18T14:51:05.217 INFO:tasks.workunit.client.0.smithi012.stdout:7/812: creat dd/d13/d23/d76/d6b/f11b x:0 0 0 2024-03-18T14:51:05.218 INFO:tasks.workunit.client.1.smithi100.stdout:4/447: dwrite d9/f10 [8388608,4194304] 0 2024-03-18T14:51:05.220 INFO:tasks.workunit.client.0.smithi012.stdout:9/827: read d3/de6/dbf/d66/d69/d7f/d8c/d8b/f9f [1945175,85424] 0 2024-03-18T14:51:05.220 INFO:tasks.workunit.client.0.smithi012.stdout:9/828: write d3/de6/dbf/d77/f56 [204485,41899] 0 2024-03-18T14:51:05.222 INFO:tasks.workunit.client.0.smithi012.stdout:0/813: symlink d1/d20/d7b/l10d 0 2024-03-18T14:51:05.222 INFO:tasks.workunit.client.0.smithi012.stdout:0/814: chown d1/d20/d23/d3b/d33/d51/la3 24 1 2024-03-18T14:51:05.226 INFO:tasks.workunit.client.0.smithi012.stdout:1/733: creat d8/db/d10/d4e/ff1 x:0 0 0 2024-03-18T14:51:05.226 INFO:tasks.workunit.client.0.smithi012.stdout:1/734: write d8/db/d10/d31/d43/d5e/d65/fda [646491,56109] 0 2024-03-18T14:51:05.226 INFO:tasks.workunit.client.0.smithi012.stdout:1/735: readlink d8/db/d10/d31/d43/d5e/d65/d70/la2 0 2024-03-18T14:51:05.226 INFO:tasks.workunit.client.1.smithi100.stdout:0/554: dread dd/f1b [0,4194304] 0 2024-03-18T14:51:05.227 INFO:tasks.workunit.client.0.smithi012.stdout:1/736: write d8/db/d10/d4e/f53 [729552,31602] 0 2024-03-18T14:51:05.231 INFO:tasks.workunit.client.0.smithi012.stdout:8/947: link d0/d5a/d19/ffd d0/d5a/d5e/dd7/d84/d54/f12f 0 2024-03-18T14:51:05.234 INFO:tasks.workunit.client.1.smithi100.stdout:3/504: truncate d3/d16/d13/d17/f7f 3284322 0 2024-03-18T14:51:05.234 INFO:tasks.workunit.client.1.smithi100.stdout:3/505: stat d3/dd/d21/f22 0 2024-03-18T14:51:05.237 INFO:tasks.workunit.client.0.smithi012.stdout:6/717: unlink ff 0 2024-03-18T14:51:05.243 INFO:tasks.workunit.client.0.smithi012.stdout:3/769: mknod dc/d30/d3a/db7/cf5 0 2024-03-18T14:51:05.243 INFO:tasks.workunit.client.0.smithi012.stdout:7/813: rename dd/d13/d23/d33/d37/l4a to dd/d13/d23/d76/d6f/db3/dc7/l11c 0 2024-03-18T14:51:05.244 INFO:tasks.workunit.client.1.smithi100.stdout:7/458: dread d5/d14/d16/d2c/f51 [0,4194304] 0 2024-03-18T14:51:05.245 INFO:tasks.workunit.client.0.smithi012.stdout:9/829: creat d3/f110 x:0 0 0 2024-03-18T14:51:05.246 INFO:tasks.workunit.client.0.smithi012.stdout:1/737: mknod d8/db/d10/d16/d6b/dd1/cf2 0 2024-03-18T14:51:05.247 INFO:tasks.workunit.client.1.smithi100.stdout:8/474: dwrite d4/dc/d31/d66/f9b [0,4194304] 0 2024-03-18T14:51:05.248 INFO:tasks.workunit.client.1.smithi100.stdout:6/454: rmdir d6/d7/df 39 2024-03-18T14:51:05.249 INFO:tasks.workunit.client.1.smithi100.stdout:6/455: write f4 [7485665,51060] 0 2024-03-18T14:51:05.251 INFO:tasks.workunit.client.1.smithi100.stdout:5/397: unlink d29/d2d/c3b 0 2024-03-18T14:51:05.252 INFO:tasks.workunit.client.0.smithi012.stdout:7/814: mknod dd/d13/d23/d76/d6f/db3/c11d 0 2024-03-18T14:51:05.253 INFO:tasks.workunit.client.0.smithi012.stdout:9/830: rmdir d3/de6/dbf/d77/d52/d95 39 2024-03-18T14:51:05.255 INFO:tasks.workunit.client.0.smithi012.stdout:1/738: creat d8/db/d10/d31/d43/ff3 x:0 0 0 2024-03-18T14:51:05.256 INFO:tasks.workunit.client.1.smithi100.stdout:9/434: rename da/dd/c39 to da/d51/d55/d7e/c85 0 2024-03-18T14:51:05.256 INFO:tasks.workunit.client.1.smithi100.stdout:9/435: write da/d46/d4d/f56 [3842421,110921] 0 2024-03-18T14:51:05.257 INFO:tasks.workunit.client.1.smithi100.stdout:2/427: dwrite d3/da/f30 [0,4194304] 0 2024-03-18T14:51:05.261 INFO:tasks.workunit.client.0.smithi012.stdout:9/831: mkdir d3/de6/dbf/d66/d6e/dc6/d111 0 2024-03-18T14:51:05.263 INFO:tasks.workunit.client.1.smithi100.stdout:8/475: creat d4/dc/dd/de/d21/d23/d47/d54/fa5 x:0 0 0 2024-03-18T14:51:05.264 INFO:tasks.workunit.client.0.smithi012.stdout:7/815: getdents dd 0 2024-03-18T14:51:05.264 INFO:tasks.workunit.client.0.smithi012.stdout:7/816: write dd/d13/d23/d76/d6f/db3/dc7/fea [377586,41549] 0 2024-03-18T14:51:05.265 INFO:tasks.workunit.client.1.smithi100.stdout:6/456: mkdir d6/d7/df/d72/d8b 0 2024-03-18T14:51:05.266 INFO:tasks.workunit.client.1.smithi100.stdout:2/428: dread d3/d6/d29/d36/d49/f7a [0,4194304] 0 2024-03-18T14:51:05.266 INFO:tasks.workunit.client.0.smithi012.stdout:9/832: rename d3/de6/dbf/d77/cb0 to d3/de6/dbf/d66/d6e/dc6/d111/c112 0 2024-03-18T14:51:05.267 INFO:tasks.workunit.client.1.smithi100.stdout:5/398: creat d29/d2d/d32/f7c x:0 0 0 2024-03-18T14:51:05.267 INFO:tasks.workunit.client.1.smithi100.stdout:5/399: dread - d29/d2d/d39/d61/f6b zero size 2024-03-18T14:51:05.268 INFO:tasks.workunit.client.0.smithi012.stdout:7/817: link dd/d4f/f69 dd/d13/d23/d33/dd5/f11e 0 2024-03-18T14:51:05.269 INFO:tasks.workunit.client.1.smithi100.stdout:9/436: symlink da/d2c/d70/l86 0 2024-03-18T14:51:05.271 INFO:tasks.workunit.client.0.smithi012.stdout:9/833: mkdir d3/de6/dbf/d66/d69/d7f/d113 0 2024-03-18T14:51:05.271 INFO:tasks.workunit.client.0.smithi012.stdout:9/834: stat d3/de6/dbf/d77/da8/l109 0 2024-03-18T14:51:05.273 INFO:tasks.workunit.client.1.smithi100.stdout:2/429: mknod d3/d13/d40/d7d/c96 0 2024-03-18T14:51:05.275 INFO:tasks.workunit.client.0.smithi012.stdout:9/835: getdents d3/d9/d5c/dbd 0 2024-03-18T14:51:05.276 INFO:tasks.workunit.client.1.smithi100.stdout:9/437: dread da/dd/d29/f2e [0,4194304] 0 2024-03-18T14:51:05.276 INFO:tasks.workunit.client.1.smithi100.stdout:2/430: symlink d3/d6/d5f/d6a/l97 0 2024-03-18T14:51:05.276 INFO:tasks.workunit.client.1.smithi100.stdout:2/431: chown d3/d6/d29/d36/c57 55577091 1 2024-03-18T14:51:05.277 INFO:tasks.workunit.client.1.smithi100.stdout:9/438: creat da/dd/d29/f87 x:0 0 0 2024-03-18T14:51:05.281 INFO:tasks.workunit.client.0.smithi012.stdout:0/815: dwrite d1/d16/d6e/fef [0,4194304] 0 2024-03-18T14:51:05.282 INFO:tasks.workunit.client.1.smithi100.stdout:6/457: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f13 [0,4194304] 0 2024-03-18T14:51:05.282 INFO:tasks.workunit.client.1.smithi100.stdout:4/448: dwrite d9/d13/d25/d51/f5d [0,4194304] 0 2024-03-18T14:51:05.284 INFO:tasks.workunit.client.1.smithi100.stdout:8/476: write d4/dc/dd/de/d21/d23/f28 [60364,120966] 0 2024-03-18T14:51:05.284 INFO:tasks.workunit.client.1.smithi100.stdout:8/477: chown d4/dc/dd/de/d21/d23/c68 51820370 1 2024-03-18T14:51:05.287 INFO:tasks.workunit.client.1.smithi100.stdout:6/458: creat d6/d7/df/d72/d8b/f8c x:0 0 0 2024-03-18T14:51:05.294 INFO:tasks.workunit.client.1.smithi100.stdout:3/506: dwrite d3/d16/d13/f39 [0,4194304] 0 2024-03-18T14:51:05.299 INFO:tasks.workunit.client.1.smithi100.stdout:8/478: write d4/dc/dd/de/d21/d23/d47/f50 [2259449,88159] 0 2024-03-18T14:51:05.302 INFO:tasks.workunit.client.1.smithi100.stdout:6/459: dread d6/d7/df/d72/f33 [0,4194304] 0 2024-03-18T14:51:05.302 INFO:tasks.workunit.client.1.smithi100.stdout:6/460: chown d6/d7/df/d72/f33 19 1 2024-03-18T14:51:05.303 INFO:tasks.workunit.client.1.smithi100.stdout:4/449: creat d9/d27/d42/f84 x:0 0 0 2024-03-18T14:51:05.305 INFO:tasks.workunit.client.1.smithi100.stdout:3/507: symlink d3/d16/d13/d71/d74/lb4 0 2024-03-18T14:51:05.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/461: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f13 [0,4194304] 0 2024-03-18T14:51:05.306 INFO:tasks.workunit.client.0.smithi012.stdout:8/948: dwrite d0/d47/ff2 [0,4194304] 0 2024-03-18T14:51:05.306 INFO:tasks.workunit.client.0.smithi012.stdout:8/949: write d0/f10 [4748585,82583] 0 2024-03-18T14:51:05.307 INFO:tasks.workunit.client.1.smithi100.stdout:8/479: creat d4/dc/dd/d48/fa6 x:0 0 0 2024-03-18T14:51:05.307 INFO:tasks.workunit.client.1.smithi100.stdout:8/480: readlink d4/dc/dd/de/d19/l7d 0 2024-03-18T14:51:05.309 INFO:tasks.workunit.client.1.smithi100.stdout:4/450: symlink d9/d13/d39/d4c/l85 0 2024-03-18T14:51:05.313 INFO:tasks.workunit.client.1.smithi100.stdout:3/508: rename d3/dd/d21/f55 to d3/d16/d13/d71/d74/fb5 0 2024-03-18T14:51:05.315 INFO:tasks.workunit.client.1.smithi100.stdout:4/451: creat d9/d3a/f86 x:0 0 0 2024-03-18T14:51:05.317 INFO:tasks.workunit.client.1.smithi100.stdout:8/481: rename d4/dc/dd/de/d19/c30 to d4/dc/dd/de/d21/ca7 0 2024-03-18T14:51:05.320 INFO:tasks.workunit.client.1.smithi100.stdout:4/452: getdents d9/d13/d5a 0 2024-03-18T14:51:05.322 INFO:tasks.workunit.client.1.smithi100.stdout:3/509: rename d3/dd/d21/fa7 to d3/fb6 0 2024-03-18T14:51:05.323 INFO:tasks.workunit.client.1.smithi100.stdout:4/453: mknod d9/d13/d5a/c87 0 2024-03-18T14:51:05.325 INFO:tasks.workunit.client.1.smithi100.stdout:4/454: rename d9/f6d to d9/f88 0 2024-03-18T14:51:05.327 INFO:tasks.workunit.client.1.smithi100.stdout:4/455: unlink d9/d17/f1d 0 2024-03-18T14:51:05.327 INFO:tasks.workunit.client.1.smithi100.stdout:4/456: stat d9/d13/d39/d4c 0 2024-03-18T14:51:05.328 INFO:tasks.workunit.client.1.smithi100.stdout:3/510: dread d3/d16/d13/d1f/d37/f54 [0,4194304] 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:5/400: dwrite d29/d2d/d32/d38/f5d [0,4194304] 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:5/401: fsync d29/d2d/d32/f5c 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:2/432: dwrite d3/d13/d40/d53/f85 [0,4194304] 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:2/433: creat d3/d6/f98 x:0 0 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:2/434: read d3/f4 [3470527,24350] 0 2024-03-18T14:51:05.339 INFO:tasks.workunit.client.1.smithi100.stdout:7/459: dwrite d5/d14/d16/d2e/d62/f97 [0,4194304] 0 2024-03-18T14:51:05.340 INFO:tasks.workunit.client.0.smithi012.stdout:1/739: dwrite d8/f1a [0,4194304] 0 2024-03-18T14:51:05.340 INFO:tasks.workunit.client.0.smithi012.stdout:1/740: chown d8/db/d10/d31/d43/d5e/d65/d84/dc0 0 1 2024-03-18T14:51:05.340 INFO:tasks.workunit.client.0.smithi012.stdout:6/718: dwrite d17/d19/d27/f68 [0,4194304] 0 2024-03-18T14:51:05.340 INFO:tasks.workunit.client.0.smithi012.stdout:6/719: fdatasync f11 0 2024-03-18T14:51:05.343 INFO:tasks.workunit.client.1.smithi100.stdout:7/460: dread d5/f72 [0,4194304] 0 2024-03-18T14:51:05.343 INFO:tasks.workunit.client.1.smithi100.stdout:7/461: chown d5/d14/c27 0 1 2024-03-18T14:51:05.343 INFO:tasks.workunit.client.1.smithi100.stdout:7/462: dread - d5/d14/f77 zero size 2024-03-18T14:51:05.343 INFO:tasks.workunit.client.1.smithi100.stdout:7/463: write d5/d22/d24/d6e/f84 [405728,50959] 0 2024-03-18T14:51:05.350 INFO:tasks.workunit.client.1.smithi100.stdout:9/439: dwrite da/d2c/d2f/f31 [0,4194304] 0 2024-03-18T14:51:05.351 INFO:tasks.workunit.client.1.smithi100.stdout:8/482: dwrite d4/dc/dd/de/f63 [0,4194304] 0 2024-03-18T14:51:05.351 INFO:tasks.workunit.client.0.smithi012.stdout:9/836: dwrite d3/d9/fd [0,4194304] 0 2024-03-18T14:51:05.351 INFO:tasks.workunit.client.0.smithi012.stdout:0/816: dwrite d1/d20/d23/d82/fb8 [0,4194304] 0 2024-03-18T14:51:05.351 INFO:tasks.workunit.client.1.smithi100.stdout:6/462: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [0,4194304] 0 2024-03-18T14:51:05.352 INFO:tasks.workunit.client.1.smithi100.stdout:3/511: dwrite d3/fb6 [0,4194304] 0 2024-03-18T14:51:05.356 INFO:tasks.workunit.client.1.smithi100.stdout:6/463: dread d6/d7/df/d72/f33 [0,4194304] 0 2024-03-18T14:51:05.356 INFO:tasks.workunit.client.1.smithi100.stdout:6/464: fdatasync d6/d7/f8 0 2024-03-18T14:51:05.370 INFO:tasks.workunit.client.1.smithi100.stdout:6/465: dread d6/d7/df/d26/d4e/d66/f2d [0,4194304] 0 2024-03-18T14:51:05.370 INFO:tasks.workunit.client.1.smithi100.stdout:8/483: dwrite d4/dc/dd/d48/fa6 [0,4194304] 0 2024-03-18T14:51:05.379 INFO:tasks.workunit.client.1.smithi100.stdout:8/484: dread d4/dc/f8a [0,4194304] 0 2024-03-18T14:51:05.384 INFO:tasks.workunit.client.0.smithi012.stdout:1/741: rename d8/f40 to d8/db/d10/d16/d6b/dc3/ff4 0 2024-03-18T14:51:05.384 INFO:tasks.workunit.client.0.smithi012.stdout:1/742: stat d8/db/d10/d31/d43/d5e/d65/d84/fde 0 2024-03-18T14:51:05.386 INFO:tasks.workunit.client.1.smithi100.stdout:6/466: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/f22 [0,4194304] 0 2024-03-18T14:51:05.386 INFO:tasks.workunit.client.1.smithi100.stdout:6/467: creat d6/d7/f8d x:0 0 0 2024-03-18T14:51:05.393 INFO:tasks.workunit.client.1.smithi100.stdout:5/402: mkdir d29/d2d/d39/d65/d7d 0 2024-03-18T14:51:05.398 INFO:tasks.workunit.client.1.smithi100.stdout:2/435: creat d3/d6/d5f/d6a/f99 x:0 0 0 2024-03-18T14:51:05.398 INFO:tasks.workunit.client.1.smithi100.stdout:2/436: chown d3/d13/d4f/c55 16126 1 2024-03-18T14:51:05.404 INFO:tasks.workunit.client.0.smithi012.stdout:9/837: unlink d3/d9/fd5 0 2024-03-18T14:51:05.405 INFO:tasks.workunit.client.0.smithi012.stdout:9/838: dread - d3/de6/dbf/d66/d21/f90 zero size 2024-03-18T14:51:05.407 INFO:tasks.workunit.client.0.smithi012.stdout:0/817: link d1/f9 d1/d20/d23/d3b/dc1/f10e 0 2024-03-18T14:51:05.407 INFO:tasks.workunit.client.0.smithi012.stdout:0/818: readlink d1/d20/d7b/lf7 0 2024-03-18T14:51:05.407 INFO:tasks.workunit.client.0.smithi012.stdout:0/819: chown d1/d20/d23/d6f/d93/dba/lfc 3 1 2024-03-18T14:51:05.410 INFO:tasks.workunit.client.1.smithi100.stdout:9/440: truncate da/db/f13 4180657 0 2024-03-18T14:51:05.410 INFO:tasks.workunit.client.1.smithi100.stdout:9/441: stat da/d46/d4d 0 2024-03-18T14:51:05.412 INFO:tasks.workunit.client.0.smithi012.stdout:6/720: getdents d17/d33 0 2024-03-18T14:51:05.412 INFO:tasks.workunit.client.0.smithi012.stdout:6/721: write d17/d19/f6d [961874,48629] 0 2024-03-18T14:51:05.415 INFO:tasks.workunit.client.0.smithi012.stdout:9/839: symlink d3/de6/dbf/d77/d52/d99/da0/ddf/dee/l114 0 2024-03-18T14:51:05.417 INFO:tasks.workunit.client.0.smithi012.stdout:0/820: link d1/d16/d79/ff5 d1/d16/d6e/dcc/f10f 0 2024-03-18T14:51:05.421 INFO:tasks.workunit.client.0.smithi012.stdout:6/722: link d17/d19/c25 d17/d19/d27/db1/cd5 0 2024-03-18T14:51:05.424 INFO:tasks.workunit.client.0.smithi012.stdout:0/821: unlink d1/d16/d19/dd0/ff8 0 2024-03-18T14:51:05.426 INFO:tasks.workunit.client.0.smithi012.stdout:6/723: getdents d17/d38/d51/dcc/dd0 0 2024-03-18T14:51:05.426 INFO:tasks.workunit.client.0.smithi012.stdout:6/724: chown d17/d19/d27/d6e/d84/lc2 14781150 1 2024-03-18T14:51:05.426 INFO:tasks.workunit.client.0.smithi012.stdout:6/725: readlink d17/la6 0 2024-03-18T14:51:05.426 INFO:tasks.workunit.client.0.smithi012.stdout:6/726: dread - d17/d19/d27/d58/d91/fca zero size 2024-03-18T14:51:05.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/727: rmdir d17 39 2024-03-18T14:51:05.431 INFO:tasks.workunit.client.1.smithi100.stdout:8/485: unlink d4/dc/dd/de/d19/f1e 0 2024-03-18T14:51:05.431 INFO:tasks.workunit.client.1.smithi100.stdout:8/486: fdatasync d4/dc/dd/de/d21/d23/d47/f86 0 2024-03-18T14:51:05.434 INFO:tasks.workunit.client.0.smithi012.stdout:6/728: rename fb to d17/d19/d27/fd6 0 2024-03-18T14:51:05.434 INFO:tasks.workunit.client.0.smithi012.stdout:6/729: fsync d17/d38/d51/d66/faf 0 2024-03-18T14:51:05.434 INFO:tasks.workunit.client.0.smithi012.stdout:0/822: dread d1/d16/d19/d8d/fab [0,4194304] 0 2024-03-18T14:51:05.434 INFO:tasks.workunit.client.1.smithi100.stdout:8/487: dread d4/dc/d1f/f32 [0,4194304] 0 2024-03-18T14:51:05.435 INFO:tasks.workunit.client.0.smithi012.stdout:9/840: dwrite d3/de6/dbf/f4d [0,4194304] 0 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.1.smithi100.stdout:6/468: mknod d6/d7/df/d26/d4e/d66/d36/d4f/d71/d7f/c8e 0 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.1.smithi100.stdout:6/469: truncate d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f88 546015 0 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.1.smithi100.stdout:6/470: creat d6/f8f x:0 0 0 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.1.smithi100.stdout:6/471: chown d6/d12/f89 1307086 1 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.1.smithi100.stdout:6/472: chown d6/d12/d2c/f7e 251652 1 2024-03-18T14:51:05.436 INFO:tasks.workunit.client.0.smithi012.stdout:0/823: readlink d1/d20/d7b/lb2 0 2024-03-18T14:51:05.438 INFO:tasks.workunit.client.1.smithi100.stdout:5/403: creat d29/f7e x:0 0 0 2024-03-18T14:51:05.438 INFO:tasks.workunit.client.1.smithi100.stdout:5/404: dread - d29/d2d/d39/d61/f69 zero size 2024-03-18T14:51:05.439 INFO:tasks.workunit.client.1.smithi100.stdout:1/474: sync 2024-03-18T14:51:05.440 INFO:tasks.workunit.client.1.smithi100.stdout:2/437: mkdir d3/d9a 0 2024-03-18T14:51:05.445 INFO:tasks.workunit.client.1.smithi100.stdout:9/442: mkdir da/d88 0 2024-03-18T14:51:05.445 INFO:tasks.workunit.client.1.smithi100.stdout:9/443: fsync da/f16 0 2024-03-18T14:51:05.446 INFO:tasks.workunit.client.1.smithi100.stdout:9/444: dread - da/dd/f42 zero size 2024-03-18T14:51:05.446 INFO:tasks.workunit.client.1.smithi100.stdout:9/445: write da/dd/f60 [870857,20733] 0 2024-03-18T14:51:05.446 INFO:tasks.workunit.client.1.smithi100.stdout:9/446: creat da/d2c/f89 x:0 0 0 2024-03-18T14:51:05.446 INFO:tasks.workunit.client.1.smithi100.stdout:9/447: write da/d2c/d2f/f5b [686762,105702] 0 2024-03-18T14:51:05.448 INFO:tasks.workunit.client.0.smithi012.stdout:9/841: creat d3/de6/dc1/f115 x:0 0 0 2024-03-18T14:51:05.449 INFO:tasks.workunit.client.1.smithi100.stdout:5/405: dread f17 [4194304,4194304] 0 2024-03-18T14:51:05.449 INFO:tasks.workunit.client.1.smithi100.stdout:5/406: truncate d29/d2d/f6f 1929727 0 2024-03-18T14:51:05.450 INFO:tasks.workunit.client.0.smithi012.stdout:6/730: read d17/d26/f28 [343732,70597] 0 2024-03-18T14:51:05.450 INFO:tasks.workunit.client.0.smithi012.stdout:6/731: fdatasync d17/f75 0 2024-03-18T14:51:05.452 INFO:tasks.workunit.client.1.smithi100.stdout:6/473: mknod d6/d7/df/d26/d4e/d66/d36/d4f/c90 0 2024-03-18T14:51:05.452 INFO:tasks.workunit.client.1.smithi100.stdout:6/474: write d6/d7/df/d72/f33 [4915043,106723] 0 2024-03-18T14:51:05.453 INFO:tasks.workunit.client.1.smithi100.stdout:1/475: unlink d8/dd/c1c 0 2024-03-18T14:51:05.455 INFO:tasks.workunit.client.0.smithi012.stdout:6/732: truncate d17/d26/f96 1459521 0 2024-03-18T14:51:05.457 INFO:tasks.workunit.client.1.smithi100.stdout:2/438: unlink d3/da/de/d48/f5d 0 2024-03-18T14:51:05.457 INFO:tasks.workunit.client.0.smithi012.stdout:9/842: dread d3/de6/dbf/d66/d69/d7f/d8c/d8b/f9f [0,4194304] 0 2024-03-18T14:51:05.462 INFO:tasks.workunit.client.1.smithi100.stdout:8/488: dwrite d4/dc/dd/d48/fa6 [0,4194304] 0 2024-03-18T14:51:05.463 INFO:tasks.workunit.client.1.smithi100.stdout:8/489: chown d4/d3c/c5e 0 1 2024-03-18T14:51:05.463 INFO:tasks.workunit.client.1.smithi100.stdout:8/490: fsync d4/dc/d31/f46 0 2024-03-18T14:51:05.463 INFO:tasks.workunit.client.0.smithi012.stdout:9/843: getdents d3/de6/dbf/d77/d52/d95/deb 0 2024-03-18T14:51:05.463 INFO:tasks.workunit.client.0.smithi012.stdout:0/824: dwrite d1/d20/d23/d6f/f75 [0,4194304] 0 2024-03-18T14:51:05.466 INFO:tasks.workunit.client.1.smithi100.stdout:9/448: rename l8 to da/dd/d29/l8a 0 2024-03-18T14:51:05.466 INFO:tasks.workunit.client.0.smithi012.stdout:9/844: stat d3/de6/dbf/d77/d52/d78/lc7 0 2024-03-18T14:51:05.466 INFO:tasks.workunit.client.0.smithi012.stdout:9/845: creat d3/de6/dbf/d77/d52/d99/da0/ddf/dee/f116 x:0 0 0 2024-03-18T14:51:05.467 INFO:tasks.workunit.client.0.smithi012.stdout:6/733: write d17/d19/d27/f68 [1792562,15395] 0 2024-03-18T14:51:05.467 INFO:tasks.workunit.client.1.smithi100.stdout:6/475: write d6/d7/df/f7c [150536,97477] 0 2024-03-18T14:51:05.467 INFO:tasks.workunit.client.1.smithi100.stdout:6/476: creat d6/f91 x:0 0 0 2024-03-18T14:51:05.468 INFO:tasks.workunit.client.1.smithi100.stdout:6/477: readlink d6/d7/df/d26/l81 0 2024-03-18T14:51:05.468 INFO:tasks.workunit.client.1.smithi100.stdout:6/478: chown d6/d7/df/d26/d4e/d66/d36/d4f/d71/f22 26 1 2024-03-18T14:51:05.469 INFO:tasks.workunit.client.1.smithi100.stdout:1/476: mkdir d8/d1e/d11/d27/d73/d89 0 2024-03-18T14:51:05.471 INFO:tasks.workunit.client.1.smithi100.stdout:2/439: symlink d3/d6/d29/d36/l9b 0 2024-03-18T14:51:05.471 INFO:tasks.workunit.client.1.smithi100.stdout:2/440: write d3/da/f25 [5655852,116432] 0 2024-03-18T14:51:05.473 INFO:tasks.workunit.client.0.smithi012.stdout:9/846: rename d3/de6/dbf/d77/d52/d95/fcf to d3/de6/dbf/d66/d69/d7f/d113/f117 0 2024-03-18T14:51:05.473 INFO:tasks.workunit.client.0.smithi012.stdout:9/847: readlink d3/de6/dbf/d77/d52/d99/da0/ddf/dee/l114 0 2024-03-18T14:51:05.473 INFO:tasks.workunit.client.0.smithi012.stdout:9/848: chown d3/d9/fa2 831160242 1 2024-03-18T14:51:05.474 INFO:tasks.workunit.client.1.smithi100.stdout:2/441: dread d3/d6/f9 [0,4194304] 0 2024-03-18T14:51:05.474 INFO:tasks.workunit.client.1.smithi100.stdout:2/442: creat d3/da/de/d20/f9c x:0 0 0 2024-03-18T14:51:05.477 INFO:tasks.workunit.client.0.smithi012.stdout:6/734: mkdir d17/d19/d27/db1/dd7 0 2024-03-18T14:51:05.479 INFO:tasks.workunit.client.0.smithi012.stdout:6/735: dread d17/d18/f45 [0,4194304] 0 2024-03-18T14:51:05.482 INFO:tasks.workunit.client.0.smithi012.stdout:9/849: dread d3/de6/dbf/d66/d69/d7f/d8c/fcd [0,4194304] 0 2024-03-18T14:51:05.482 INFO:tasks.workunit.client.0.smithi012.stdout:9/850: fsync d3/f3b 0 2024-03-18T14:51:05.484 INFO:tasks.workunit.client.1.smithi100.stdout:9/449: dwrite da/dd/d29/d71/f82 [0,4194304] 0 2024-03-18T14:51:05.485 INFO:tasks.workunit.client.0.smithi012.stdout:0/825: dwrite d1/d20/d7b/f30 [0,4194304] 0 2024-03-18T14:51:05.485 INFO:tasks.workunit.client.0.smithi012.stdout:5/781: sync 2024-03-18T14:51:05.486 INFO:tasks.workunit.client.1.smithi100.stdout:8/491: creat d4/fa8 x:0 0 0 2024-03-18T14:51:05.488 INFO:tasks.workunit.client.0.smithi012.stdout:9/851: creat d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/ddc/f118 x:0 0 0 2024-03-18T14:51:05.488 INFO:tasks.workunit.client.0.smithi012.stdout:9/852: readlink d3/de6/dbf/d77/d52/d99/da0/ddf/dee/l114 0 2024-03-18T14:51:05.490 INFO:tasks.workunit.client.1.smithi100.stdout:5/407: rename d29/d2d/c33 to d29/d4a/c7f 0 2024-03-18T14:51:05.494 INFO:tasks.workunit.client.0.smithi012.stdout:6/736: dwrite d17/f2c [0,4194304] 0 2024-03-18T14:51:05.494 INFO:tasks.workunit.client.0.smithi012.stdout:6/737: getdents d17/d38/d51/da8 0 2024-03-18T14:51:05.494 INFO:tasks.workunit.client.0.smithi012.stdout:0/826: symlink d1/d20/d4e/d54/l110 0 2024-03-18T14:51:05.495 INFO:tasks.workunit.client.1.smithi100.stdout:5/408: dread f17 [4194304,4194304] 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/450: write da/f16 [1580641,61883] 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/451: readlink da/d51/d55/l7c 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/452: chown da/dd/d3b/l5e 16 1 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/453: fsync da/db/f13 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/454: stat da/d25 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:9/455: fsync da/db/f4c 0 2024-03-18T14:51:05.496 INFO:tasks.workunit.client.1.smithi100.stdout:6/479: symlink d6/d7/df/d26/d4e/d66/d36/d4f/d8a/l92 0 2024-03-18T14:51:05.497 INFO:tasks.workunit.client.0.smithi012.stdout:5/782: mkdir d3/dad/df2/d102 0 2024-03-18T14:51:05.499 INFO:tasks.workunit.client.1.smithi100.stdout:1/477: unlink d8/dd/f56 0 2024-03-18T14:51:05.500 INFO:tasks.workunit.client.0.smithi012.stdout:9/853: symlink d3/de6/dbf/d77/d52/l119 0 2024-03-18T14:51:05.506 INFO:tasks.workunit.client.1.smithi100.stdout:1/478: dread d8/d2a/f6b [0,4194304] 0 2024-03-18T14:51:05.506 INFO:tasks.workunit.client.1.smithi100.stdout:8/492: mknod d4/dc/dd/de/d19/ca9 0 2024-03-18T14:51:05.507 INFO:tasks.workunit.client.0.smithi012.stdout:6/738: readlink l3 0 2024-03-18T14:51:05.509 INFO:tasks.workunit.client.1.smithi100.stdout:2/443: rename d3/d13/d4f/c94 to d3/d6/d5f/d63/c9d 0 2024-03-18T14:51:05.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/444: dread - d3/d6/d29/d36/d49/f51 zero size 2024-03-18T14:51:05.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/445: getdents d3/d8c 0 2024-03-18T14:51:05.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/446: creat d3/d6/d29/d36/f9e x:0 0 0 2024-03-18T14:51:05.511 INFO:tasks.workunit.client.0.smithi012.stdout:2/833: sync 2024-03-18T14:51:05.511 INFO:tasks.workunit.client.0.smithi012.stdout:5/783: creat d3/de0/f103 x:0 0 0 2024-03-18T14:51:05.511 INFO:tasks.workunit.client.0.smithi012.stdout:5/784: write d3/d15/d42/dc6/d65/fa1 [485641,99779] 0 2024-03-18T14:51:05.513 INFO:tasks.workunit.client.1.smithi100.stdout:7/464: write d5/fd [694927,108495] 0 2024-03-18T14:51:05.513 INFO:tasks.workunit.client.1.smithi100.stdout:7/465: write d5/d22/d24/f79 [882141,44014] 0 2024-03-18T14:51:05.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/409: creat d29/d4a/f80 x:0 0 0 2024-03-18T14:51:05.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/410: chown d29/d2d/d39/f4e 5725 1 2024-03-18T14:51:05.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/411: write f25 [835766,68600] 0 2024-03-18T14:51:05.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/412: write d29/d2d/d39/d61/f69 [950730,77474] 0 2024-03-18T14:51:05.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/413: write f23 [4753619,89448] 0 2024-03-18T14:51:05.515 INFO:tasks.workunit.client.0.smithi012.stdout:0/827: dwrite d1/d16/d19/fb3 [0,4194304] 0 2024-03-18T14:51:05.515 INFO:tasks.workunit.client.0.smithi012.stdout:9/854: creat d3/de6/dbf/d77/d52/d95/deb/f11a x:0 0 0 2024-03-18T14:51:05.515 INFO:tasks.workunit.client.0.smithi012.stdout:9/855: fsync d3/de6/dbf/f6a 0 2024-03-18T14:51:05.516 INFO:tasks.workunit.client.1.smithi100.stdout:0/555: sync 2024-03-18T14:51:05.518 INFO:tasks.workunit.client.0.smithi012.stdout:5/785: rename d3/d15/d42/dc6/d73/d6b/df3/d8b/fdb to d3/d29/d36/d9b/ddf/f104 0 2024-03-18T14:51:05.520 INFO:tasks.workunit.client.0.smithi012.stdout:2/834: read d0/d6b/dbf/d33/d71/fc1 [2356960,121768] 0 2024-03-18T14:51:05.521 INFO:tasks.workunit.client.1.smithi100.stdout:6/480: mknod d6/d7/c93 0 2024-03-18T14:51:05.524 INFO:tasks.workunit.client.0.smithi012.stdout:2/835: write d0/d6b/dbf/d33/d77/df5/d103/fd7 [363188,3952] 0 2024-03-18T14:51:05.526 INFO:tasks.workunit.client.1.smithi100.stdout:8/493: truncate d4/fa 648882 0 2024-03-18T14:51:05.526 INFO:tasks.workunit.client.1.smithi100.stdout:8/494: write d4/dc/dd/de/d21/d23/f27 [7829786,61433] 0 2024-03-18T14:51:05.529 INFO:tasks.workunit.client.0.smithi012.stdout:0/828: link d1/d16/d31/f1d d1/d20/d4e/d54/d8a/f111 0 2024-03-18T14:51:05.535 INFO:tasks.workunit.client.0.smithi012.stdout:2/836: mkdir d0/d6b/dbf/d33/d71/de5/d108 0 2024-03-18T14:51:05.536 INFO:tasks.workunit.client.0.smithi012.stdout:2/837: dread - d0/d6b/d9f/fd4 zero size 2024-03-18T14:51:05.536 INFO:tasks.workunit.client.1.smithi100.stdout:7/466: mkdir d5/d22/d24/dac 0 2024-03-18T14:51:05.536 INFO:tasks.workunit.client.1.smithi100.stdout:7/467: chown d5/lc 59 1 2024-03-18T14:51:05.539 INFO:tasks.workunit.client.0.smithi012.stdout:2/838: rename d0/d3c/d40/d47/lb8 to d0/d6b/dbf/d33/d77/da0/l109 0 2024-03-18T14:51:05.539 INFO:tasks.workunit.client.0.smithi012.stdout:2/839: write d0/d6b/dbf/d33/d77/f86 [184234,109843] 0 2024-03-18T14:51:05.540 INFO:tasks.workunit.client.1.smithi100.stdout:5/414: symlink d29/l81 0 2024-03-18T14:51:05.541 INFO:tasks.workunit.client.1.smithi100.stdout:6/481: mknod d6/d7/df/d72/d8b/c94 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:0/829: dread d1/d16/d19/fb3 [0,4194304] 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:0/830: truncate d1/d20/d23/d82/fc0 1087283 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:2/840: mkdir d0/d6b/dbf/d33/d77/d10a 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:6/739: dwrite d17/d19/f3c [0,4194304] 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:5/786: dwrite d3/d1e/d33/f74 [0,4194304] 0 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:5/787: chown d3/db6/fca 69 1 2024-03-18T14:51:05.543 INFO:tasks.workunit.client.0.smithi012.stdout:5/788: creat d3/d15/d42/f105 x:0 0 0 2024-03-18T14:51:05.544 INFO:tasks.workunit.client.1.smithi100.stdout:8/495: mknod d4/dc/d97/caa 0 2024-03-18T14:51:05.544 INFO:tasks.workunit.client.1.smithi100.stdout:8/496: chown d4/l6e 82686 1 2024-03-18T14:51:05.544 INFO:tasks.workunit.client.0.smithi012.stdout:9/856: dwrite d3/de6/dbf/d66/d69/fb2 [0,4194304] 0 2024-03-18T14:51:05.544 INFO:tasks.workunit.client.0.smithi012.stdout:9/857: readlink d3/de6/dbf/d77/da8/le2 0 2024-03-18T14:51:05.549 INFO:tasks.workunit.client.0.smithi012.stdout:5/789: dread d3/d15/d5f/fd2 [0,4194304] 0 2024-03-18T14:51:05.550 INFO:tasks.workunit.client.0.smithi012.stdout:2/841: creat d0/d6b/dbf/d33/d80/de1/f10b x:0 0 0 2024-03-18T14:51:05.550 INFO:tasks.workunit.client.0.smithi012.stdout:2/842: write d0/d6b/dbf/d33/dc9/fcd [1853669,927] 0 2024-03-18T14:51:05.556 INFO:tasks.workunit.client.1.smithi100.stdout:7/468: mknod d5/d14/d16/d44/cad 0 2024-03-18T14:51:05.558 INFO:tasks.workunit.client.0.smithi012.stdout:9/858: dwrite d3/de6/dbf/d66/d69/d7f/fe0 [0,4194304] 0 2024-03-18T14:51:05.558 INFO:tasks.workunit.client.0.smithi012.stdout:9/859: write d3/de6/dbf/d66/d69/d7f/d8c/dac/fec [1554399,106711] 0 2024-03-18T14:51:05.558 INFO:tasks.workunit.client.0.smithi012.stdout:9/860: chown d3/d9/f17 739545 1 2024-03-18T14:51:05.559 INFO:tasks.workunit.client.0.smithi012.stdout:9/861: fdatasync d3/f3b 0 2024-03-18T14:51:05.560 INFO:tasks.workunit.client.1.smithi100.stdout:5/415: mknod d29/d2d/d39/d71/c82 0 2024-03-18T14:51:05.561 INFO:tasks.workunit.client.1.smithi100.stdout:2/447: rmdir d3/da/de/d20 39 2024-03-18T14:51:05.563 INFO:tasks.workunit.client.1.smithi100.stdout:8/497: link d4/dc/d31/c6a d4/dc/dd/de/d19/d2c/da2/cab 0 2024-03-18T14:51:05.563 INFO:tasks.workunit.client.1.smithi100.stdout:8/498: chown d4/dc/dd/de/d19/d37/l73 407123913 1 2024-03-18T14:51:05.565 INFO:tasks.workunit.client.0.smithi012.stdout:4/718: sync 2024-03-18T14:51:05.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/770: sync 2024-03-18T14:51:05.566 INFO:tasks.workunit.client.1.smithi100.stdout:9/456: rename da/d25 to da/d46/d8b 0 2024-03-18T14:51:05.566 INFO:tasks.workunit.client.1.smithi100.stdout:9/457: write da/d46/d4d/d67/f75 [111356,8214] 0 2024-03-18T14:51:05.568 INFO:tasks.workunit.client.1.smithi100.stdout:8/499: dread d4/dc/dd/de/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:05.568 INFO:tasks.workunit.client.1.smithi100.stdout:7/469: link d5/d22/d3b/c55 d5/d22/d24/d6e/cae 0 2024-03-18T14:51:05.570 INFO:tasks.workunit.client.1.smithi100.stdout:2/448: dread d3/d13/d40/d53/f85 [0,4194304] 0 2024-03-18T14:51:05.570 INFO:tasks.workunit.client.1.smithi100.stdout:5/416: mkdir d29/d2d/d39/d83 0 2024-03-18T14:51:05.574 INFO:tasks.workunit.client.1.smithi100.stdout:4/457: sync 2024-03-18T14:51:05.574 INFO:tasks.workunit.client.1.smithi100.stdout:3/512: sync 2024-03-18T14:51:05.574 INFO:tasks.workunit.client.1.smithi100.stdout:3/513: creat d3/dd/d49/fb7 x:0 0 0 2024-03-18T14:51:05.574 INFO:tasks.workunit.client.1.smithi100.stdout:3/514: dread - d3/dd/d49/f87 zero size 2024-03-18T14:51:05.576 INFO:tasks.workunit.client.1.smithi100.stdout:6/482: symlink d6/d7/df/l95 0 2024-03-18T14:51:05.576 INFO:tasks.workunit.client.1.smithi100.stdout:6/483: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f88 [655334,9463] 0 2024-03-18T14:51:05.576 INFO:tasks.workunit.client.1.smithi100.stdout:6/484: write d6/d7/df/d26/d4e/d66/d36/f54 [117783,94030] 0 2024-03-18T14:51:05.578 INFO:tasks.workunit.client.0.smithi012.stdout:6/740: chown d17/d38/c73 812083489 1 2024-03-18T14:51:05.581 INFO:tasks.workunit.client.1.smithi100.stdout:1/479: rename d8/d82 to d8/d1e/d11/d8a 0 2024-03-18T14:51:05.586 INFO:tasks.workunit.client.1.smithi100.stdout:9/458: link da/dd/d29/d71/d72/f78 da/d88/f8c 0 2024-03-18T14:51:05.586 INFO:tasks.workunit.client.1.smithi100.stdout:9/459: chown da/dd/c2a 288 1 2024-03-18T14:51:05.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/500: link d4/dc/dd/de/d21/d23/d47/fa4 d4/dc/d1f/d59/fac 0 2024-03-18T14:51:05.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/501: readlink d4/dc/dd/l91 0 2024-03-18T14:51:05.590 INFO:tasks.workunit.client.1.smithi100.stdout:8/502: creat d4/dc/dd/de/d21/d23/d47/d54/fad x:0 0 0 2024-03-18T14:51:05.594 INFO:tasks.workunit.client.1.smithi100.stdout:4/458: symlink d9/d13/d39/d4c/l89 0 2024-03-18T14:51:05.598 INFO:tasks.workunit.client.0.smithi012.stdout:0/831: rmdir d1/d20/d23/d6f 39 2024-03-18T14:51:05.598 INFO:tasks.workunit.client.0.smithi012.stdout:0/832: truncate d1/d20/d23/d3b/ff6 26802 0 2024-03-18T14:51:05.602 INFO:tasks.workunit.client.1.smithi100.stdout:5/417: dwrite d29/d2d/d39/d60/f6d [0,4194304] 0 2024-03-18T14:51:05.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/485: dwrite d6/d12/f5c [0,4194304] 0 2024-03-18T14:51:05.602 INFO:tasks.workunit.client.1.smithi100.stdout:3/515: unlink d3/dd/f11 0 2024-03-18T14:51:05.602 INFO:tasks.workunit.client.1.smithi100.stdout:3/516: write d3/dd/f24 [1298743,113734] 0 2024-03-18T14:51:05.604 INFO:tasks.workunit.client.0.smithi012.stdout:5/790: rename d3/d15/d42/dc6/dfe/f100 to d3/d15/f106 0 2024-03-18T14:51:05.607 INFO:tasks.workunit.client.0.smithi012.stdout:2/843: symlink d0/d6b/dbf/d4f/l10c 0 2024-03-18T14:51:05.609 INFO:tasks.workunit.client.0.smithi012.stdout:5/791: dread d3/d1e/f1a [0,4194304] 0 2024-03-18T14:51:05.610 INFO:tasks.workunit.client.1.smithi100.stdout:5/418: dread d29/d2d/d39/d60/f6d [0,4194304] 0 2024-03-18T14:51:05.610 INFO:tasks.workunit.client.1.smithi100.stdout:5/419: stat d29/d2d/d39/d65 0 2024-03-18T14:51:05.610 INFO:tasks.workunit.client.1.smithi100.stdout:5/420: readlink d29/d2d/d39/d61/l67 0 2024-03-18T14:51:05.613 INFO:tasks.workunit.client.1.smithi100.stdout:3/517: write d3/d16/d13/d1f/f78 [1441535,91222] 0 2024-03-18T14:51:05.613 INFO:tasks.workunit.client.0.smithi012.stdout:1/743: sync 2024-03-18T14:51:05.614 INFO:tasks.workunit.client.0.smithi012.stdout:8/950: sync 2024-03-18T14:51:05.617 INFO:tasks.workunit.client.0.smithi012.stdout:7/818: sync 2024-03-18T14:51:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:4/719: link d6/d19/d39/l8a d6/da/d89/da1/dbb/ddc/lf2 0 2024-03-18T14:51:05.621 INFO:tasks.workunit.client.0.smithi012.stdout:4/720: write d6/f2e [4063035,85939] 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/833: creat d1/d2d/f112 x:0 0 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/834: readlink d1/d20/d23/d3b/d33/d51/la3 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/835: fdatasync d1/d2d/f112 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/836: readlink d1/d20/l43 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/837: getdents d1/d20/d4e/d54/d67/d10c 0 2024-03-18T14:51:05.623 INFO:tasks.workunit.client.0.smithi012.stdout:0/838: chown d1/d20/d23/d7f/da7/ld3 40392 1 2024-03-18T14:51:05.624 INFO:tasks.workunit.client.1.smithi100.stdout:4/459: dwrite d9/f10 [8388608,4194304] 0 2024-03-18T14:51:05.624 INFO:tasks.workunit.client.1.smithi100.stdout:6/486: dwrite d6/d12/f43 [0,4194304] 0 2024-03-18T14:51:05.625 INFO:tasks.workunit.client.1.smithi100.stdout:0/556: rename dd/d31/d91/d7b/c9f to dd/d2b/d95/d79/d7d/cca 0 2024-03-18T14:51:05.625 INFO:tasks.workunit.client.1.smithi100.stdout:0/557: stat dd/d2b/d95/d79/d7d/da9/dac/fc6 0 2024-03-18T14:51:05.627 INFO:tasks.workunit.client.0.smithi012.stdout:6/741: rename d17/d33 to d17/d38/d51/dcc/dd8 0 2024-03-18T14:51:05.630 INFO:tasks.workunit.client.1.smithi100.stdout:1/480: link d8/d1e/d11/d27/l4e d8/d1e/l8b 0 2024-03-18T14:51:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:2/844: symlink d0/d6b/dbf/d33/d77/da0/l10d 0 2024-03-18T14:51:05.633 INFO:tasks.workunit.client.0.smithi012.stdout:2/845: write d0/d3c/d40/d47/f49 [3290858,51857] 0 2024-03-18T14:51:05.633 INFO:tasks.workunit.client.1.smithi100.stdout:0/558: read dd/d61/f72 [687514,33524] 0 2024-03-18T14:51:05.633 INFO:tasks.workunit.client.1.smithi100.stdout:0/559: fsync dd/d2b/f3d 0 2024-03-18T14:51:05.635 INFO:tasks.workunit.client.0.smithi012.stdout:5/792: creat d3/d15/d42/dc6/d73/f107 x:0 0 0 2024-03-18T14:51:05.637 INFO:tasks.workunit.client.1.smithi100.stdout:6/487: dwrite d6/d7/df/d26/d4e/d66/d36/f6e [0,4194304] 0 2024-03-18T14:51:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:8/951: truncate d0/d5a/d5e/dd7/d84/d54/f85 2299731 0 2024-03-18T14:51:05.638 INFO:tasks.workunit.client.0.smithi012.stdout:8/952: creat d0/d47/d59/dc9/f130 x:0 0 0 2024-03-18T14:51:05.639 INFO:tasks.workunit.client.1.smithi100.stdout:9/460: mknod da/dd/d3e/c8d 0 2024-03-18T14:51:05.641 INFO:tasks.workunit.client.0.smithi012.stdout:1/744: mknod d8/db/d10/d31/d43/d5e/d65/d84/dc1/cf5 0 2024-03-18T14:51:05.645 INFO:tasks.workunit.client.0.smithi012.stdout:7/819: mknod dd/d13/d23/d2f/c11f 0 2024-03-18T14:51:05.648 INFO:tasks.workunit.client.0.smithi012.stdout:4/721: symlink d6/d1a/d98/daf/lf3 0 2024-03-18T14:51:05.649 INFO:tasks.workunit.client.1.smithi100.stdout:7/470: rmdir d5/d30/d4d/d5a 39 2024-03-18T14:51:05.651 INFO:tasks.workunit.client.0.smithi012.stdout:0/839: mkdir d1/d2d/d113 0 2024-03-18T14:51:05.654 INFO:tasks.workunit.client.0.smithi012.stdout:6/742: creat d17/d38/d51/d66/d7d/fd9 x:0 0 0 2024-03-18T14:51:05.657 INFO:tasks.workunit.client.0.smithi012.stdout:2/846: creat d0/da2/f10e x:0 0 0 2024-03-18T14:51:05.658 INFO:tasks.workunit.client.0.smithi012.stdout:5/793: symlink d3/d15/d42/dc6/l108 0 2024-03-18T14:51:05.667 INFO:tasks.workunit.client.1.smithi100.stdout:5/421: link d29/f57 d29/d2d/d39/f84 0 2024-03-18T14:51:05.670 INFO:tasks.workunit.client.1.smithi100.stdout:4/460: symlink d9/d27/d62/d7b/l8a 0 2024-03-18T14:51:05.672 INFO:tasks.workunit.client.0.smithi012.stdout:8/953: truncate d0/d5a/d5e/f82 432011 0 2024-03-18T14:51:05.673 INFO:tasks.workunit.client.0.smithi012.stdout:5/794: dwrite d3/d15/d5f/fc0 [0,4194304] 0 2024-03-18T14:51:05.676 INFO:tasks.workunit.client.0.smithi012.stdout:1/745: mkdir d8/db/d10/d31/d43/d5e/df6 0 2024-03-18T14:51:05.678 INFO:tasks.workunit.client.0.smithi012.stdout:7/820: mknod dd/d13/db7/c120 0 2024-03-18T14:51:05.678 INFO:tasks.workunit.client.0.smithi012.stdout:7/821: creat dd/d13/d23/d76/d6f/f121 x:0 0 0 2024-03-18T14:51:05.681 INFO:tasks.workunit.client.0.smithi012.stdout:4/722: unlink d6/f4c 0 2024-03-18T14:51:05.682 INFO:tasks.workunit.client.1.smithi100.stdout:1/481: creat d8/d1e/d11/d27/f8c x:0 0 0 2024-03-18T14:51:05.684 INFO:tasks.workunit.client.1.smithi100.stdout:0/560: symlink dd/d20/lcb 0 2024-03-18T14:51:05.686 INFO:tasks.workunit.client.0.smithi012.stdout:6/743: creat d17/d26/d36/d4d/d82/dcd/fda x:0 0 0 2024-03-18T14:51:05.687 INFO:tasks.workunit.client.0.smithi012.stdout:6/744: truncate d17/d26/fbc 645881 0 2024-03-18T14:51:05.689 INFO:tasks.workunit.client.0.smithi012.stdout:6/745: dread d17/d26/fbc [0,4194304] 0 2024-03-18T14:51:05.689 INFO:tasks.workunit.client.0.smithi012.stdout:2/847: symlink d0/d107/l10f 0 2024-03-18T14:51:05.691 INFO:tasks.workunit.client.1.smithi100.stdout:5/422: dwrite f7 [4194304,4194304] 0 2024-03-18T14:51:05.691 INFO:tasks.workunit.client.1.smithi100.stdout:1/482: dread d8/f69 [0,4194304] 0 2024-03-18T14:51:05.693 INFO:tasks.workunit.client.1.smithi100.stdout:6/488: truncate d6/d7/df/d26/d4e/d66/d36/d4f/f73 2042134 0 2024-03-18T14:51:05.696 INFO:tasks.workunit.client.0.smithi012.stdout:8/954: link d0/f7 d0/d5a/d5e/dd7/d84/d2a/d3e/f131 0 2024-03-18T14:51:05.696 INFO:tasks.workunit.client.0.smithi012.stdout:8/955: dread - d0/d5a/d5e/dd7/d122/f12d zero size 2024-03-18T14:51:05.699 INFO:tasks.workunit.client.1.smithi100.stdout:9/461: rename da/d51/f52 to da/d46/f8e 0 2024-03-18T14:51:05.703 INFO:tasks.workunit.client.1.smithi100.stdout:8/503: rmdir d4/dc/d1f/d33 39 2024-03-18T14:51:05.703 INFO:tasks.workunit.client.1.smithi100.stdout:8/504: read - d4/dc/dd/de/f93 zero size 2024-03-18T14:51:05.705 INFO:tasks.workunit.client.0.smithi012.stdout:6/746: dwrite d17/d38/d51/f53 [0,4194304] 0 2024-03-18T14:51:05.706 INFO:tasks.workunit.client.0.smithi012.stdout:5/795: readlink d3/d15/d42/dc6/d73/l77 0 2024-03-18T14:51:05.706 INFO:tasks.workunit.client.0.smithi012.stdout:5/796: readlink d3/d29/d35/d5b/l66 0 2024-03-18T14:51:05.706 INFO:tasks.workunit.client.0.smithi012.stdout:5/797: dread - d3/dad/ff6 zero size 2024-03-18T14:51:05.707 INFO:tasks.workunit.client.0.smithi012.stdout:6/747: dread d17/d18/db8/f55 [0,4194304] 0 2024-03-18T14:51:05.707 INFO:tasks.workunit.client.0.smithi012.stdout:6/748: readlink d17/d19/d27/d2f/lab 0 2024-03-18T14:51:05.708 INFO:tasks.workunit.client.1.smithi100.stdout:7/471: link d5/d30/c94 d5/d90/da1/caf 0 2024-03-18T14:51:05.708 INFO:tasks.workunit.client.0.smithi012.stdout:5/798: dread d3/d15/f37 [0,4194304] 0 2024-03-18T14:51:05.710 INFO:tasks.workunit.client.0.smithi012.stdout:1/746: symlink d8/db/d10/d16/da1/db9/lf7 0 2024-03-18T14:51:05.712 INFO:tasks.workunit.client.1.smithi100.stdout:9/462: write da/d46/f8e [2705102,121153] 0 2024-03-18T14:51:05.712 INFO:tasks.workunit.client.1.smithi100.stdout:4/461: truncate d9/d17/f23 925268 0 2024-03-18T14:51:05.712 INFO:tasks.workunit.client.1.smithi100.stdout:4/462: truncate d9/f3b 700623 0 2024-03-18T14:51:05.721 INFO:tasks.workunit.client.0.smithi012.stdout:7/822: truncate dd/d13/d23/d33/fff 2322873 0 2024-03-18T14:51:05.724 INFO:tasks.workunit.client.1.smithi100.stdout:5/423: creat d29/d2d/d39/d83/f85 x:0 0 0 2024-03-18T14:51:05.727 INFO:tasks.workunit.client.0.smithi012.stdout:4/723: link d6/da/d89/d95/fd4 d6/da/d89/d97/dc5/dd3/ff4 0 2024-03-18T14:51:05.729 INFO:tasks.workunit.client.0.smithi012.stdout:1/747: dwrite d8/db/d10/d16/d54/fa3 [0,4194304] 0 2024-03-18T14:51:05.730 INFO:tasks.workunit.client.0.smithi012.stdout:0/840: getdents d1/d20/d23/d3b 0 2024-03-18T14:51:05.730 INFO:tasks.workunit.client.1.smithi100.stdout:6/489: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/f96 x:0 0 0 2024-03-18T14:51:05.732 INFO:tasks.workunit.client.1.smithi100.stdout:8/505: rename d4/dc/d1f/d59/d6c/d7c/l9a to d4/dc/dd/de/lae 0 2024-03-18T14:51:05.732 INFO:tasks.workunit.client.1.smithi100.stdout:8/506: truncate d4/dc/d1f/f5d 5151499 0 2024-03-18T14:51:05.734 INFO:tasks.workunit.client.0.smithi012.stdout:2/848: creat d0/d6c/db7/f110 x:0 0 0 2024-03-18T14:51:05.734 INFO:tasks.workunit.client.0.smithi012.stdout:2/849: chown d0/d10 17682218 1 2024-03-18T14:51:05.735 INFO:tasks.workunit.client.0.smithi012.stdout:8/956: symlink d0/d5a/d19/d57/l132 0 2024-03-18T14:51:05.736 INFO:tasks.workunit.client.1.smithi100.stdout:7/472: creat d5/d22/d24/d39/fb0 x:0 0 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:4/463: dwrite d9/d27/d42/f6a [0,4194304] 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:4/464: stat d9/d17/l4f 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:9/463: dwrite da/d2c/f89 [0,4194304] 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:9/464: dread - da/d46/d4d/d67/f68 zero size 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:9/465: write da/dd/d29/f5a [1111500,2484] 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:9/466: write da/dd/d29/d71/d72/f78 [885300,31075] 0 2024-03-18T14:51:05.744 INFO:tasks.workunit.client.1.smithi100.stdout:6/490: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f22 [0,4194304] 0 2024-03-18T14:51:05.745 INFO:tasks.workunit.client.1.smithi100.stdout:9/467: creat da/dd/f8f x:0 0 0 2024-03-18T14:51:05.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/491: readlink d6/d7/df/d26/d4e/d66/d36/l49 0 2024-03-18T14:51:05.745 INFO:tasks.workunit.client.1.smithi100.stdout:5/424: link d29/d2d/d39/d71/l72 d29/d2d/d39/d65/d7d/l86 0 2024-03-18T14:51:05.745 INFO:tasks.workunit.client.0.smithi012.stdout:7/823: dwrite dd/d13/d23/d2f/fb9 [0,4194304] 0 2024-03-18T14:51:05.745 INFO:tasks.workunit.client.0.smithi012.stdout:6/749: rmdir d17/d38/d51/da8 39 2024-03-18T14:51:05.746 INFO:tasks.workunit.client.1.smithi100.stdout:8/507: mknod d4/dc/dd/d48/caf 0 2024-03-18T14:51:05.751 INFO:tasks.workunit.client.0.smithi012.stdout:5/799: creat d3/d15/d42/dc6/d73/dd3/dfb/f109 x:0 0 0 2024-03-18T14:51:05.754 INFO:tasks.workunit.client.1.smithi100.stdout:7/473: mkdir d5/d22/d3b/d9f/da0/db1 0 2024-03-18T14:51:05.755 INFO:tasks.workunit.client.1.smithi100.stdout:2/449: sync 2024-03-18T14:51:05.755 INFO:tasks.workunit.client.1.smithi100.stdout:2/450: stat d3/da/f43 0 2024-03-18T14:51:05.760 INFO:tasks.workunit.client.1.smithi100.stdout:4/465: symlink d9/d27/d42/l8b 0 2024-03-18T14:51:05.761 INFO:tasks.workunit.client.0.smithi012.stdout:4/724: symlink d6/d19/da5/lf5 0 2024-03-18T14:51:05.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/508: dwrite d4/dc/d1f/f9c [0,4194304] 0 2024-03-18T14:51:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:1/748: unlink d8/db/d10/d31/d43/d5e/d65/d84/f92 0 2024-03-18T14:51:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:1/749: write d8/db/d10/d31/d43/d5e/fbc [775092,14615] 0 2024-03-18T14:51:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:1/750: truncate d8/db/d10/d31/d81/fd3 254033 0 2024-03-18T14:51:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:1/751: chown d8/d1e/f9f 27201539 1 2024-03-18T14:51:05.771 INFO:tasks.workunit.client.0.smithi012.stdout:1/752: write d8/db/d10/d31/d43/d5e/d65/d84/fde [44355,78408] 0 2024-03-18T14:51:05.772 INFO:tasks.workunit.client.0.smithi012.stdout:1/753: write d8/db/d10/d16/da1/d2e/d9c/d9d/fdb [3896707,66137] 0 2024-03-18T14:51:05.772 INFO:tasks.workunit.client.0.smithi012.stdout:1/754: truncate d8/d1e/f7c 1840977 0 2024-03-18T14:51:05.772 INFO:tasks.workunit.client.0.smithi012.stdout:1/755: write d8/db/d10/d31/d81/fa4 [857139,108755] 0 2024-03-18T14:51:05.772 INFO:tasks.workunit.client.0.smithi012.stdout:1/756: chown d8/db/d10/d16/d54/fa3 0 1 2024-03-18T14:51:05.772 INFO:tasks.workunit.client.1.smithi100.stdout:9/468: mknod da/dd/c90 0 2024-03-18T14:51:05.773 INFO:tasks.workunit.client.0.smithi012.stdout:0/841: creat d1/d20/d23/d3b/d33/dea/f114 x:0 0 0 2024-03-18T14:51:05.773 INFO:tasks.workunit.client.0.smithi012.stdout:0/842: fdatasync d1/d20/d7b/f81 0 2024-03-18T14:51:05.774 INFO:tasks.workunit.client.1.smithi100.stdout:6/492: getdents d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27 0 2024-03-18T14:51:05.775 INFO:tasks.workunit.client.0.smithi012.stdout:2/850: link d0/d6b/dbf/f73 d0/d6c/f111 0 2024-03-18T14:51:05.776 INFO:tasks.workunit.client.1.smithi100.stdout:5/425: symlink d29/d2d/d39/d71/l87 0 2024-03-18T14:51:05.776 INFO:tasks.workunit.client.1.smithi100.stdout:5/426: chown d29/d2d/d32/d38/c48 381237609 1 2024-03-18T14:51:05.777 INFO:tasks.workunit.client.0.smithi012.stdout:8/957: truncate d0/d47/f4f 5291286 0 2024-03-18T14:51:05.777 INFO:tasks.workunit.client.0.smithi012.stdout:8/958: chown d0/d5a/d5e/dd7/d84/d54/d9f 751 1 2024-03-18T14:51:05.782 INFO:tasks.workunit.client.1.smithi100.stdout:7/474: symlink d5/d22/d24/d6e/lb2 0 2024-03-18T14:51:05.783 INFO:tasks.workunit.client.0.smithi012.stdout:1/757: dread d8/db/d10/d16/da1/d2e/d9c/fb1 [4194304,4194304] 0 2024-03-18T14:51:05.783 INFO:tasks.workunit.client.0.smithi012.stdout:1/758: write d8/db/d10/d31/d81/fd3 [1023995,23935] 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:7/824: mkdir dd/d13/d23/d2f/de1/d122 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:7/825: write dd/d13/f73 [5129379,128735] 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:1/759: dread d8/d1e/d27/d44/f97 [0,4194304] 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:1/760: write d8/db/d10/d31/fbb [299109,45281] 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.0.smithi012.stdout:1/761: truncate d8/db/f25 5202212 0 2024-03-18T14:51:05.784 INFO:tasks.workunit.client.1.smithi100.stdout:2/451: creat d3/d1a/d76/f9f x:0 0 0 2024-03-18T14:51:05.785 INFO:tasks.workunit.client.1.smithi100.stdout:4/466: dwrite d9/d27/d42/f6a [0,4194304] 0 2024-03-18T14:51:05.785 INFO:tasks.workunit.client.1.smithi100.stdout:4/467: readlink l5 0 2024-03-18T14:51:05.787 INFO:tasks.workunit.client.0.smithi012.stdout:6/750: mknod d17/d19/d27/db1/cdb 0 2024-03-18T14:51:05.790 INFO:tasks.workunit.client.0.smithi012.stdout:2/851: dread d0/d10/f16 [0,4194304] 0 2024-03-18T14:51:05.790 INFO:tasks.workunit.client.0.smithi012.stdout:2/852: fsync d0/d6b/d9f/fc5 0 2024-03-18T14:51:05.793 INFO:tasks.workunit.client.0.smithi012.stdout:8/959: dread d0/d5a/d19/dad/f4b [4194304,4194304] 0 2024-03-18T14:51:05.796 INFO:tasks.workunit.client.0.smithi012.stdout:4/725: symlink d6/d19/lf6 0 2024-03-18T14:51:05.796 INFO:tasks.workunit.client.0.smithi012.stdout:4/726: creat d6/d1a/d92/ff7 x:0 0 0 2024-03-18T14:51:05.796 INFO:tasks.workunit.client.0.smithi012.stdout:4/727: creat d6/db6/ff8 x:0 0 0 2024-03-18T14:51:05.797 INFO:tasks.workunit.client.1.smithi100.stdout:4/468: dread d9/f3b [0,4194304] 0 2024-03-18T14:51:05.797 INFO:tasks.workunit.client.1.smithi100.stdout:4/469: truncate d9/d3a/f86 405654 0 2024-03-18T14:51:05.806 INFO:tasks.workunit.client.0.smithi012.stdout:0/843: unlink d1/d16/d6e/fc9 0 2024-03-18T14:51:05.806 INFO:tasks.workunit.client.0.smithi012.stdout:7/826: dwrite dd/d13/d23/f99 [0,4194304] 0 2024-03-18T14:51:05.810 INFO:tasks.workunit.client.1.smithi100.stdout:4/470: dread d9/d3a/f69 [0,4194304] 0 2024-03-18T14:51:05.810 INFO:tasks.workunit.client.1.smithi100.stdout:4/471: creat d9/f8c x:0 0 0 2024-03-18T14:51:05.821 INFO:tasks.workunit.client.1.smithi100.stdout:6/493: rmdir d6/d7/df/d26/d4e/d66/d36/d4f/d71 39 2024-03-18T14:51:05.824 INFO:tasks.workunit.client.1.smithi100.stdout:5/427: truncate d29/f50 566395 0 2024-03-18T14:51:05.824 INFO:tasks.workunit.client.0.smithi012.stdout:1/762: truncate d8/d1e/d27/d44/fa0 991472 0 2024-03-18T14:51:05.825 INFO:tasks.workunit.client.1.smithi100.stdout:7/475: mkdir d5/d22/db3 0 2024-03-18T14:51:05.826 INFO:tasks.workunit.client.0.smithi012.stdout:6/751: creat d17/d38/fdc x:0 0 0 2024-03-18T14:51:05.827 INFO:tasks.workunit.client.0.smithi012.stdout:6/752: truncate d17/d38/d51/d66/faf 1077276 0 2024-03-18T14:51:05.828 INFO:tasks.workunit.client.1.smithi100.stdout:2/452: truncate d3/d6/f67 3838619 0 2024-03-18T14:51:05.828 INFO:tasks.workunit.client.1.smithi100.stdout:2/453: chown d3/d13/l75 5814 1 2024-03-18T14:51:05.828 INFO:tasks.workunit.client.1.smithi100.stdout:2/454: chown d3/d6 28145211 1 2024-03-18T14:51:05.829 INFO:tasks.workunit.client.0.smithi012.stdout:8/960: symlink d0/d5a/d5e/ddf/l133 0 2024-03-18T14:51:05.830 INFO:tasks.workunit.client.0.smithi012.stdout:5/800: rmdir d3/d84/dfa 39 2024-03-18T14:51:05.830 INFO:tasks.workunit.client.0.smithi012.stdout:5/801: dread - d3/d29/d36/d9b/fbd zero size 2024-03-18T14:51:05.831 INFO:tasks.workunit.client.1.smithi100.stdout:5/428: write d29/f5a [2640520,31550] 0 2024-03-18T14:51:05.832 INFO:tasks.workunit.client.1.smithi100.stdout:5/429: readlink d29/l7a 0 2024-03-18T14:51:05.832 INFO:tasks.workunit.client.1.smithi100.stdout:5/430: dread - d29/d2d/d39/d61/f6b zero size 2024-03-18T14:51:05.833 INFO:tasks.workunit.client.0.smithi012.stdout:9/862: sync 2024-03-18T14:51:05.833 INFO:tasks.workunit.client.0.smithi012.stdout:3/771: sync 2024-03-18T14:51:05.833 INFO:tasks.workunit.client.0.smithi012.stdout:9/863: chown d3/de6/dbf/d77/da8/f96 24041 1 2024-03-18T14:51:05.837 INFO:tasks.workunit.client.0.smithi012.stdout:4/728: creat d6/d1a/ff9 x:0 0 0 2024-03-18T14:51:05.841 INFO:tasks.workunit.client.0.smithi012.stdout:9/864: dread d3/f63 [0,4194304] 0 2024-03-18T14:51:05.841 INFO:tasks.workunit.client.0.smithi012.stdout:9/865: chown d3/de6/dbf/d77/d52/c79 7 1 2024-03-18T14:51:05.841 INFO:tasks.workunit.client.0.smithi012.stdout:4/729: dread d6/da/d58/f9e [0,4194304] 0 2024-03-18T14:51:05.844 INFO:tasks.workunit.client.1.smithi100.stdout:4/472: mkdir d9/d13/d25/d51/d8d 0 2024-03-18T14:51:05.844 INFO:tasks.workunit.client.1.smithi100.stdout:4/473: fsync d9/d27/f4b 0 2024-03-18T14:51:05.844 INFO:tasks.workunit.client.1.smithi100.stdout:4/474: dread - d9/f7f zero size 2024-03-18T14:51:05.846 INFO:tasks.workunit.client.1.smithi100.stdout:6/494: unlink d6/d7/df/l57 0 2024-03-18T14:51:05.848 INFO:tasks.workunit.client.1.smithi100.stdout:7/476: mkdir d5/d30/d4d/db4 0 2024-03-18T14:51:05.849 INFO:tasks.workunit.client.0.smithi012.stdout:0/844: mknod d1/d20/d4e/d54/d67/d10c/c115 0 2024-03-18T14:51:05.849 INFO:tasks.workunit.client.1.smithi100.stdout:7/477: write d5/d14/d16/d2c/f5b [4244504,98041] 0 2024-03-18T14:51:05.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/431: mknod d29/d2d/d39/d71/c88 0 2024-03-18T14:51:05.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/432: fdatasync d29/d4a/f54 0 2024-03-18T14:51:05.852 INFO:tasks.workunit.client.0.smithi012.stdout:6/753: dwrite d17/d38/d51/d66/d7d/fd9 [0,4194304] 0 2024-03-18T14:51:05.853 INFO:tasks.workunit.client.0.smithi012.stdout:7/827: creat dd/d13/d23/d76/d109/dfa/f123 x:0 0 0 2024-03-18T14:51:05.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/509: rename d4/dc/d1f/d59/d6c/d7c to d4/dc/dd/de/d19/db0 0 2024-03-18T14:51:05.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/510: chown d4/dc/d1f/f32 175038 1 2024-03-18T14:51:05.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/511: creat d4/dc/d1f/d59/fb1 x:0 0 0 2024-03-18T14:51:05.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/512: truncate d4/dc/d31/f46 822834 0 2024-03-18T14:51:05.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/513: dread - d4/dc/dd/de/d21/d23/d25/f71 zero size 2024-03-18T14:51:05.858 INFO:tasks.workunit.client.1.smithi100.stdout:4/475: creat d9/d27/d62/d7b/f8e x:0 0 0 2024-03-18T14:51:05.861 INFO:tasks.workunit.client.1.smithi100.stdout:6/495: mknod d6/d7/df/d26/d4e/d66/d36/d4f/c97 0 2024-03-18T14:51:05.861 INFO:tasks.workunit.client.1.smithi100.stdout:6/496: chown d6/d12 6155 1 2024-03-18T14:51:05.862 INFO:tasks.workunit.client.0.smithi012.stdout:8/961: mknod d0/d47/d59/d67/c134 0 2024-03-18T14:51:05.862 INFO:tasks.workunit.client.0.smithi012.stdout:8/962: chown d0/d47/la0 0 1 2024-03-18T14:51:05.863 INFO:tasks.workunit.client.1.smithi100.stdout:6/497: dread d6/d7/df/d26/d4e/d66/d36/f54 [0,4194304] 0 2024-03-18T14:51:05.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/433: rename d29/d2d/d39/d61/f69 to d29/d2d/d39/d61/f89 0 2024-03-18T14:51:05.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/434: readlink d29/d2d/d39/l51 0 2024-03-18T14:51:05.864 INFO:tasks.workunit.client.0.smithi012.stdout:5/802: creat d3/dad/df2/d102/f10a x:0 0 0 2024-03-18T14:51:05.866 INFO:tasks.workunit.client.1.smithi100.stdout:9/469: rename da/dd/d3b/c6d to da/d46/c91 0 2024-03-18T14:51:05.867 INFO:tasks.workunit.client.1.smithi100.stdout:4/476: dread d9/f11 [0,4194304] 0 2024-03-18T14:51:05.867 INFO:tasks.workunit.client.1.smithi100.stdout:7/478: dwrite d5/f35 [0,4194304] 0 2024-03-18T14:51:05.867 INFO:tasks.workunit.client.0.smithi012.stdout:3/772: mknod dc/d3c/d59/cf6 0 2024-03-18T14:51:05.871 INFO:tasks.workunit.client.0.smithi012.stdout:9/866: link d3/de6/dbf/d77/d52/d95/deb/f10d d3/de6/dbf/d77/d52/d95/f11b 0 2024-03-18T14:51:05.871 INFO:tasks.workunit.client.0.smithi012.stdout:9/867: write d3/d9/f39 [2579206,95315] 0 2024-03-18T14:51:05.871 INFO:tasks.workunit.client.0.smithi012.stdout:9/868: creat d3/de6/dbf/d66/d69/d7f/d8c/d8b/f11c x:0 0 0 2024-03-18T14:51:05.871 INFO:tasks.workunit.client.0.smithi012.stdout:9/869: write d3/de6/dbf/d66/d69/d7f/d8c/d8b/f108 [492538,127147] 0 2024-03-18T14:51:05.873 INFO:tasks.workunit.client.1.smithi100.stdout:8/514: creat d4/dc/d31/d85/fb2 x:0 0 0 2024-03-18T14:51:05.875 INFO:tasks.workunit.client.0.smithi012.stdout:4/730: unlink d6/d19/d6c/fe1 0 2024-03-18T14:51:05.877 INFO:tasks.workunit.client.1.smithi100.stdout:5/435: symlink d29/d2d/d39/d74/l8a 0 2024-03-18T14:51:05.877 INFO:tasks.workunit.client.1.smithi100.stdout:5/436: chown f26 3940 1 2024-03-18T14:51:05.877 INFO:tasks.workunit.client.1.smithi100.stdout:7/479: dread d5/d14/d16/d2c/d86/f50 [0,4194304] 0 2024-03-18T14:51:05.879 INFO:tasks.workunit.client.1.smithi100.stdout:2/455: rename d3/d1a/d6c/l93 to d3/da/de/d20/d82/d95/la0 0 2024-03-18T14:51:05.880 INFO:tasks.workunit.client.0.smithi012.stdout:0/845: mkdir d1/d16/d6e/d109/d116 0 2024-03-18T14:51:05.880 INFO:tasks.workunit.client.0.smithi012.stdout:0/846: creat d1/d2d/f117 x:0 0 0 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.0.smithi012.stdout:0/847: getdents d1/d16/d6e/d109/d116 0 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.0.smithi012.stdout:0/848: dread - d1/d20/d4e/d54/fe6 zero size 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.1.smithi100.stdout:0/561: sync 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.1.smithi100.stdout:1/483: sync 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.1.smithi100.stdout:3/518: sync 2024-03-18T14:51:05.881 INFO:tasks.workunit.client.1.smithi100.stdout:9/470: mknod da/d51/d55/c92 0 2024-03-18T14:51:05.884 INFO:tasks.workunit.client.1.smithi100.stdout:4/477: creat d9/d13/d39/f8f x:0 0 0 2024-03-18T14:51:05.884 INFO:tasks.workunit.client.1.smithi100.stdout:4/478: write d9/f3b [1289621,61971] 0 2024-03-18T14:51:05.885 INFO:tasks.workunit.client.1.smithi100.stdout:8/515: getdents d4/dc/dd/de/d19/d2c/da2 0 2024-03-18T14:51:05.885 INFO:tasks.workunit.client.1.smithi100.stdout:8/516: fsync d4/dc/d1f/d33/f8d 0 2024-03-18T14:51:05.888 INFO:tasks.workunit.client.0.smithi012.stdout:6/754: symlink d17/ldd 0 2024-03-18T14:51:05.888 INFO:tasks.workunit.client.0.smithi012.stdout:5/803: dwrite d3/d15/f106 [4194304,4194304] 0 2024-03-18T14:51:05.889 INFO:tasks.workunit.client.1.smithi100.stdout:6/498: dwrite d6/d12/f1f [4194304,4194304] 0 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/731: read d6/da/fd [2685619,24916] 0 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/732: dread - d6/d1a/d30/d46/d6f/d7c/fa8 zero size 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/733: dread - d6/d1a/d30/d46/d6f/d7c/fa8 zero size 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/734: truncate d6/d19/d6c/d96/df0/ff1 105058 0 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/735: fdatasync d6/d1a/d92/f5d 0 2024-03-18T14:51:05.890 INFO:tasks.workunit.client.0.smithi012.stdout:4/736: fdatasync d6/d1a/f2a 0 2024-03-18T14:51:05.891 INFO:tasks.workunit.client.1.smithi100.stdout:3/519: dread d3/d16/d13/d1f/f78 [0,4194304] 0 2024-03-18T14:51:05.891 INFO:tasks.workunit.client.1.smithi100.stdout:3/520: stat d3/dd/c6d 0 2024-03-18T14:51:05.891 INFO:tasks.workunit.client.1.smithi100.stdout:5/437: link d29/d2d/f6f d29/d2d/d39/d83/f8b 0 2024-03-18T14:51:05.893 INFO:tasks.workunit.client.0.smithi012.stdout:7/828: unlink dd/d13/d23/d76/d6f/dd9/l10d 0 2024-03-18T14:51:05.893 INFO:tasks.workunit.client.0.smithi012.stdout:7/829: fdatasync dd/d13/d23/d2f/d53/f5d 0 2024-03-18T14:51:05.893 INFO:tasks.workunit.client.0.smithi012.stdout:7/830: fsync dd/d13/d23/d2f/d53/dc6/df1/ff2 0 2024-03-18T14:51:05.893 INFO:tasks.workunit.client.0.smithi012.stdout:7/831: readlink dd/lda 0 2024-03-18T14:51:05.895 INFO:tasks.workunit.client.1.smithi100.stdout:7/480: rename d5/d90/da1/caf to d5/d14/d16/d44/cb5 0 2024-03-18T14:51:05.898 INFO:tasks.workunit.client.1.smithi100.stdout:2/456: symlink d3/d6/d29/d36/d4a/la1 0 2024-03-18T14:51:05.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/804: dread d3/d29/d36/f45 [0,4194304] 0 2024-03-18T14:51:05.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/805: creat d3/d29/d9d/dba/f10b x:0 0 0 2024-03-18T14:51:05.899 INFO:tasks.workunit.client.1.smithi100.stdout:2/457: dread d3/d6/d29/d36/d49/f54 [0,4194304] 0 2024-03-18T14:51:05.900 INFO:tasks.workunit.client.0.smithi012.stdout:4/737: write d6/f8c [3387309,14179] 0 2024-03-18T14:51:05.900 INFO:tasks.workunit.client.1.smithi100.stdout:3/521: read d3/d16/d13/d17/f33 [229861,85181] 0 2024-03-18T14:51:05.901 INFO:tasks.workunit.client.0.smithi012.stdout:5/806: dread d3/d1e/f1a [0,4194304] 0 2024-03-18T14:51:05.901 INFO:tasks.workunit.client.1.smithi100.stdout:1/484: mkdir d8/d30/d61/d8d 0 2024-03-18T14:51:05.902 INFO:tasks.workunit.client.0.smithi012.stdout:1/763: rmdir d8/db/d10/d16/da1/d2e 39 2024-03-18T14:51:05.903 INFO:tasks.workunit.client.1.smithi100.stdout:5/438: dread d29/d2d/d32/f35 [4194304,4194304] 0 2024-03-18T14:51:05.906 INFO:tasks.workunit.client.1.smithi100.stdout:0/562: mkdir dd/d31/d91/d56/d71/dcc 0 2024-03-18T14:51:05.907 INFO:tasks.workunit.client.0.smithi012.stdout:8/963: rename d0/f16 to d0/d5a/f135 0 2024-03-18T14:51:05.911 INFO:tasks.workunit.client.0.smithi012.stdout:3/773: symlink dc/d30/d3a/da0/dc5/dc7/de7/lf7 0 2024-03-18T14:51:05.914 INFO:tasks.workunit.client.1.smithi100.stdout:8/517: link d4/dc/l58 d4/dc/dd/de/d19/d2c/lb3 0 2024-03-18T14:51:05.918 INFO:tasks.workunit.client.0.smithi012.stdout:9/870: mkdir d3/de6/dbf/d66/d11d 0 2024-03-18T14:51:05.918 INFO:tasks.workunit.client.0.smithi012.stdout:9/871: chown d3/d9/d5c/l88 57 1 2024-03-18T14:51:05.919 INFO:tasks.workunit.client.1.smithi100.stdout:6/499: symlink d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/l98 0 2024-03-18T14:51:05.920 INFO:tasks.workunit.client.1.smithi100.stdout:5/439: dwrite d29/d2d/d39/f56 [0,4194304] 0 2024-03-18T14:51:05.924 INFO:tasks.workunit.client.0.smithi012.stdout:0/849: truncate d1/d20/d23/d3b/d33/d51/fa8 3100737 0 2024-03-18T14:51:05.926 INFO:tasks.workunit.client.0.smithi012.stdout:6/755: mkdir d17/d18/dc1/dde 0 2024-03-18T14:51:05.930 INFO:tasks.workunit.client.0.smithi012.stdout:7/832: symlink dd/d13/d23/d76/d6f/db3/l124 0 2024-03-18T14:51:05.930 INFO:tasks.workunit.client.0.smithi012.stdout:7/833: chown dd/d13/d23/d2f/d53/dc6/ff9 308904 1 2024-03-18T14:51:05.932 INFO:tasks.workunit.client.1.smithi100.stdout:5/440: dread d29/f2b [0,4194304] 0 2024-03-18T14:51:05.935 INFO:tasks.workunit.client.1.smithi100.stdout:5/441: dread d29/d2d/d39/d61/f89 [0,4194304] 0 2024-03-18T14:51:05.935 INFO:tasks.workunit.client.1.smithi100.stdout:2/458: mknod d3/d6/ca2 0 2024-03-18T14:51:05.937 INFO:tasks.workunit.client.1.smithi100.stdout:7/481: dread d5/d14/d16/d2c/f43 [0,4194304] 0 2024-03-18T14:51:05.938 INFO:tasks.workunit.client.1.smithi100.stdout:3/522: link d3/l34 d3/dd/d21/lb8 0 2024-03-18T14:51:05.940 INFO:tasks.workunit.client.0.smithi012.stdout:4/738: link d6/d1a/d64/f78 d6/da/d89/d94/dbe/dd6/ffa 0 2024-03-18T14:51:05.942 INFO:tasks.workunit.client.0.smithi012.stdout:7/834: write dd/d13/d23/d2f/fb9 [2190742,95075] 0 2024-03-18T14:51:05.943 INFO:tasks.workunit.client.1.smithi100.stdout:0/563: mkdir dd/d61/dcd 0 2024-03-18T14:51:05.944 INFO:tasks.workunit.client.1.smithi100.stdout:9/471: getdents da/dd/d29/d71/d72 0 2024-03-18T14:51:05.945 INFO:tasks.workunit.client.0.smithi012.stdout:0/850: dwrite d1/d20/d23/d3b/dc1/fdd [0,4194304] 0 2024-03-18T14:51:05.945 INFO:tasks.workunit.client.0.smithi012.stdout:0/851: write d1/d20/d23/d26/f36 [406515,101848] 0 2024-03-18T14:51:05.946 INFO:tasks.workunit.client.1.smithi100.stdout:8/518: symlink d4/dc/d31/lb4 0 2024-03-18T14:51:05.947 INFO:tasks.workunit.client.0.smithi012.stdout:1/764: link d8/db/d10/d16/d54/l58 d8/db/d10/d16/da1/db9/lf8 0 2024-03-18T14:51:05.947 INFO:tasks.workunit.client.0.smithi012.stdout:1/765: write d8/db/d10/d31/fbb [938122,108552] 0 2024-03-18T14:51:05.947 INFO:tasks.workunit.client.0.smithi012.stdout:1/766: chown d8/d1e/d27/d44/f4f 1 1 2024-03-18T14:51:05.950 INFO:tasks.workunit.client.0.smithi012.stdout:8/964: unlink d0/d47/f10d 0 2024-03-18T14:51:05.954 INFO:tasks.workunit.client.0.smithi012.stdout:7/835: write dd/d13/d23/d76/f68 [837726,120207] 0 2024-03-18T14:51:05.955 INFO:tasks.workunit.client.0.smithi012.stdout:3/774: link dc/d3c/d59/d86/d9a/d64/c8d dc/d30/d3a/da0/cf8 0 2024-03-18T14:51:05.957 INFO:tasks.workunit.client.1.smithi100.stdout:2/459: symlink d3/d13/d40/la3 0 2024-03-18T14:51:05.959 INFO:tasks.workunit.client.0.smithi012.stdout:9/872: creat d3/d49/f11e x:0 0 0 2024-03-18T14:51:05.961 INFO:tasks.workunit.client.1.smithi100.stdout:7/482: creat d5/d22/d24/dac/fb6 x:0 0 0 2024-03-18T14:51:05.963 INFO:tasks.workunit.client.0.smithi012.stdout:6/756: creat d17/d19/d27/d58/fdf x:0 0 0 2024-03-18T14:51:05.964 INFO:tasks.workunit.client.0.smithi012.stdout:6/757: read - d17/d26/d36/d4d/fba zero size 2024-03-18T14:51:05.964 INFO:tasks.workunit.client.0.smithi012.stdout:6/758: truncate d17/f2c 4750359 0 2024-03-18T14:51:05.964 INFO:tasks.workunit.client.0.smithi012.stdout:1/767: dwrite d8/d1e/d27/f3c [4194304,4194304] 0 2024-03-18T14:51:05.965 INFO:tasks.workunit.client.0.smithi012.stdout:7/836: read dd/d13/d23/d2f/d7a/dab/fbe [112806,15032] 0 2024-03-18T14:51:05.965 INFO:tasks.workunit.client.0.smithi012.stdout:7/837: readlink dd/d13/d23/d33/d48/ld1 0 2024-03-18T14:51:05.965 INFO:tasks.workunit.client.0.smithi012.stdout:7/838: fdatasync dd/d13/d23/d76/f102 0 2024-03-18T14:51:05.965 INFO:tasks.workunit.client.0.smithi012.stdout:7/839: chown dd/d13/d23/f55 822 1 2024-03-18T14:51:05.967 INFO:tasks.workunit.client.1.smithi100.stdout:2/460: dread d3/fc [0,4194304] 0 2024-03-18T14:51:05.967 INFO:tasks.workunit.client.1.smithi100.stdout:2/461: write d3/d1a/f2d [1455416,13109] 0 2024-03-18T14:51:05.969 INFO:tasks.workunit.client.0.smithi012.stdout:3/775: dread dc/d30/fac [0,4194304] 0 2024-03-18T14:51:05.969 INFO:tasks.workunit.client.0.smithi012.stdout:3/776: read dc/f32 [34817,108895] 0 2024-03-18T14:51:05.969 INFO:tasks.workunit.client.0.smithi012.stdout:3/777: write dc/d3c/d59/d86/d9a/d54/f58 [3219575,105281] 0 2024-03-18T14:51:05.969 INFO:tasks.workunit.client.1.smithi100.stdout:0/564: mkdir dd/d20/d44/dce 0 2024-03-18T14:51:05.971 INFO:tasks.workunit.client.0.smithi012.stdout:6/759: dread d17/d18/f9e [0,4194304] 0 2024-03-18T14:51:05.972 INFO:tasks.workunit.client.0.smithi012.stdout:4/739: getdents d6/da/d89/d94/da2 0 2024-03-18T14:51:05.972 INFO:tasks.workunit.client.0.smithi012.stdout:9/873: dread d3/d49/fd4 [0,4194304] 0 2024-03-18T14:51:05.974 INFO:tasks.workunit.client.1.smithi100.stdout:5/442: rmdir d29/d2d/d39/d65 39 2024-03-18T14:51:05.974 INFO:tasks.workunit.client.1.smithi100.stdout:5/443: chown d29/d2d/f6f 11678747 1 2024-03-18T14:51:05.974 INFO:tasks.workunit.client.1.smithi100.stdout:5/444: dread - d29/d2d/d39/d65/f6a zero size 2024-03-18T14:51:05.975 INFO:tasks.workunit.client.0.smithi012.stdout:1/768: dread d8/db/d10/d16/d54/f82 [0,4194304] 0 2024-03-18T14:51:05.980 INFO:tasks.workunit.client.1.smithi100.stdout:2/462: creat d3/d1a/d6c/fa4 x:0 0 0 2024-03-18T14:51:05.981 INFO:tasks.workunit.client.1.smithi100.stdout:2/463: creat d3/da/de/fa5 x:0 0 0 2024-03-18T14:51:05.981 INFO:tasks.workunit.client.1.smithi100.stdout:2/464: chown d3/da/de/fa5 5659400 1 2024-03-18T14:51:05.981 INFO:tasks.workunit.client.1.smithi100.stdout:2/465: write d3/da/f33 [63786,123392] 0 2024-03-18T14:51:05.981 INFO:tasks.workunit.client.1.smithi100.stdout:2/466: write d3/da/de/fa5 [420409,128277] 0 2024-03-18T14:51:05.982 INFO:tasks.workunit.client.1.smithi100.stdout:8/519: truncate d4/ff 5808691 0 2024-03-18T14:51:05.982 INFO:tasks.workunit.client.1.smithi100.stdout:8/520: fsync d4/fa 0 2024-03-18T14:51:05.986 INFO:tasks.workunit.client.1.smithi100.stdout:3/523: rename d3/d16/d13/d17/d88 to d3/d16/d13/d71/db9 0 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/524: dread - d3/d16/d13/d17/f96 zero size 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/525: fsync d3/dd/f28 0 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/526: fdatasync d3/dd/f38 0 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.0.smithi012.stdout:0/852: creat d1/d20/d4e/f118 x:0 0 0 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.0.smithi012.stdout:0/853: creat d1/d20/d23/d7f/f119 x:0 0 0 2024-03-18T14:51:05.987 INFO:tasks.workunit.client.0.smithi012.stdout:0/854: write d1/d20/d23/d6f/d93/dba/fe1 [984755,352] 0 2024-03-18T14:51:05.991 INFO:tasks.workunit.client.0.smithi012.stdout:8/965: rename d0/d5a/d19/f101 to d0/d5a/d126/f136 0 2024-03-18T14:51:05.993 INFO:tasks.workunit.client.1.smithi100.stdout:2/467: creat d3/d13/fa6 x:0 0 0 2024-03-18T14:51:05.993 INFO:tasks.workunit.client.1.smithi100.stdout:2/468: write d3/d6/d5f/d6a/f99 [773549,15142] 0 2024-03-18T14:51:05.995 INFO:tasks.workunit.client.0.smithi012.stdout:4/740: dread d6/d19/f2f [0,4194304] 0 2024-03-18T14:51:05.995 INFO:tasks.workunit.client.0.smithi012.stdout:7/840: symlink dd/d13/d23/d2f/d7a/dab/dbc/l125 0 2024-03-18T14:51:05.999 INFO:tasks.workunit.client.0.smithi012.stdout:3/778: truncate dc/d30/d35/f2e 3127852 0 2024-03-18T14:51:06.003 INFO:tasks.workunit.client.1.smithi100.stdout:9/472: rename da/d2c/d2f/f30 to da/d46/d4d/d62/f93 0 2024-03-18T14:51:06.003 INFO:tasks.workunit.client.1.smithi100.stdout:9/473: chown da/db 861725042 1 2024-03-18T14:51:06.004 INFO:tasks.workunit.client.1.smithi100.stdout:9/474: dread - da/d51/d55/f5c zero size 2024-03-18T14:51:06.004 INFO:tasks.workunit.client.1.smithi100.stdout:9/475: stat da/c12 0 2024-03-18T14:51:06.004 INFO:tasks.workunit.client.1.smithi100.stdout:9/476: creat da/dd/f94 x:0 0 0 2024-03-18T14:51:06.004 INFO:tasks.workunit.client.0.smithi012.stdout:6/760: mknod d17/d19/ce0 0 2024-03-18T14:51:06.004 INFO:tasks.workunit.client.0.smithi012.stdout:6/761: read - d17/d38/fc7 zero size 2024-03-18T14:51:06.005 INFO:tasks.workunit.client.1.smithi100.stdout:2/469: truncate d3/da/de/d20/d82/d95/f70 189757 0 2024-03-18T14:51:06.006 INFO:tasks.workunit.client.0.smithi012.stdout:8/966: dread d0/d5a/d19/dad/f46 [0,4194304] 0 2024-03-18T14:51:06.006 INFO:tasks.workunit.client.0.smithi012.stdout:8/967: fdatasync d0/d5a/d19/d63/dbe/f113 0 2024-03-18T14:51:06.006 INFO:tasks.workunit.client.0.smithi012.stdout:9/874: getdents d3/de6/dbf/d66/d21/d35/d48 0 2024-03-18T14:51:06.006 INFO:tasks.workunit.client.0.smithi012.stdout:9/875: fdatasync d3/de6/dbf/d77/d52/d95/f11b 0 2024-03-18T14:51:06.009 INFO:tasks.workunit.client.0.smithi012.stdout:1/769: creat d8/db/d10/d56/dcc/ff9 x:0 0 0 2024-03-18T14:51:06.009 INFO:tasks.workunit.client.0.smithi012.stdout:1/770: write d8/db/d10/d31/d45/f6f [424058,65060] 0 2024-03-18T14:51:06.012 INFO:tasks.workunit.client.1.smithi100.stdout:7/483: rename d5/d22/d24/d39/c42 to d5/d22/d24/cb7 0 2024-03-18T14:51:06.012 INFO:tasks.workunit.client.1.smithi100.stdout:7/484: truncate d5/d14/d16/d2e/d7b/f65 482345 0 2024-03-18T14:51:06.015 INFO:tasks.workunit.client.1.smithi100.stdout:9/477: mknod da/dd/d29/d71/d72/c95 0 2024-03-18T14:51:06.018 INFO:tasks.workunit.client.1.smithi100.stdout:0/565: dwrite dd/d31/d91/d65/f74 [0,4194304] 0 2024-03-18T14:51:06.019 INFO:tasks.workunit.client.1.smithi100.stdout:7/485: unlink d5/d22/d3b/d47/c9e 0 2024-03-18T14:51:06.021 INFO:tasks.workunit.client.0.smithi012.stdout:6/762: dread d17/d38/d51/d72/f74 [0,4194304] 0 2024-03-18T14:51:06.021 INFO:tasks.workunit.client.0.smithi012.stdout:3/779: link dc/d30/fc4 dc/d3c/d4c/ff9 0 2024-03-18T14:51:06.021 INFO:tasks.workunit.client.0.smithi012.stdout:3/780: chown dc/d3c/d4c/d57/ca9 27498504 1 2024-03-18T14:51:06.021 INFO:tasks.workunit.client.0.smithi012.stdout:8/968: symlink d0/d47/l137 0 2024-03-18T14:51:06.023 INFO:tasks.workunit.client.0.smithi012.stdout:9/876: truncate d3/d9/d5c/f8f 2012884 0 2024-03-18T14:51:06.024 INFO:tasks.workunit.client.1.smithi100.stdout:0/566: dread dd/f1a [0,4194304] 0 2024-03-18T14:51:06.026 INFO:tasks.workunit.client.0.smithi012.stdout:3/781: unlink dc/d3c/c9d 0 2024-03-18T14:51:06.027 INFO:tasks.workunit.client.0.smithi012.stdout:3/782: write dc/d30/d3a/f84 [427336,34522] 0 2024-03-18T14:51:06.028 INFO:tasks.workunit.client.0.smithi012.stdout:8/969: creat d0/d5a/d5e/dd7/d84/d2a/d3e/d119/f138 x:0 0 0 2024-03-18T14:51:06.028 INFO:tasks.workunit.client.0.smithi012.stdout:8/970: chown d0/d15/l51 20659996 1 2024-03-18T14:51:06.028 INFO:tasks.workunit.client.0.smithi012.stdout:8/971: chown d0/d5a/d5e/dd7/d84/f6b 53326308 1 2024-03-18T14:51:06.030 INFO:tasks.workunit.client.1.smithi100.stdout:0/567: rename dd/d31/d91/db8 to dd/d31/d91/d56/d71/dcc/dcf 0 2024-03-18T14:51:06.032 INFO:tasks.workunit.client.0.smithi012.stdout:3/783: read dc/f22 [3305505,44924] 0 2024-03-18T14:51:06.034 INFO:tasks.workunit.client.0.smithi012.stdout:6/763: rmdir d17/d38/d51/d66/d7d 39 2024-03-18T14:51:06.034 INFO:tasks.workunit.client.0.smithi012.stdout:8/972: creat d0/d5a/d5e/f139 x:0 0 0 2024-03-18T14:51:06.037 INFO:tasks.workunit.client.0.smithi012.stdout:8/973: rename d0/d15/d4e to d0/d5a/d5e/dd7/d84/d54/d62/d13a 0 2024-03-18T14:51:06.037 INFO:tasks.workunit.client.0.smithi012.stdout:8/974: chown d0/d5a/d5e/ddf/d50/d114/l116 231 1 2024-03-18T14:51:06.038 INFO:tasks.workunit.client.0.smithi012.stdout:8/975: chown d0/d5a/d5e/ddf/f44 471 1 2024-03-18T14:51:06.039 INFO:tasks.workunit.client.0.smithi012.stdout:8/976: unlink d0/d47/d59/fca 0 2024-03-18T14:51:06.040 INFO:tasks.workunit.client.0.smithi012.stdout:8/977: truncate d0/d47/d59/dc9/f130 976914 0 2024-03-18T14:51:06.040 INFO:tasks.workunit.client.0.smithi012.stdout:8/978: chown d0/d5a/fcb 61524 1 2024-03-18T14:51:06.040 INFO:tasks.workunit.client.1.smithi100.stdout:5/445: dwrite f10 [4194304,4194304] 0 2024-03-18T14:51:06.041 INFO:tasks.workunit.client.0.smithi012.stdout:5/807: dwrite d3/d15/d5f/f70 [0,4194304] 0 2024-03-18T14:51:06.041 INFO:tasks.workunit.client.1.smithi100.stdout:8/521: dwrite d4/fa0 [0,4194304] 0 2024-03-18T14:51:06.042 INFO:tasks.workunit.client.0.smithi012.stdout:6/764: dread d17/d38/d51/dcc/dd8/fa5 [0,4194304] 0 2024-03-18T14:51:06.043 INFO:tasks.workunit.client.1.smithi100.stdout:5/446: link d29/d4a/f54 d29/f8c 0 2024-03-18T14:51:06.048 INFO:tasks.workunit.client.0.smithi012.stdout:6/765: creat d17/d19/d27/db1/dd7/fe1 x:0 0 0 2024-03-18T14:51:06.050 INFO:tasks.workunit.client.0.smithi012.stdout:6/766: symlink d17/d18/dc1/le2 0 2024-03-18T14:51:06.052 INFO:tasks.workunit.client.1.smithi100.stdout:8/522: mknod d4/dc/dd/de/d19/d2c/cb5 0 2024-03-18T14:51:06.055 INFO:tasks.workunit.client.0.smithi012.stdout:3/784: dread dc/d3c/d59/d86/d9a/f5d [0,4194304] 0 2024-03-18T14:51:06.056 INFO:tasks.workunit.client.1.smithi100.stdout:0/568: dread dd/d31/d91/d56/d71/f90 [0,4194304] 0 2024-03-18T14:51:06.056 INFO:tasks.workunit.client.1.smithi100.stdout:0/569: chown dd/d31/d91/d65 3686393 1 2024-03-18T14:51:06.057 INFO:tasks.workunit.client.1.smithi100.stdout:3/527: dwrite d3/d16/d13/d1f/d3f/d4f/f81 [0,4194304] 0 2024-03-18T14:51:06.057 INFO:tasks.workunit.client.1.smithi100.stdout:3/528: dread - d3/d16/d13/d71/f9a zero size 2024-03-18T14:51:06.060 INFO:tasks.workunit.client.1.smithi100.stdout:5/447: symlink d29/d2d/d39/d60/l8d 0 2024-03-18T14:51:06.061 INFO:tasks.workunit.client.1.smithi100.stdout:5/448: creat d29/d4a/f8e x:0 0 0 2024-03-18T14:51:06.061 INFO:tasks.workunit.client.0.smithi012.stdout:8/979: write d0/d5a/f9a [3144881,109140] 0 2024-03-18T14:51:06.061 INFO:tasks.workunit.client.0.smithi012.stdout:0/855: dwrite d1/d20/d23/d6f/f75 [0,4194304] 0 2024-03-18T14:51:06.066 INFO:tasks.workunit.client.1.smithi100.stdout:8/523: dread d4/dc/dd/de/d21/d23/f28 [0,4194304] 0 2024-03-18T14:51:06.066 INFO:tasks.workunit.client.1.smithi100.stdout:8/524: write d4/dc/dd/de/f93 [984108,83504] 0 2024-03-18T14:51:06.071 INFO:tasks.workunit.client.1.smithi100.stdout:2/470: dwrite d3/d6/f44 [4194304,4194304] 0 2024-03-18T14:51:06.071 INFO:tasks.workunit.client.1.smithi100.stdout:2/471: creat d3/d13/d40/fa7 x:0 0 0 2024-03-18T14:51:06.071 INFO:tasks.workunit.client.1.smithi100.stdout:2/472: truncate d3/da/de/d48/f8e 300605 0 2024-03-18T14:51:06.071 INFO:tasks.workunit.client.0.smithi012.stdout:1/771: dwrite d8/db/d10/f60 [0,4194304] 0 2024-03-18T14:51:06.074 INFO:tasks.workunit.client.1.smithi100.stdout:9/478: dwrite da/d2c/d2f/f4a [0,4194304] 0 2024-03-18T14:51:06.074 INFO:tasks.workunit.client.0.smithi012.stdout:9/877: dwrite d3/de6/dbf/d66/d69/d7f/d113/f117 [4194304,4194304] 0 2024-03-18T14:51:06.074 INFO:tasks.workunit.client.0.smithi012.stdout:9/878: write d3/de6/dbf/d77/da8/fb4 [100679,106946] 0 2024-03-18T14:51:06.075 INFO:tasks.workunit.client.1.smithi100.stdout:0/570: dread dd/d31/f57 [0,4194304] 0 2024-03-18T14:51:06.075 INFO:tasks.workunit.client.1.smithi100.stdout:0/571: readlink dd/d31/d91/d56/d59/lb3 0 2024-03-18T14:51:06.075 INFO:tasks.workunit.client.1.smithi100.stdout:0/572: fsync dd/fc9 0 2024-03-18T14:51:06.075 INFO:tasks.workunit.client.1.smithi100.stdout:7/486: dwrite d5/d22/d24/f8b [0,4194304] 0 2024-03-18T14:51:06.075 INFO:tasks.workunit.client.0.smithi012.stdout:4/741: dwrite d6/f8c [0,4194304] 0 2024-03-18T14:51:06.076 INFO:tasks.workunit.client.0.smithi012.stdout:4/742: readlink d6/d1a/d30/d46/l4d 0 2024-03-18T14:51:06.077 INFO:tasks.workunit.client.0.smithi012.stdout:5/808: dwrite d3/d29/d36/d9b/ddf/f104 [0,4194304] 0 2024-03-18T14:51:06.077 INFO:tasks.workunit.client.0.smithi012.stdout:4/743: dread d6/d1a/d30/d46/f48 [0,4194304] 0 2024-03-18T14:51:06.077 INFO:tasks.workunit.client.0.smithi012.stdout:4/744: creat d6/d19/d6c/d96/df0/ffb x:0 0 0 2024-03-18T14:51:06.077 INFO:tasks.workunit.client.0.smithi012.stdout:7/841: dwrite dd/d13/d23/d76/d6f/db3/fd0 [0,4194304] 0 2024-03-18T14:51:06.080 INFO:tasks.workunit.client.1.smithi100.stdout:7/487: dread d5/f72 [0,4194304] 0 2024-03-18T14:51:06.080 INFO:tasks.workunit.client.1.smithi100.stdout:7/488: chown d5/c9 715140 1 2024-03-18T14:51:06.089 INFO:tasks.workunit.client.1.smithi100.stdout:2/473: dwrite d3/d6/d29/d36/d49/f5b [0,4194304] 0 2024-03-18T14:51:06.092 INFO:tasks.workunit.client.0.smithi012.stdout:7/842: write dd/d13/d23/d76/d6f/db3/ff3 [151777,45123] 0 2024-03-18T14:51:06.111 INFO:tasks.workunit.client.0.smithi012.stdout:7/843: dwrite dd/f10 [0,4194304] 0 2024-03-18T14:51:06.123 INFO:tasks.workunit.client.0.smithi012.stdout:5/809: dwrite d3/d29/f7a [0,4194304] 0 2024-03-18T14:51:06.134 INFO:tasks.workunit.client.0.smithi012.stdout:6/767: getdents d17/d26/d36/d4d/d82/d97 0 2024-03-18T14:51:06.134 INFO:tasks.workunit.client.0.smithi012.stdout:6/768: creat d17/d26/d36/d4d/d82/fe3 x:0 0 0 2024-03-18T14:51:06.138 INFO:tasks.workunit.client.1.smithi100.stdout:3/529: creat d3/dd/d21/d3e/fba x:0 0 0 2024-03-18T14:51:06.147 INFO:tasks.workunit.client.0.smithi012.stdout:8/980: creat d0/d5a/d19/d57/daa/f13b x:0 0 0 2024-03-18T14:51:06.147 INFO:tasks.workunit.client.0.smithi012.stdout:8/981: write d0/d5a/d19/f93 [4528330,15854] 0 2024-03-18T14:51:06.151 INFO:tasks.workunit.client.0.smithi012.stdout:0/856: mknod d1/d20/d4e/d54/d67/dbc/c11a 0 2024-03-18T14:51:06.151 INFO:tasks.workunit.client.0.smithi012.stdout:0/857: truncate d1/d20/d23/d26/fa4 2066903 0 2024-03-18T14:51:06.152 INFO:tasks.workunit.client.1.smithi100.stdout:3/530: write d3/d16/d13/d1f/d3f/d4f/d60/f6b [3372380,46807] 0 2024-03-18T14:51:06.152 INFO:tasks.workunit.client.1.smithi100.stdout:3/531: creat d3/dd/d21/d3e/fbb x:0 0 0 2024-03-18T14:51:06.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/532: stat d3/d16/d13/d17/d7d/fa3 0 2024-03-18T14:51:06.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/533: creat d3/d16/d13/d17/dab/fbc x:0 0 0 2024-03-18T14:51:06.153 INFO:tasks.workunit.client.0.smithi012.stdout:0/858: read d1/f9 [934519,35747] 0 2024-03-18T14:51:06.153 INFO:tasks.workunit.client.0.smithi012.stdout:0/859: write d1/d16/d6e/dcc/f10f [964058,43282] 0 2024-03-18T14:51:06.153 INFO:tasks.workunit.client.0.smithi012.stdout:0/860: chown d1/d16/d31/cee 130 1 2024-03-18T14:51:06.154 INFO:tasks.workunit.client.0.smithi012.stdout:0/861: fsync d1/d20/d23/d88/fd2 0 2024-03-18T14:51:06.156 INFO:tasks.workunit.client.1.smithi100.stdout:3/534: dread d3/d16/d13/d1f/d3f/d4f/d60/f63 [0,4194304] 0 2024-03-18T14:51:06.170 INFO:tasks.workunit.client.1.smithi100.stdout:9/479: symlink da/d51/d55/d7e/l96 0 2024-03-18T14:51:06.179 INFO:tasks.workunit.client.1.smithi100.stdout:9/480: write da/d2c/d2f/f4a [1489010,18683] 0 2024-03-18T14:51:06.196 INFO:tasks.workunit.client.1.smithi100.stdout:0/573: unlink f3 0 2024-03-18T14:51:06.206 INFO:tasks.workunit.client.1.smithi100.stdout:7/489: mknod d5/d14/d16/d63/cb8 0 2024-03-18T14:51:06.220 INFO:tasks.workunit.client.1.smithi100.stdout:7/490: dwrite d5/d22/d24/d6e/f84 [0,4194304] 0 2024-03-18T14:51:06.221 INFO:tasks.workunit.client.1.smithi100.stdout:7/491: dread - d5/d14/d16/d2c/d86/d85/d87/f88 zero size 2024-03-18T14:51:06.260 INFO:tasks.workunit.client.0.smithi012.stdout:7/844: link dd/d13/d23/d2f/d53/dc6/df1/ff2 dd/d13/d23/d33/d37/f126 0 2024-03-18T14:51:06.287 INFO:tasks.workunit.client.1.smithi100.stdout:8/525: getdents d4/dc/d1f/d59 0 2024-03-18T14:51:06.287 INFO:tasks.workunit.client.1.smithi100.stdout:8/526: stat d4/dc/dd/f44 0 2024-03-18T14:51:06.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/481: mknod da/d46/d8b/c97 0 2024-03-18T14:51:06.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/482: write da/f16 [4361919,31977] 0 2024-03-18T14:51:06.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/483: creat da/dd/d29/d71/f98 x:0 0 0 2024-03-18T14:51:06.296 INFO:tasks.workunit.client.1.smithi100.stdout:9/484: chown da/dd/d3c/d6b 435232 1 2024-03-18T14:51:06.302 INFO:tasks.workunit.client.1.smithi100.stdout:2/474: rmdir d3/d1a/d6c 39 2024-03-18T14:51:06.305 INFO:tasks.workunit.client.1.smithi100.stdout:9/485: dread da/fc [0,4194304] 0 2024-03-18T14:51:06.308 INFO:tasks.workunit.client.1.smithi100.stdout:9/486: dread f9 [0,4194304] 0 2024-03-18T14:51:06.311 INFO:tasks.workunit.client.1.smithi100.stdout:8/527: creat d4/dc/dd/de/d19/d37/fb6 x:0 0 0 2024-03-18T14:51:06.311 INFO:tasks.workunit.client.1.smithi100.stdout:8/528: creat d4/dc/d31/d85/fb7 x:0 0 0 2024-03-18T14:51:06.311 INFO:tasks.workunit.client.1.smithi100.stdout:8/529: chown d4/dc/dd/de/f63 231 1 2024-03-18T14:51:06.311 INFO:tasks.workunit.client.1.smithi100.stdout:8/530: stat d4/dc/dd/de/d19/d37/f65 0 2024-03-18T14:51:06.312 INFO:tasks.workunit.client.1.smithi100.stdout:8/531: chown d4/f90 1075452 1 2024-03-18T14:51:06.319 INFO:tasks.workunit.client.1.smithi100.stdout:2/475: mknod d3/d1a/ca8 0 2024-03-18T14:51:06.321 INFO:tasks.workunit.client.1.smithi100.stdout:9/487: mknod da/dd/d3b/c99 0 2024-03-18T14:51:06.322 INFO:tasks.workunit.client.0.smithi012.stdout:5/810: fsync d3/d15/d42/dc6/d73/f92 0 2024-03-18T14:51:06.325 INFO:tasks.workunit.client.1.smithi100.stdout:8/532: dwrite d4/dc/dd/de/f93 [0,4194304] 0 2024-03-18T14:51:06.328 INFO:tasks.workunit.client.1.smithi100.stdout:9/488: symlink da/dd/d29/l9a 0 2024-03-18T14:51:06.328 INFO:tasks.workunit.client.1.smithi100.stdout:9/489: truncate da/d46/d8b/f3a 496224 0 2024-03-18T14:51:06.328 INFO:tasks.workunit.client.1.smithi100.stdout:9/490: write da/d46/d4d/d62/f93 [92296,90977] 0 2024-03-18T14:51:06.328 INFO:tasks.workunit.client.1.smithi100.stdout:8/533: dread d4/dc/dd/de/d19/f2f [0,4194304] 0 2024-03-18T14:51:06.335 INFO:tasks.workunit.client.1.smithi100.stdout:9/491: creat da/dd/d29/d71/d72/f9b x:0 0 0 2024-03-18T14:51:06.339 INFO:tasks.workunit.client.1.smithi100.stdout:9/492: creat da/dd/d3b/f9c x:0 0 0 2024-03-18T14:51:06.339 INFO:tasks.workunit.client.1.smithi100.stdout:9/493: readlink da/dd/d3c/l65 0 2024-03-18T14:51:06.343 INFO:tasks.workunit.client.0.smithi012.stdout:8/982: mknod d0/d5a/d5e/dd7/d84/d54/c13c 0 2024-03-18T14:51:06.344 INFO:tasks.workunit.client.0.smithi012.stdout:0/862: mknod d1/d20/d23/d26/c11b 0 2024-03-18T14:51:06.344 INFO:tasks.workunit.client.0.smithi012.stdout:0/863: write d1/d20/d23/d6f/d93/dba/fe1 [1719096,58791] 0 2024-03-18T14:51:06.351 INFO:tasks.workunit.client.0.smithi012.stdout:7/845: creat dd/d13/d23/d105/f127 x:0 0 0 2024-03-18T14:51:06.352 INFO:tasks.workunit.client.0.smithi012.stdout:9/879: rename d3/de6/dbf/d66/faa to d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/ddc/f11f 0 2024-03-18T14:51:06.353 INFO:tasks.workunit.client.1.smithi100.stdout:9/494: dwrite da/dd/f94 [0,4194304] 0 2024-03-18T14:51:06.353 INFO:tasks.workunit.client.0.smithi012.stdout:5/811: creat d3/d84/dfa/f10c x:0 0 0 2024-03-18T14:51:06.354 INFO:tasks.workunit.client.0.smithi012.stdout:9/880: write d3/d9/f17 [102453,84528] 0 2024-03-18T14:51:06.354 INFO:tasks.workunit.client.0.smithi012.stdout:6/769: symlink d17/d38/le4 0 2024-03-18T14:51:06.356 INFO:tasks.workunit.client.0.smithi012.stdout:8/983: truncate d0/d5a/d19/fe5 3393898 0 2024-03-18T14:51:06.358 INFO:tasks.workunit.client.0.smithi012.stdout:0/864: rmdir d1 39 2024-03-18T14:51:06.358 INFO:tasks.workunit.client.0.smithi012.stdout:0/865: creat d1/d20/d4e/d54/d67/f11c x:0 0 0 2024-03-18T14:51:06.360 INFO:tasks.workunit.client.0.smithi012.stdout:7/846: truncate dd/d13/d23/d2f/d7a/dab/fbe 3202409 0 2024-03-18T14:51:06.362 INFO:tasks.workunit.client.0.smithi012.stdout:4/745: creat d6/da/ffc x:0 0 0 2024-03-18T14:51:06.363 INFO:tasks.workunit.client.0.smithi012.stdout:5/812: link d3/d15/d42/dc6/f5c d3/d29/d9d/f10d 0 2024-03-18T14:51:06.364 INFO:tasks.workunit.client.0.smithi012.stdout:5/813: chown d3/d15/d42/dc6/d73 4749926 1 2024-03-18T14:51:06.365 INFO:tasks.workunit.client.0.smithi012.stdout:9/881: mknod d3/d9/d5c/dbd/c120 0 2024-03-18T14:51:06.366 INFO:tasks.workunit.client.0.smithi012.stdout:6/770: symlink d17/d38/d51/dcc/dd8/le5 0 2024-03-18T14:51:06.366 INFO:tasks.workunit.client.0.smithi012.stdout:6/771: chown f11 472 1 2024-03-18T14:51:06.366 INFO:tasks.workunit.client.0.smithi012.stdout:6/772: write d17/d26/d36/d4d/d82/dcd/fda [753896,110940] 0 2024-03-18T14:51:06.368 INFO:tasks.workunit.client.1.smithi100.stdout:9/495: dwrite da/dd/d3c/d4b/f58 [0,4194304] 0 2024-03-18T14:51:06.370 INFO:tasks.workunit.client.0.smithi012.stdout:7/847: dread fa [4194304,4194304] 0 2024-03-18T14:51:06.370 INFO:tasks.workunit.client.0.smithi012.stdout:7/848: write dd/d13/d23/d33/dcc/f115 [128080,35306] 0 2024-03-18T14:51:06.370 INFO:tasks.workunit.client.0.smithi012.stdout:8/984: creat d0/d15/f13d x:0 0 0 2024-03-18T14:51:06.373 INFO:tasks.workunit.client.1.smithi100.stdout:8/534: rmdir d4/dc/d1f/d59 39 2024-03-18T14:51:06.373 INFO:tasks.workunit.client.0.smithi012.stdout:0/866: truncate d1/d20/d23/d3b/dc1/fdd 2367227 0 2024-03-18T14:51:06.374 INFO:tasks.workunit.client.0.smithi012.stdout:0/867: chown d1/d20/d4e/d54/d8a/fb9 240148697 1 2024-03-18T14:51:06.378 INFO:tasks.workunit.client.1.smithi100.stdout:9/496: write da/dd/d29/f34 [1388781,130987] 0 2024-03-18T14:51:06.378 INFO:tasks.workunit.client.1.smithi100.stdout:9/497: truncate da/d46/d8b/f3a 1351901 0 2024-03-18T14:51:06.380 INFO:tasks.workunit.client.1.smithi100.stdout:8/535: mknod d4/cb8 0 2024-03-18T14:51:06.380 INFO:tasks.workunit.client.1.smithi100.stdout:8/536: creat d4/dc/dd/de/d21/d23/d47/d54/fb9 x:0 0 0 2024-03-18T14:51:06.382 INFO:tasks.workunit.client.0.smithi012.stdout:9/882: mknod d3/de6/dbf/c121 0 2024-03-18T14:51:06.383 INFO:tasks.workunit.client.0.smithi012.stdout:6/773: symlink d17/d26/d36/d4d/d82/d97/le6 0 2024-03-18T14:51:06.384 INFO:tasks.workunit.client.0.smithi012.stdout:6/774: dread d17/d26/fbc [0,4194304] 0 2024-03-18T14:51:06.390 INFO:tasks.workunit.client.0.smithi012.stdout:8/985: mknod d0/d5a/d5e/dd7/d122/c13e 0 2024-03-18T14:51:06.394 INFO:tasks.workunit.client.1.smithi100.stdout:9/498: dread da/f16 [0,4194304] 0 2024-03-18T14:51:06.394 INFO:tasks.workunit.client.1.smithi100.stdout:9/499: write da/dd/d29/f34 [5145079,92158] 0 2024-03-18T14:51:06.394 INFO:tasks.workunit.client.1.smithi100.stdout:9/500: chown da/dd/d29/d71/d72/c95 524848 1 2024-03-18T14:51:06.394 INFO:tasks.workunit.client.0.smithi012.stdout:6/775: mknod d17/d38/d51/dcc/ce7 0 2024-03-18T14:51:06.395 INFO:tasks.workunit.client.0.smithi012.stdout:7/849: rmdir dd/d13/d23/d60/d9c 39 2024-03-18T14:51:06.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/986: symlink d0/d5a/d5e/dd7/d84/d2a/d3e/l13f 0 2024-03-18T14:51:06.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/987: chown d0/d5a/d19/d63/l7a 6 1 2024-03-18T14:51:06.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/988: fdatasync d0/d5a/d5e/ddf/d50/fb3 0 2024-03-18T14:51:06.398 INFO:tasks.workunit.client.0.smithi012.stdout:8/989: write d0/d47/d59/dc9/fe6 [772974,58231] 0 2024-03-18T14:51:06.399 INFO:tasks.workunit.client.0.smithi012.stdout:6/776: truncate d17/d38/d51/d66/d7d/fd9 740554 0 2024-03-18T14:51:06.402 INFO:tasks.workunit.client.0.smithi012.stdout:9/883: dwrite d3/de6/dbf/d77/d52/fff [0,4194304] 0 2024-03-18T14:51:06.409 INFO:tasks.workunit.client.0.smithi012.stdout:8/990: getdents d0/d5a/d5e/ddf/d50/d114 0 2024-03-18T14:51:06.411 INFO:tasks.workunit.client.0.smithi012.stdout:8/991: creat d0/d5a/d5e/dd7/d84/d54/d62/d13a/f140 x:0 0 0 2024-03-18T14:51:06.412 INFO:tasks.workunit.client.0.smithi012.stdout:8/992: symlink d0/d5a/d19/d57/d9e/l141 0 2024-03-18T14:51:06.426 INFO:tasks.workunit.client.0.smithi012.stdout:4/746: rename d6/cc to d6/da/d89/d94/da2/cfd 0 2024-03-18T14:51:06.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/777: rename d17/d38/d51/d63 to d17/d19/d27/de8 0 2024-03-18T14:51:06.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/778: stat d17/d26/d36/d4d/d82/d97/fd4 0 2024-03-18T14:51:06.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/779: fsync d17/d19/d27/f54 0 2024-03-18T14:51:06.428 INFO:tasks.workunit.client.0.smithi012.stdout:6/780: chown d17/d19/d27 662783 1 2024-03-18T14:51:06.429 INFO:tasks.workunit.client.0.smithi012.stdout:8/993: dwrite d0/d5a/d5e/f139 [0,4194304] 0 2024-03-18T14:51:06.436 INFO:tasks.workunit.client.0.smithi012.stdout:0/868: mknod d1/d20/d23/d7f/c11d 0 2024-03-18T14:51:06.437 INFO:tasks.workunit.client.0.smithi012.stdout:0/869: symlink d1/d2d/l11e 0 2024-03-18T14:51:06.438 INFO:tasks.workunit.client.0.smithi012.stdout:6/781: write d17/d19/f20 [2609115,1837] 0 2024-03-18T14:51:06.439 INFO:tasks.workunit.client.0.smithi012.stdout:8/994: write d0/d5a/d5e/dd7/d84/fc6 [1064852,128737] 0 2024-03-18T14:51:06.439 INFO:tasks.workunit.client.0.smithi012.stdout:8/995: stat d0/d5a/f135 0 2024-03-18T14:51:06.439 INFO:tasks.workunit.client.0.smithi012.stdout:0/870: link d1/d16/d31/cac d1/d20/d4e/d54/c11f 0 2024-03-18T14:51:06.441 INFO:tasks.workunit.client.0.smithi012.stdout:6/782: link d17/d38/d51/dcc/dd8/cae d17/d19/d27/d58/ce9 0 2024-03-18T14:51:06.445 INFO:tasks.workunit.client.0.smithi012.stdout:8/996: unlink d0/d5a/d5e/dd7/d84/d54/ff6 0 2024-03-18T14:51:06.450 INFO:tasks.workunit.client.0.smithi012.stdout:8/997: dread d0/d5a/d19/dad/f4b [4194304,4194304] 0 2024-03-18T14:51:06.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/783: dwrite d17/d38/d51/dcc/dd8/f88 [0,4194304] 0 2024-03-18T14:51:06.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/784: write d17/d38/d51/fbf [435484,118441] 0 2024-03-18T14:51:06.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/785: fsync d17/d38/fc7 0 2024-03-18T14:51:06.471 INFO:tasks.workunit.client.1.smithi100.stdout:0/574: rename dd/d2b/f3d to dd/d2b/d95/d79/d7d/fd0 0 2024-03-18T14:51:06.472 INFO:tasks.workunit.client.1.smithi100.stdout:7/492: rename d5/l28 to d5/d14/d16/d2e/d62/lb9 0 2024-03-18T14:51:06.472 INFO:tasks.workunit.client.1.smithi100.stdout:7/493: truncate d5/d14/d16/d2e/d62/f8c 216237 0 2024-03-18T14:51:06.473 INFO:tasks.workunit.client.1.smithi100.stdout:0/575: unlink dd/d20/ca1 0 2024-03-18T14:51:06.473 INFO:tasks.workunit.client.1.smithi100.stdout:0/576: fsync dd/f42 0 2024-03-18T14:51:06.476 INFO:tasks.workunit.client.1.smithi100.stdout:0/577: unlink dd/d31/d91/d65/f74 0 2024-03-18T14:51:06.476 INFO:tasks.workunit.client.1.smithi100.stdout:0/578: getdents dd/d20/d44/dce 0 2024-03-18T14:51:06.478 INFO:tasks.workunit.client.1.smithi100.stdout:5/449: mkdir d29/d2d/d8f 0 2024-03-18T14:51:06.478 INFO:tasks.workunit.client.1.smithi100.stdout:5/450: symlink d29/d2d/d32/l90 0 2024-03-18T14:51:06.479 INFO:tasks.workunit.client.1.smithi100.stdout:5/451: chown f26 751 1 2024-03-18T14:51:06.481 INFO:tasks.workunit.client.1.smithi100.stdout:5/452: creat d29/f91 x:0 0 0 2024-03-18T14:51:06.481 INFO:tasks.workunit.client.1.smithi100.stdout:5/453: chown f15 1062 1 2024-03-18T14:51:06.482 INFO:tasks.workunit.client.1.smithi100.stdout:7/494: write d5/d22/d24/f8b [1289416,79982] 0 2024-03-18T14:51:06.482 INFO:tasks.workunit.client.1.smithi100.stdout:7/495: fsync d5/d14/d16/d2e/d62/f97 0 2024-03-18T14:51:06.483 INFO:tasks.workunit.client.1.smithi100.stdout:7/496: readlink d5/d14/d16/d44/l70 0 2024-03-18T14:51:06.484 INFO:tasks.workunit.client.1.smithi100.stdout:7/497: truncate d5/d22/d24/d6e/f84 2837762 0 2024-03-18T14:51:06.486 INFO:tasks.workunit.client.1.smithi100.stdout:7/498: creat d5/d14/d16/d2c/fba x:0 0 0 2024-03-18T14:51:06.498 INFO:tasks.workunit.client.0.smithi012.stdout:0/871: dread d1/d16/d6e/f9b [0,4194304] 0 2024-03-18T14:51:06.501 INFO:tasks.workunit.client.0.smithi012.stdout:0/872: rename d1/d20/d4e/d54/fe6 to d1/d20/d23/f120 0 2024-03-18T14:51:06.501 INFO:tasks.workunit.client.0.smithi012.stdout:0/873: write d1/d2d/f62 [1802985,88056] 0 2024-03-18T14:51:06.502 INFO:tasks.workunit.client.0.smithi012.stdout:8/998: dread d0/fdd [0,4194304] 0 2024-03-18T14:51:06.506 INFO:tasks.workunit.client.0.smithi012.stdout:8/999: link d0/d5a/d19/d57/daa/f13b d0/d5a/d19/d63/f142 0 2024-03-18T14:51:06.507 INFO:tasks.workunit.client.0.smithi012.stdout:7/850: dwrite dd/d13/d23/d60/d9c/fd7 [0,4194304] 0 2024-03-18T14:51:06.508 INFO:tasks.workunit.client.1.smithi100.stdout:0/579: dwrite dd/d31/d91/d56/d71/fa3 [0,4194304] 0 2024-03-18T14:51:06.508 INFO:tasks.workunit.client.1.smithi100.stdout:0/580: truncate dd/fb1 1188680 0 2024-03-18T14:51:06.508 INFO:tasks.workunit.client.1.smithi100.stdout:0/581: fsync dd/d2b/d95/d79/fa4 0 2024-03-18T14:51:06.509 INFO:tasks.workunit.client.0.smithi012.stdout:9/884: dwrite d3/de6/dbf/d66/d21/f90 [0,4194304] 0 2024-03-18T14:51:06.509 INFO:tasks.workunit.client.0.smithi012.stdout:6/786: dwrite d17/d38/d51/fac [0,4194304] 0 2024-03-18T14:51:06.509 INFO:tasks.workunit.client.0.smithi012.stdout:9/885: chown d3/de6/dbf/d66/d21/d35/l68 408885 1 2024-03-18T14:51:06.512 INFO:tasks.workunit.client.0.smithi012.stdout:7/851: symlink dd/d13/d23/d33/dd5/l128 0 2024-03-18T14:51:06.512 INFO:tasks.workunit.client.0.smithi012.stdout:7/852: chown dd/d13/d23/d76 58 1 2024-03-18T14:51:06.513 INFO:tasks.workunit.client.1.smithi100.stdout:0/582: link dd/d31/d91/d56/c68 dd/d31/d91/d56/d71/cd1 0 2024-03-18T14:51:06.516 INFO:tasks.workunit.client.1.smithi100.stdout:2/476: rename d3/d6/d29/d36/f5e to d3/d6/fa9 0 2024-03-18T14:51:06.516 INFO:tasks.workunit.client.1.smithi100.stdout:2/477: stat d3/d6/l7f 0 2024-03-18T14:51:06.516 INFO:tasks.workunit.client.1.smithi100.stdout:2/478: chown d3/f4 278 1 2024-03-18T14:51:06.519 INFO:tasks.workunit.client.1.smithi100.stdout:7/499: rename d5/d14/d16/d2c/d86/d85/d87/l8d to d5/d30/lbb 0 2024-03-18T14:51:06.520 INFO:tasks.workunit.client.0.smithi012.stdout:7/853: dread dd/d13/d23/d2f/f9f [0,4194304] 0 2024-03-18T14:51:06.520 INFO:tasks.workunit.client.0.smithi012.stdout:7/854: chown dd/d13/d23/d2f/d53/ff7 289363 1 2024-03-18T14:51:06.521 INFO:tasks.workunit.client.1.smithi100.stdout:2/479: creat d3/da/de/d20/d82/d95/faa x:0 0 0 2024-03-18T14:51:06.602 INFO:tasks.workunit.client.0.smithi012.stdout:4/747: link d6/da/d58/f9e d6/d19/ffe 0 2024-03-18T14:51:06.604 INFO:tasks.workunit.client.0.smithi012.stdout:1/772: link d8/db/d10/d16/da1/d2e/fd4 d8/d1e/d27/ffa 0 2024-03-18T14:51:06.605 INFO:tasks.workunit.client.1.smithi100.stdout:7/500: symlink d5/d22/d3b/d9f/lbc 0 2024-03-18T14:51:06.605 INFO:tasks.workunit.client.1.smithi100.stdout:7/501: chown d5/d14/d16/d44 120349 1 2024-03-18T14:51:06.669 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:06 smithi012 ceph-mon[29182]: pgmap v7: 65 pgs: 65 active+clean; 2.4 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 203 MiB/s rd, 218 MiB/s wr, 356 op/s 2024-03-18T14:51:06.669 INFO:tasks.workunit.client.1.smithi100.stdout:7/502: mknod d5/d90/cbd 0 2024-03-18T14:51:06.670 INFO:tasks.workunit.client.0.smithi012.stdout:6/787: truncate d17/d26/f28 2721912 0 2024-03-18T14:51:06.672 INFO:tasks.workunit.client.1.smithi100.stdout:2/480: rmdir d3/d13/d40/d53 39 2024-03-18T14:51:06.674 INFO:tasks.workunit.client.0.smithi012.stdout:9/886: link d3/de6/dbf/d66/d69/d7f/f92 d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/ddc/f122 0 2024-03-18T14:51:06.675 INFO:tasks.workunit.client.0.smithi012.stdout:9/887: write d3/de6/dbf/d66/d21/d35/fb9 [1201660,75950] 0 2024-03-18T14:51:06.677 INFO:tasks.workunit.client.1.smithi100.stdout:7/503: dread d5/d14/d16/d2e/d62/d66/f54 [0,4194304] 0 2024-03-18T14:51:06.677 INFO:tasks.workunit.client.1.smithi100.stdout:7/504: chown d5/d22/d24/d6e/caa 0 1 2024-03-18T14:51:06.704 INFO:tasks.workunit.client.1.smithi100.stdout:2/481: rename d3/f4 to d3/da/de/fab 0 2024-03-18T14:51:06.708 INFO:tasks.workunit.client.0.smithi012.stdout:5/814: symlink d3/d15/d42/l10e 0 2024-03-18T14:51:06.709 INFO:tasks.workunit.client.0.smithi012.stdout:7/855: unlink dd/d4f/ca3 0 2024-03-18T14:51:06.710 INFO:tasks.workunit.client.0.smithi012.stdout:4/748: creat d6/da/d89/d97/dc5/fff x:0 0 0 2024-03-18T14:51:06.712 INFO:tasks.workunit.client.0.smithi012.stdout:6/788: dwrite d17/d19/d27/d58/f77 [0,4194304] 0 2024-03-18T14:51:06.713 INFO:tasks.workunit.client.0.smithi012.stdout:1/773: creat d8/d1e/ffb x:0 0 0 2024-03-18T14:51:06.716 INFO:tasks.workunit.client.1.smithi100.stdout:2/482: mkdir d3/d13/d40/dac 0 2024-03-18T14:51:06.716 INFO:tasks.workunit.client.1.smithi100.stdout:2/483: readlink d3/l52 0 2024-03-18T14:51:06.717 INFO:tasks.workunit.client.1.smithi100.stdout:4/479: sync 2024-03-18T14:51:06.717 INFO:tasks.workunit.client.1.smithi100.stdout:3/535: sync 2024-03-18T14:51:06.717 INFO:tasks.workunit.client.1.smithi100.stdout:1/485: sync 2024-03-18T14:51:06.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/500: sync 2024-03-18T14:51:06.717 INFO:tasks.workunit.client.1.smithi100.stdout:4/480: chown d9/d27/d42/f84 160 1 2024-03-18T14:51:06.718 INFO:tasks.workunit.client.1.smithi100.stdout:4/481: fsync d9/d13/d25/f4d 0 2024-03-18T14:51:06.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/505: mknod d5/d14/d16/d2c/cbe 0 2024-03-18T14:51:06.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/506: fsync d5/d30/f5d 0 2024-03-18T14:51:06.718 INFO:tasks.workunit.client.1.smithi100.stdout:7/507: chown d5/d14/d16/d2c/d86/l61 3424 1 2024-03-18T14:51:06.719 INFO:tasks.workunit.client.0.smithi012.stdout:4/749: dread d6/f12 [0,4194304] 0 2024-03-18T14:51:06.721 INFO:tasks.workunit.client.0.smithi012.stdout:9/888: rename d3/de6/dbf/d77/d52/fff to d3/de6/dbf/d77/d52/d99/da0/ddf/f123 0 2024-03-18T14:51:06.722 INFO:tasks.workunit.client.1.smithi100.stdout:2/484: mkdir d3/da/de/d20/d82/dad 0 2024-03-18T14:51:06.724 INFO:tasks.workunit.client.0.smithi012.stdout:6/789: write d17/d19/d27/d58/d91/fc3 [1828945,44435] 0 2024-03-18T14:51:06.724 INFO:tasks.workunit.client.0.smithi012.stdout:6/790: chown d17/d38/d51/dcc/dd8/fa5 6 1 2024-03-18T14:51:06.724 INFO:tasks.workunit.client.0.smithi012.stdout:5/815: creat d3/dad/df2/f10f x:0 0 0 2024-03-18T14:51:06.726 INFO:tasks.workunit.client.1.smithi100.stdout:6/501: dread d6/d7/df/d72/f33 [0,4194304] 0 2024-03-18T14:51:06.727 INFO:tasks.workunit.client.1.smithi100.stdout:4/482: creat d9/d27/d62/d7b/d82/f90 x:0 0 0 2024-03-18T14:51:06.727 INFO:tasks.workunit.client.1.smithi100.stdout:4/483: write d9/f1f [1434395,53752] 0 2024-03-18T14:51:06.728 INFO:tasks.workunit.client.0.smithi012.stdout:1/774: stat d8/db/l42 0 2024-03-18T14:51:06.728 INFO:tasks.workunit.client.0.smithi012.stdout:1/775: chown d8/d1e/l88 18257 1 2024-03-18T14:51:06.729 INFO:tasks.workunit.client.1.smithi100.stdout:3/536: write d3/dd/d49/f8f [1232899,115200] 0 2024-03-18T14:51:06.738 INFO:tasks.workunit.client.0.smithi012.stdout:9/889: link d3/de6/dbf/d66/d69/d7f/d8c/d8b/f11c d3/de6/dbf/d66/d69/f124 0 2024-03-18T14:51:06.738 INFO:tasks.workunit.client.0.smithi012.stdout:9/890: chown d3/de6/dbf/lcc 1 1 2024-03-18T14:51:06.739 INFO:tasks.workunit.client.1.smithi100.stdout:2/485: write d3/da/de/f28 [1177444,30173] 0 2024-03-18T14:51:06.740 INFO:tasks.workunit.client.1.smithi100.stdout:2/486: write d3/d6/f98 [855415,2006] 0 2024-03-18T14:51:06.742 INFO:tasks.workunit.client.1.smithi100.stdout:3/537: rename d3/l4b to d3/d16/d13/d71/db9/lbd 0 2024-03-18T14:51:06.744 INFO:tasks.workunit.client.1.smithi100.stdout:7/508: getdents d5/d14 0 2024-03-18T14:51:06.746 INFO:tasks.workunit.client.0.smithi012.stdout:5/816: symlink d3/d1e/l110 0 2024-03-18T14:51:06.746 INFO:tasks.workunit.client.0.smithi012.stdout:5/817: stat d3/d15/d42/dc6/d73/d6b/df3/f7c 0 2024-03-18T14:51:06.748 INFO:tasks.workunit.client.1.smithi100.stdout:3/538: symlink d3/d16/d13/d17/d7d/lbe 0 2024-03-18T14:51:06.748 INFO:tasks.workunit.client.1.smithi100.stdout:1/486: dwrite d8/f7e [0,4194304] 0 2024-03-18T14:51:06.750 INFO:tasks.workunit.client.0.smithi012.stdout:7/856: rename dd/d13/d23/d33 to dd/d13/d23/d76/d109/d129 0 2024-03-18T14:51:06.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/509: creat d5/d14/d16/d2e/d62/d66/fbf x:0 0 0 2024-03-18T14:51:06.753 INFO:tasks.workunit.client.0.smithi012.stdout:9/891: dread d3/de6/f31 [0,4194304] 0 2024-03-18T14:51:06.757 INFO:tasks.workunit.client.1.smithi100.stdout:3/539: mknod d3/dd/cbf 0 2024-03-18T14:51:06.757 INFO:tasks.workunit.client.1.smithi100.stdout:3/540: fdatasync d3/dd/d21/d3e/fba 0 2024-03-18T14:51:06.757 INFO:tasks.workunit.client.1.smithi100.stdout:3/541: creat d3/dd/d21/d3e/fc0 x:0 0 0 2024-03-18T14:51:06.758 INFO:tasks.workunit.client.0.smithi012.stdout:5/818: link d3/d84/f9f d3/d29/d36/d9b/ddf/def/f111 0 2024-03-18T14:51:06.762 INFO:tasks.workunit.client.0.smithi012.stdout:7/857: creat dd/d13/d23/d76/d6f/dd9/ddd/f12a x:0 0 0 2024-03-18T14:51:06.763 INFO:tasks.workunit.client.1.smithi100.stdout:3/542: dread d3/dd/d21/d3e/f79 [4194304,4194304] 0 2024-03-18T14:51:06.763 INFO:tasks.workunit.client.1.smithi100.stdout:3/543: write d3/dd/d21/d3e/fbb [409190,78502] 0 2024-03-18T14:51:06.764 INFO:tasks.workunit.client.0.smithi012.stdout:4/750: dwrite d6/da/f3b [0,4194304] 0 2024-03-18T14:51:06.764 INFO:tasks.workunit.client.1.smithi100.stdout:2/487: read d3/d13/d40/d53/f66 [1998641,125428] 0 2024-03-18T14:51:06.767 INFO:tasks.workunit.client.0.smithi012.stdout:9/892: rmdir d3/de6/dbf/d66/d69/d7f/d8c/dac 39 2024-03-18T14:51:06.767 INFO:tasks.workunit.client.0.smithi012.stdout:9/893: fdatasync d3/de6/dbf/f4d 0 2024-03-18T14:51:06.767 INFO:tasks.workunit.client.0.smithi012.stdout:9/894: write d3/d9/d5c/f85 [4293216,22153] 0 2024-03-18T14:51:06.770 INFO:tasks.workunit.client.1.smithi100.stdout:2/488: rename d3/da/de/d48/l5c to d3/d1a/d76/lae 0 2024-03-18T14:51:06.770 INFO:tasks.workunit.client.0.smithi012.stdout:4/751: dread d6/d1a/d30/d46/d6f/d7c/f81 [0,4194304] 0 2024-03-18T14:51:06.773 INFO:tasks.workunit.client.0.smithi012.stdout:5/819: getdents d3/d29/d9d 0 2024-03-18T14:51:06.777 INFO:tasks.workunit.client.0.smithi012.stdout:9/895: mkdir d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/ddc/d125 0 2024-03-18T14:51:06.778 INFO:tasks.workunit.client.0.smithi012.stdout:2/853: sync 2024-03-18T14:51:06.778 INFO:tasks.workunit.client.0.smithi012.stdout:3/785: sync 2024-03-18T14:51:06.779 INFO:tasks.workunit.client.1.smithi100.stdout:6/502: dwrite d6/d7/df/d26/d4e/d66/d36/f6e [0,4194304] 0 2024-03-18T14:51:06.780 INFO:tasks.workunit.client.0.smithi012.stdout:4/752: mknod d6/d1a/d30/d46/d6f/d8b/c100 0 2024-03-18T14:51:06.780 INFO:tasks.workunit.client.0.smithi012.stdout:4/753: readlink d6/d1a/d64/lae 0 2024-03-18T14:51:06.780 INFO:tasks.workunit.client.0.smithi012.stdout:4/754: creat d6/d19/d6c/f101 x:0 0 0 2024-03-18T14:51:06.780 INFO:tasks.workunit.client.0.smithi012.stdout:1/776: dwrite d8/db/d10/d31/d43/d5e/d65/d70/fcb [0,4194304] 0 2024-03-18T14:51:06.780 INFO:tasks.workunit.client.1.smithi100.stdout:4/484: dwrite d9/d13/d39/d4c/f52 [0,4194304] 0 2024-03-18T14:51:06.781 INFO:tasks.workunit.client.1.smithi100.stdout:4/485: dread d9/d3a/f86 [0,4194304] 0 2024-03-18T14:51:06.783 INFO:tasks.workunit.client.0.smithi012.stdout:9/896: symlink d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/d10b/l126 0 2024-03-18T14:51:06.783 INFO:tasks.workunit.client.0.smithi012.stdout:9/897: dread - d3/ff8 zero size 2024-03-18T14:51:06.783 INFO:tasks.workunit.client.1.smithi100.stdout:7/510: dwrite d5/d22/d24/f2d [0,4194304] 0 2024-03-18T14:51:06.783 INFO:tasks.workunit.client.1.smithi100.stdout:1/487: dwrite d8/d2a/f6b [4194304,4194304] 0 2024-03-18T14:51:06.784 INFO:tasks.workunit.client.0.smithi012.stdout:7/858: dwrite dd/d13/d23/d105/f127 [0,4194304] 0 2024-03-18T14:51:06.785 INFO:tasks.workunit.client.1.smithi100.stdout:3/544: dread d3/dd/d21/d3e/fbb [0,4194304] 0 2024-03-18T14:51:06.785 INFO:tasks.workunit.client.0.smithi012.stdout:3/786: unlink dc/d30/d3a/da0/cf8 0 2024-03-18T14:51:06.793 INFO:tasks.workunit.client.1.smithi100.stdout:4/486: dread d9/d13/d25/d51/f61 [4194304,4194304] 0 2024-03-18T14:51:06.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:06 smithi100 ceph-mon[37807]: pgmap v7: 65 pgs: 65 active+clean; 2.4 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 203 MiB/s rd, 218 MiB/s wr, 356 op/s 2024-03-18T14:51:06.795 INFO:tasks.workunit.client.0.smithi012.stdout:2/854: creat d0/d3c/d40/f112 x:0 0 0 2024-03-18T14:51:06.795 INFO:tasks.workunit.client.1.smithi100.stdout:8/537: sync 2024-03-18T14:51:06.795 INFO:tasks.workunit.client.1.smithi100.stdout:0/583: sync 2024-03-18T14:51:06.795 INFO:tasks.workunit.client.1.smithi100.stdout:5/454: sync 2024-03-18T14:51:06.795 INFO:tasks.workunit.client.1.smithi100.stdout:9/501: sync 2024-03-18T14:51:06.796 INFO:tasks.workunit.client.1.smithi100.stdout:9/502: fdatasync da/d46/d8b/f37 0 2024-03-18T14:51:06.800 INFO:tasks.workunit.client.0.smithi012.stdout:4/755: symlink d6/d19/d39/l102 0 2024-03-18T14:51:06.800 INFO:tasks.workunit.client.0.smithi012.stdout:4/756: truncate d6/d19/d6c/d96/fc7 391428 0 2024-03-18T14:51:06.806 INFO:tasks.workunit.client.1.smithi100.stdout:6/503: dwrite d6/d7/f8d [0,4194304] 0 2024-03-18T14:51:06.806 INFO:tasks.workunit.client.1.smithi100.stdout:6/504: chown d6/d7/df/d26/f84 6914 1 2024-03-18T14:51:06.807 INFO:tasks.workunit.client.1.smithi100.stdout:4/487: dread d9/d17/f45 [0,4194304] 0 2024-03-18T14:51:06.807 INFO:tasks.workunit.client.1.smithi100.stdout:4/488: write d9/fe [945640,53994] 0 2024-03-18T14:51:06.808 INFO:tasks.workunit.client.0.smithi012.stdout:1/777: symlink d8/db/d10/d31/d43/d5e/d65/d70/lfc 0 2024-03-18T14:51:06.808 INFO:tasks.workunit.client.0.smithi012.stdout:1/778: readlink d8/db/l42 0 2024-03-18T14:51:06.808 INFO:tasks.workunit.client.0.smithi012.stdout:3/787: dwrite dc/d30/d3a/db7/dd3/fe9 [0,4194304] 0 2024-03-18T14:51:06.814 INFO:tasks.workunit.client.0.smithi012.stdout:5/820: rename d3/d1e/d33/ce7 to d3/d15/d42/dc6/d73/c112 0 2024-03-18T14:51:06.817 INFO:tasks.workunit.client.0.smithi012.stdout:9/898: getdents d3/de6/dbf/d77/da8 0 2024-03-18T14:51:06.817 INFO:tasks.workunit.client.0.smithi012.stdout:9/899: fdatasync d3/de6/dbf/d66/d69/d7f/d8c/dac/dd1/ddc/f118 0 2024-03-18T14:51:06.819 INFO:tasks.workunit.client.1.smithi100.stdout:1/488: mknod d8/d1e/d11/d27/d83/c8e 0 2024-03-18T14:51:06.824 INFO:tasks.workunit.client.0.smithi012.stdout:5/821: dread d3/d29/d36/d9b/ddf/f104 [0,4194304] 0 2024-03-18T14:51:06.824 INFO:tasks.workunit.client.0.smithi012.stdout:5/822: write d3/d15/d42/dc6/d73/d6b/df3/fa8 [3156185,76117] 0 2024-03-18T14:51:06.825 INFO:tasks.workunit.client.1.smithi100.stdout:4/489: dwrite d9/d27/f2e [0,4194304] 0 2024-03-18T14:51:06.825 INFO:tasks.workunit.client.1.smithi100.stdout:4/490: truncate d9/f7f 332465 0 2024-03-18T14:51:06.825 INFO:tasks.workunit.client.1.smithi100.stdout:4/491: write d9/d13/d5a/d5b/f70 [1164750,105550] 0 2024-03-18T14:51:06.832 INFO:tasks.workunit.client.0.smithi012.stdout:2/855: symlink d0/d6b/d9f/l113 0 2024-03-18T14:51:06.832 INFO:tasks.workunit.client.0.smithi012.stdout:2/856: chown d0/d6b/dbf/d33/f62 0 1 2024-03-18T14:51:06.834 INFO:tasks.workunit.client.1.smithi100.stdout:8/538: link d4/dc/d1f/d59/d6c/c92 d4/dc/dd/de/d19/d37/cba 0 2024-03-18T14:51:06.834 INFO:tasks.workunit.client.1.smithi100.stdout:8/539: dread - d4/fa8 zero size 2024-03-18T14:51:06.834 INFO:tasks.workunit.client.1.smithi100.stdout:8/540: stat d4/dc/dd/de/d21/d23 0 2024-03-18T14:51:06.836 INFO:tasks.workunit.client.0.smithi012.stdout:4/757: mknod d6/da/d58/c103 0 2024-03-18T14:51:06.836 INFO:tasks.workunit.client.0.smithi012.stdout:4/758: truncate d6/da/d89/d97/f9a 17606 0 2024-03-18T14:51:06.837 INFO:tasks.workunit.client.1.smithi100.stdout:5/455: mkdir d29/d2d/d32/d92 0 2024-03-18T14:51:06.838 INFO:tasks.workunit.client.1.smithi100.stdout:0/584: creat dd/d31/d91/d56/fd2 x:0 0 0 2024-03-18T14:51:06.840 INFO:tasks.workunit.client.0.smithi012.stdout:1/779: symlink d8/db/d10/d31/d43/d5e/d65/d84/lfd 0 2024-03-18T14:51:06.841 INFO:tasks.workunit.client.0.smithi012.stdout:9/900: dwrite d3/de6/dbf/d77/d52/d65/fc4 [0,4194304] 0 2024-03-18T14:51:06.842 INFO:tasks.workunit.client.1.smithi100.stdout:9/503: creat da/d51/d55/d7e/f9d x:0 0 0 2024-03-18T14:51:06.843 INFO:tasks.workunit.client.1.smithi100.stdout:9/504: chown da/dd/d29/d71 3 1 2024-03-18T14:51:06.843 INFO:tasks.workunit.client.1.smithi100.stdout:4/492: dwrite d9/d17/f45 [0,4194304] 0 2024-03-18T14:51:06.851 INFO:tasks.workunit.client.0.smithi012.stdout:3/788: mknod dc/d30/d3a/da0/dc5/deb/cfa 0 2024-03-18T14:51:06.852 INFO:tasks.workunit.client.1.smithi100.stdout:6/505: mknod d6/d7/df/d26/d4e/d66/d36/c99 0 2024-03-18T14:51:06.852 INFO:tasks.workunit.client.1.smithi100.stdout:6/506: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f5d [580622,35145] 0 2024-03-18T14:51:06.854 INFO:tasks.workunit.client.0.smithi012.stdout:5/823: mkdir d3/d15/d42/dc6/d73/d6b/d8f/d113 0 2024-03-18T14:51:06.855 INFO:tasks.workunit.client.1.smithi100.stdout:1/489: rename d8/dd to d8/dd/d8f 22 2024-03-18T14:51:06.857 INFO:tasks.workunit.client.0.smithi012.stdout:2/857: symlink d0/d6b/dbf/d33/d77/df5/d103/df6/l114 0 2024-03-18T14:51:06.859 INFO:tasks.workunit.client.1.smithi100.stdout:3/545: getdents d3/d16/d13/d17/d7d 0 2024-03-18T14:51:06.859 INFO:tasks.workunit.client.1.smithi100.stdout:3/546: chown d3/dd/d21/c85 162 1 2024-03-18T14:51:06.859 INFO:tasks.workunit.client.1.smithi100.stdout:3/547: readlink d3/l34 0 2024-03-18T14:51:06.860 INFO:tasks.workunit.client.1.smithi100.stdout:0/585: dwrite fa [0,4194304] 0 2024-03-18T14:51:06.860 INFO:tasks.workunit.client.1.smithi100.stdout:0/586: dread - dd/d2b/d95/d79/d7d/fa8 zero size 2024-03-18T14:51:06.861 INFO:tasks.workunit.client.0.smithi012.stdout:4/759: mknod d6/da/d89/d94/dbe/dd6/c104 0 2024-03-18T14:51:06.861 INFO:tasks.workunit.client.0.smithi012.stdout:4/760: write d6/da/ffc [7972,96346] 0 2024-03-18T14:51:06.861 INFO:tasks.workunit.client.0.smithi012.stdout:4/761: chown d6/da/d89/d97/dc5/dd3/fb1 514 1 2024-03-18T14:51:06.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/541: rmdir d4/dc/d31 39 2024-03-18T14:51:06.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/542: chown d4/dc/dd/de/d21/d23/d25/f71 11 1 2024-03-18T14:51:06.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/543: readlink d4/dc/dd/de/l29 0 2024-03-18T14:51:06.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/544: truncate d4/dc/dd/de/d21/d23/f62 1264615 0 2024-03-18T14:51:06.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/545: write d4/dc/d1f/d33/f36 [1602879,62023] 0 2024-03-18T14:51:06.863 INFO:tasks.workunit.client.0.smithi012.stdout:1/780: dwrite d8/db/d10/d56/f8f [0,4194304] 0 2024-03-18T14:51:06.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/505: mkdir da/dd/d29/d9e 0 2024-03-18T14:51:06.865 INFO:tasks.workunit.client.1.smithi100.stdout:9/506: truncate da/dd/d29/d71/d72/f78 1878070 0 2024-03-18T14:51:06.866 INFO:tasks.workunit.client.0.smithi012.stdout:9/901: mkdir d3/de6/dbf/d77/d52/d95/da9/d127 0 2024-03-18T14:51:06.868 INFO:tasks.workunit.client.1.smithi100.stdout:4/493: mknod d9/d27/d62/d7b/d82/c91 0 2024-03-18T14:51:06.869 INFO:tasks.workunit.client.0.smithi012.stdout:3/789: mknod dc/d30/d35/cfb 0 2024-03-18T14:51:06.872 INFO:tasks.workunit.client.1.smithi100.stdout:6/507: rename d6/d12/c35 to d6/d7/df/d26/d4e/c9a 0 2024-03-18T14:51:06.872 INFO:tasks.workunit.client.0.smithi012.stdout:1/781: dread d8/db/d10/d31/d81/fae [0,4194304] 0 2024-03-18T14:51:06.874 INFO:tasks.workunit.client.1.smithi100.stdout:1/490: creat d8/d2a/d37/d50/f90 x:0 0 0 2024-03-18T14:51:06.874 INFO:tasks.workunit.client.1.smithi100.stdout:1/491: chown d8/d1e/d11/d27/d3f/f78 25004510 1 2024-03-18T14:51:06.876 INFO:tasks.workunit.client.0.smithi012.stdout:2/858: dwrite d0/d6b/dbf/d4f/fb1 [0,4194304] 0 2024-03-18T14:51:06.876 INFO:tasks.workunit.client.0.smithi012.stdout:2/859: chown d0/d6b/dbf/d33/d71/c82 1 1 2024-03-18T14:51:06.876 INFO:tasks.workunit.client.0.smithi012.stdout:4/762: rmdir d6/d1a/d30/de9 0 2024-03-18T14:51:06.879 INFO:tasks.workunit.client.0.smithi012.stdout:9/902: dread d3/d9/f39 [0,4194304] 0 2024-03-18T14:51:06.883 INFO:tasks.workunit.client.0.smithi012.stdout:3/790: truncate dc/d30/d3a/f4b 445830 0 2024-03-18T14:51:06.883 INFO:tasks.workunit.client.0.smithi012.stdout:3/791: fsync dc/d3c/d59/f5a 0 2024-03-18T14:51:06.883 INFO:tasks.workunit.client.0.smithi012.stdout:3/792: fsync dc/d91/fbb 0 2024-03-18T14:51:06.884 INFO:tasks.workunit.client.0.smithi012.stdout:5/824: truncate d3/d15/d42/dc6/d73/f7e 3171766 0 2024-03-18T14:51:06.887 INFO:tasks.workunit.client.0.smithi012.stdout:1/782: symlink d8/db/d10/d31/d43/d5e/d65/d70/lfe 0 2024-03-18T14:51:06.897 INFO:tasks.workunit.client.1.smithi100.stdout:0/587: creat dd/d31/d91/d56/d71/dcc/dcf/fd3 x:0 0 0 2024-03-18T14:51:06.899 INFO:tasks.workunit.client.1.smithi100.stdout:5/456: rmdir d29/d2d/d39/d60 39 2024-03-18T14:51:06.899 INFO:tasks.workunit.client.1.smithi100.stdout:5/457: chown d29/d2d/d32/d38/c3f 541647801 1 2024-03-18T14:51:06.901 INFO:tasks.workunit.client.0.smithi012.stdout:4/763: dwrite d6/d19/d39/f50 [4194304,4194304] 0 2024-03-18T14:51:06.901 INFO:tasks.workunit.client.0.smithi012.stdout:4/764: creat d6/d1a/d30/d46/d6f/d8b/f105 x:0 0 0 2024-03-18T14:51:06.903 INFO:tasks.workunit.client.1.smithi100.stdout:9/507: creat da/d46/d4d/f9f x:0 0 0 2024-03-18T14:51:06.903 INFO:tasks.workunit.client.1.smithi100.stdout:9/508: fsync da/d2c/d2f/f4a 0 2024-03-18T14:51:06.908 INFO:tasks.workunit.client.0.smithi012.stdout:2/860: creat d0/d3c/f115 x:0 0 0 2024-03-18T14:51:06.910 INFO:tasks.workunit.client.0.smithi012.stdout:9/903: mknod d3/de6/dbf/d77/d52/d78/c128 0 2024-03-18T14:51:06.910 INFO:tasks.workunit.client.0.smithi012.stdout:9/904: chown d3/de6/dbf/d77/d52/d95/deb 1039 1 2024-03-18T14:51:06.911 INFO:tasks.workunit.client.1.smithi100.stdout:4/494: mkdir d9/d13/d25/d92 0 2024-03-18T14:51:06.911 INFO:tasks.workunit.client.1.smithi100.stdout:4/495: creat d9/d27/d62/d7b/f93 x:0 0 0 2024-03-18T14:51:06.913 INFO:tasks.workunit.client.1.smithi100.stdout:8/546: rename d4/dc/dd/de/d21/d23/d47/d54/d55 to d4/d5a/dbb 0 2024-03-18T14:51:06.918 INFO:tasks.workunit.client.0.smithi012.stdout:3/793: mkdir dc/d30/d3a/da0/dc5/dc7/dfc 0 2024-03-18T14:51:06.918 INFO:tasks.workunit.client.0.smithi012.stdout:3/794: fdatasync dc/d3c/d4c/fc6 0 2024-03-18T14:51:06.921 INFO:tasks.workunit.client.1.smithi100.stdout:6/508: link d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/c46 d6/d12/d78/c9b 0 2024-03-18T14:51:06.921 INFO:tasks.workunit.client.1.smithi100.stdout:6/509: write d6/d7/f8 [2205753,102925] 0 2024-03-18T14:51:06.923 INFO:tasks.workunit.client.1.smithi100.stdout:1/492: mkdir d8/d2a/d37/d50/d6f/d81/d91 0 2024-03-18T14:51:06.925 INFO:tasks.workunit.client.0.smithi012.stdout:5/825: creat d3/d1e/f114 x:0 0 0 2024-03-18T14:51:06.925 INFO:tasks.workunit.client.0.smithi012.stdout:5/826: creat d3/dad/df2/d102/f115 x:0 0 0 2024-03-18T14:51:06.925 INFO:tasks.workunit.client.1.smithi100.stdout:6/510: dread d6/d12/d2c/f4c [0,4194304] 0 2024-03-18T14:51:06.926 INFO:tasks.workunit.client.1.smithi100.stdout:3/548: rmdir d3/d16/d13/d71 39 2024-03-18T14:51:06.927 INFO:tasks.workunit.client.0.smithi012.stdout:1/783: truncate d8/db/d10/d31/d81/fd3 93530 0 2024-03-18T14:51:06.928 INFO:tasks.workunit.client.1.smithi100.stdout:0/588: mknod dd/d2b/cd4 0 2024-03-18T14:51:06.932 INFO:tasks.workunit.client.0.smithi012.stdout:5/827: dread d3/f9e [0,4194304] 0 2024-03-18T14:51:06.934 INFO:tasks.workunit.client.0.smithi012.stdout:3/795: mknod dc/d3c/cfd 0 2024-03-18T14:51:06.935 INFO:tasks.workunit.client.0.smithi012.stdout:3/796: dread dc/d3c/d59/db2/fb5 [0,4194304] 0 2024-03-18T14:51:06.936 INFO:tasks.workunit.client.1.smithi100.stdout:5/458: dwrite d29/d4a/f8e [0,4194304] 0 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:4/765: dwrite d6/f21 [0,4194304] 0 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:4/766: chown d6/d1a/d30/d46/d6f/d7c 16194 1 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/905: dwrite d3/de6/dbf/d77/d52/f6d [0,4194304] 0 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/906: truncate d3/de6/dbf/d77/d52/d99/da0/ddf/dee/f116 254673 0 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/907: chown d3/de6/dbf/d66/d69/d7f/d8c/d8b/f108 3210526 1 2024-03-18T14:51:06.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/908: write d3/de6/dbf/d77/d52/d65/fe5 [470452,86152] 0 2024-03-18T14:51:06.938 INFO:tasks.workunit.client.1.smithi100.stdout:4/496: unlink d9/f88 0 2024-03-18T14:51:06.938 INFO:tasks.workunit.client.1.smithi100.stdout:4/497: truncate d9/fe 4554786 0 2024-03-18T14:51:06.940 INFO:tasks.workunit.client.1.smithi100.stdout:8/547: mknod d4/d5a/dbb/cbc 0 2024-03-18T14:51:06.940 INFO:tasks.workunit.client.1.smithi100.stdout:4/498: dread d9/d27/d42/f72 [0,4194304] 0 2024-03-18T14:51:06.942 INFO:tasks.workunit.client.1.smithi100.stdout:2/489: sync 2024-03-18T14:51:06.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/511: sync 2024-03-18T14:51:06.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/512: chown d5/l8 9743172 1 2024-03-18T14:51:06.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/513: stat d5/c12 0 2024-03-18T14:51:06.945 INFO:tasks.workunit.client.0.smithi012.stdout:9/909: dread d3/de6/dbf/d66/d69/d7f/d8c/dac/fcb [0,4194304] 0 2024-03-18T14:51:06.945 INFO:tasks.workunit.client.0.smithi012.stdout:9/910: fsync d3/de6/dbf/d66/d69/d7f/d8c/d8b/f11c 0 2024-03-18T14:51:06.945 INFO:tasks.workunit.client.1.smithi100.stdout:1/493: write d8/d2a/f41 [3512754,68271] 0 2024-03-18T14:51:06.946 INFO:tasks.workunit.client.0.smithi012.stdout:5/828: rmdir d3/d15/d5f/df7 0 2024-03-18T14:51:06.947 INFO:tasks.workunit.client.0.smithi012.stdout:2/861: dwrite d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:51:06.948 INFO:tasks.workunit.client.1.smithi100.stdout:6/511: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f9c x:0 0 0 2024-03-18T14:51:06.949 INFO:tasks.workunit.client.0.smithi012.stdout:1/784: read d8/db/d10/d31/f4d [537156,55349] 0 2024-03-18T14:51:06.949 INFO:tasks.workunit.client.0.smithi012.stdout:1/785: dread - d8/db/d10/d31/d43/f69 zero size 2024-03-18T14:51:06.951 INFO:tasks.workunit.client.1.smithi100.stdout:8/548: dread d4/dc/dd/de/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:06.951 INFO:tasks.workunit.client.1.smithi100.stdout:8/549: truncate d4/dc/d31/d85/fb7 1036156 0 2024-03-18T14:51:06.951 INFO:tasks.workunit.client.1.smithi100.stdout:8/550: read d4/dc/dd/de/d21/d23/d47/f50 [45015,60312] 0 2024-03-18T14:51:06.952 INFO:tasks.workunit.client.0.smithi012.stdout:4/767: dwrite d6/da/d89/fce [0,4194304] 0 2024-03-18T14:51:06.953 INFO:tasks.workunit.client.1.smithi100.stdout:9/509: rmdir da/dd/d3c/d4b 39 2024-03-18T14:51:06.954 INFO:tasks.workunit.client.1.smithi100.stdout:1/494: dread d8/d1e/d11/d27/f4d [0,4194304] 0 2024-03-18T14:51:06.954 INFO:tasks.workunit.client.1.smithi100.stdout:1/495: fsync d8/d2a/d37/f5f 0 2024-03-18T14:51:06.954 INFO:tasks.workunit.client.1.smithi100.stdout:1/496: fsync d8/d1e/d11/d6d/f88 0 2024-03-18T14:51:06.954 INFO:tasks.workunit.client.0.smithi012.stdout:4/768: dread d6/d1a/f2a [0,4194304] 0 2024-03-18T14:51:06.955 INFO:tasks.workunit.client.0.smithi012.stdout:4/769: write d6/da/d89/d97/dc5/dd3/fb1 [272818,30727] 0 2024-03-18T14:51:06.955 INFO:tasks.workunit.client.0.smithi012.stdout:4/770: chown d6/d19/f2f 234504 1 2024-03-18T14:51:06.957 INFO:tasks.workunit.client.0.smithi012.stdout:3/797: dread dc/d30/d3a/fdf [0,4194304] 0 2024-03-18T14:51:06.959 INFO:tasks.workunit.client.0.smithi012.stdout:2/862: creat d0/d107/f116 x:0 0 0 2024-03-18T14:51:06.959 INFO:tasks.workunit.client.0.smithi012.stdout:2/863: fdatasync d0/da2/f10e 0 2024-03-18T14:51:06.959 INFO:tasks.workunit.client.0.smithi012.stdout:3/798: dread dc/f32 [0,4194304] 0 2024-03-18T14:51:06.960 INFO:tasks.workunit.client.0.smithi012.stdout:9/911: read d3/de6/fc2 [3769885,51477] 0 2024-03-18T14:51:06.965 INFO:tasks.workunit.client.1.smithi100.stdout:9/510: write da/dd/f94 [3767114,115022] 0 2024-03-18T14:51:06.965 INFO:tasks.workunit.client.1.smithi100.stdout:9/511: truncate da/d46/d4d/d67/f68 57261 0 2024-03-18T14:51:06.966 INFO:tasks.workunit.client.0.smithi012.stdout:5/829: read d3/d84/f85 [1341588,26312] 0 2024-03-18T14:51:06.969 INFO:tasks.workunit.client.1.smithi100.stdout:5/459: symlink d29/d4a/l93 0 2024-03-18T14:51:06.969 INFO:tasks.workunit.client.1.smithi100.stdout:5/460: dread - d29/f7e zero size 2024-03-18T14:51:06.969 INFO:tasks.workunit.client.1.smithi100.stdout:5/461: truncate d29/d2d/d39/d65/f6a 592942 0 2024-03-18T14:51:06.969 INFO:tasks.workunit.client.1.smithi100.stdout:5/462: stat d29/d2d/c5b 0 2024-03-18T14:51:06.969 INFO:tasks.workunit.client.1.smithi100.stdout:5/463: chown f26 213674 1 2024-03-18T14:51:06.971 INFO:tasks.workunit.client.0.smithi012.stdout:4/771: rmdir d6/d1a/d30/d46/d6f 39 2024-03-18T14:51:06.974 INFO:tasks.workunit.client.1.smithi100.stdout:4/499: creat d9/d13/d39/d76/f94 x:0 0 0 2024-03-18T14:51:06.974 INFO:tasks.workunit.client.1.smithi100.stdout:4/500: chown d9/d13/d39/f8f 15823 1 2024-03-18T14:51:06.976 INFO:tasks.workunit.client.0.smithi012.stdout:2/864: creat d0/d6b/dbf/d33/d71/de5/f117 x:0 0 0 2024-03-18T14:51:06.977 INFO:tasks.workunit.client.0.smithi012.stdout:1/786: dwrite d8/db/d10/d31/d43/d5e/d65/d84/fde [0,4194304] 0 2024-03-18T14:51:06.978 INFO:tasks.workunit.client.0.smithi012.stdout:1/787: fdatasync d8/d1e/d27/fb7 0 2024-03-18T14:51:06.978 INFO:tasks.workunit.client.0.smithi012.stdout:1/788: truncate d8/d1e/d27/d44/f97 1253646 0 2024-03-18T14:51:06.979 INFO:tasks.workunit.client.0.smithi012.stdout:3/799: creat dc/d30/d3a/da0/dc5/ffe x:0 0 0 2024-03-18T14:51:06.980 INFO:tasks.workunit.client.1.smithi100.stdout:7/514: getdents d5/d22/d3b 0 2024-03-18T14:51:06.982 INFO:tasks.workunit.client.0.smithi012.stdout:5/830: symlink d3/d84/dfa/l116 0 2024-03-18T14:51:06.984 INFO:tasks.workunit.client.1.smithi100.stdout:8/551: dwrite d4/dc/d31/d66/f9b [4194304,4194304] 0 2024-03-18T14:51:06.985 INFO:tasks.workunit.client.0.smithi012.stdout:9/912: dwrite d3/de6/dbf/d66/d21/f46 [0,4194304] 0 2024-03-18T14:51:06.985 INFO:tasks.workunit.client.0.smithi012.stdout:4/772: mkdir d6/d1a/d30/d46/d6f/d8b/de7/d106 0 2024-03-18T14:51:06.987 INFO:tasks.workunit.client.1.smithi100.stdout:6/512: rename d6/d7/df/d26/d4e/d66/c34 to d6/d7/df/d26/d4e/d66/d36/d4f/d8a/c9d 0 2024-03-18T14:51:06.989 INFO:tasks.workunit.client.0.smithi012.stdout:2/865: mknod d0/d3c/d40/d47/d85/c118 0 2024-03-18T14:51:06.989 INFO:tasks.workunit.client.0.smithi012.stdout:2/866: write d0/d3c/d40/d47/d85/fdb [546135,104103] 0 2024-03-18T14:51:06.992 INFO:tasks.workunit.client.0.smithi012.stdout:3/800: dread dc/d30/d35/f70 [0,4194304] 0 2024-03-18T14:51:06.993 INFO:tasks.workunit.client.1.smithi100.stdout:3/549: link d3/d16/d13/d1f/d37/f54 d3/fc1 0 2024-03-18T14:51:06.997 INFO:tasks.workunit.client.0.smithi012.stdout:2/867: dread d0/d3c/d40/f5a [4194304,4194304] 0 2024-03-18T14:51:06.999 INFO:tasks.workunit.client.0.smithi012.stdout:5/831: symlink d3/de0/l117 0 2024-03-18T14:51:06.999 INFO:tasks.workunit.client.0.smithi012.stdout:5/832: stat d3/d1e/f8c 0 2024-03-18T14:51:07.005 INFO:tasks.workunit.client.1.smithi100.stdout:1/497: unlink d8/d1e/d11/d27/d3f/f78 0 2024-03-18T14:51:07.005 INFO:tasks.workunit.client.1.smithi100.stdout:6/513: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f79 [7001607,101760] 0 2024-03-18T14:51:07.007 INFO:tasks.workunit.client.0.smithi012.stdout:1/789: dwrite d8/db/d10/d31/d81/f8c [0,4194304] 0 2024-03-18T14:51:07.007 INFO:tasks.workunit.client.0.smithi012.stdout:1/790: chown d8/db/d10/d31/l78 3777 1 2024-03-18T14:51:07.009 INFO:tasks.workunit.client.1.smithi100.stdout:9/512: mkdir da/dd/da0 0 2024-03-18T14:51:07.009 INFO:tasks.workunit.client.1.smithi100.stdout:9/513: dread - da/dd/d29/d71/d72/f81 zero size 2024-03-18T14:51:07.009 INFO:tasks.workunit.client.1.smithi100.stdout:9/514: fsync da/d88/f8c 0 2024-03-18T14:51:07.010 INFO:tasks.workunit.client.1.smithi100.stdout:9/515: creat da/dd/fa1 x:0 0 0 2024-03-18T14:51:07.011 INFO:tasks.workunit.client.1.smithi100.stdout:5/464: mknod d29/d2d/d39/c94 0 2024-03-18T14:51:07.011 INFO:tasks.workunit.client.1.smithi100.stdout:5/465: chown d29/d2d/d39/d83/f85 83 1 2024-03-18T14:51:07.012 INFO:tasks.workunit.client.1.smithi100.stdout:2/490: rmdir d3/d1a/d76 39 2024-03-18T14:51:07.013 INFO:tasks.workunit.client.1.smithi100.stdout:7/515: mknod d5/d22/cc0 0 2024-03-18T14:51:07.015 INFO:tasks.workunit.client.1.smithi100.stdout:8/552: creat d4/dc/d1f/d59/fbd x:0 0 0 2024-03-18T14:51:07.016 INFO:tasks.workunit.client.0.smithi012.stdout:3/801: creat dc/d30/dab/fff x:0 0 0 2024-03-18T14:51:07.018 INFO:tasks.workunit.client.1.smithi100.stdout:3/550: creat d3/fc2 x:0 0 0 2024-03-18T14:51:07.018 INFO:tasks.workunit.client.1.smithi100.stdout:3/551: dread - d3/dd/d21/d3e/f7e zero size 2024-03-18T14:51:07.018 INFO:tasks.workunit.client.1.smithi100.stdout:3/552: chown d3/dd/d21/d3e/f32 4 1 2024-03-18T14:51:07.020 INFO:tasks.workunit.client.0.smithi012.stdout:2/868: mknod d0/d6b/dbf/d33/d71/d90/dea/c119 0 2024-03-18T14:51:07.020 INFO:tasks.workunit.client.0.smithi012.stdout:2/869: chown d0/cf 73760 1 2024-03-18T14:51:07.021 INFO:tasks.workunit.client.0.smithi012.stdout:9/913: dwrite d3/de6/dbf/d66/f51 [0,4194304] 0 2024-03-18T14:51:07.022 INFO:tasks.workunit.client.1.smithi100.stdout:4/501: dread d9/d27/f2e [0,4194304] 0 2024-03-18T14:51:07.022 INFO:tasks.workunit.client.1.smithi100.stdout:1/498: creat d8/d30/d61/d8d/f92 x:0 0 0 2024-03-18T14:51:07.024 INFO:tasks.workunit.client.1.smithi100.stdout:6/514: symlink d6/d7/df/d26/d4e/l9e 0 2024-03-18T14:51:07.024 INFO:tasks.workunit.client.0.smithi012.stdout:4/773: dwrite d6/d1a/f62 [0,4194304] 0 2024-03-18T14:51:07.024 INFO:tasks.workunit.client.0.smithi012.stdout:4/774: truncate d6/d19/fad 4313709 0 2024-03-18T14:51:07.027 INFO:tasks.workunit.client.0.smithi012.stdout:1/791: mknod d8/d1e/d27/d44/cff 0 2024-03-18T14:51:07.028 INFO:tasks.workunit.client.0.smithi012.stdout:1/792: write d8/db/d10/d31/d43/d5e/da5/fb8 [5720999,130006] 0 2024-03-18T14:51:07.029 INFO:tasks.workunit.client.1.smithi100.stdout:9/516: symlink da/dd/d3c/d4b/la2 0 2024-03-18T14:51:07.030 INFO:tasks.workunit.client.0.smithi012.stdout:1/793: dread d8/db/d10/d31/d45/f6d [0,4194304] 0 2024-03-18T14:51:07.030 INFO:tasks.workunit.client.0.smithi012.stdout:1/794: truncate d8/db/d10/d31/f68 1200335 0 2024-03-18T14:51:07.030 INFO:tasks.workunit.client.0.smithi012.stdout:1/795: write d8/db/d10/d4e/ff1 [281291,72830] 0 2024-03-18T14:51:07.032 INFO:tasks.workunit.client.0.smithi012.stdout:5/833: dwrite d3/db6/fec [0,4194304] 0 2024-03-18T14:51:07.035 INFO:tasks.workunit.client.1.smithi100.stdout:7/516: dwrite d5/d22/d24/f8b [0,4194304] 0 2024-03-18T14:51:07.035 INFO:tasks.workunit.client.1.smithi100.stdout:7/517: creat d5/d14/d16/d2e/d62/d66/fc1 x:0 0 0 2024-03-18T14:51:07.036 INFO:tasks.workunit.client.0.smithi012.stdout:3/802: mknod dc/d30/c100 0 2024-03-18T14:51:07.036 INFO:tasks.workunit.client.0.smithi012.stdout:3/803: write dc/d30/d35/f65 [4043261,79699] 0 2024-03-18T14:51:07.036 INFO:tasks.workunit.client.1.smithi100.stdout:5/466: link d29/l75 d29/d2d/d39/d74/l95 0 2024-03-18T14:51:07.040 INFO:tasks.workunit.client.0.smithi012.stdout:5/834: dread d3/d84/f85 [0,4194304] 0 2024-03-18T14:51:07.044 INFO:tasks.workunit.client.1.smithi100.stdout:2/491: mknod d3/da/de/d20/caf 0 2024-03-18T14:51:07.044 INFO:tasks.workunit.client.1.smithi100.stdout:2/492: chown d3/d1a/l4d 492510913 1 2024-03-18T14:51:07.049 INFO:tasks.workunit.client.1.smithi100.stdout:8/553: symlink d4/d3c/lbe 0 2024-03-18T14:51:07.052 INFO:tasks.workunit.client.1.smithi100.stdout:3/553: rename d3/d16/d13/d71/d74/lb4 to d3/d16/d13/d1f/d3f/d4f/d65/lc3 0 2024-03-18T14:51:07.052 INFO:tasks.workunit.client.0.smithi012.stdout:9/914: dwrite d3/de6/dbf/d77/da8/f96 [0,4194304] 0 2024-03-18T14:51:07.055 INFO:tasks.workunit.client.1.smithi100.stdout:4/502: symlink d9/d3a/l95 0 2024-03-18T14:51:07.055 INFO:tasks.workunit.client.1.smithi100.stdout:4/503: chown d9/f11 2016 1 2024-03-18T14:51:07.057 INFO:tasks.workunit.client.0.smithi012.stdout:4/775: rmdir d6/d1a/d30/d46/d6f/d8b/de7/d106 0 2024-03-18T14:51:07.057 INFO:tasks.workunit.client.0.smithi012.stdout:4/776: creat d6/d1a/d30/d46/d6f/d8b/f107 x:0 0 0 2024-03-18T14:51:07.058 INFO:tasks.workunit.client.0.smithi012.stdout:2/870: dwrite d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:51:07.058 INFO:tasks.workunit.client.0.smithi012.stdout:2/871: chown d0/d6b/dbf/l2b 2235 1 2024-03-18T14:51:07.058 INFO:tasks.workunit.client.0.smithi012.stdout:2/872: write d0/d6b/dbf/d33/d71/d90/fae [1476309,12061] 0 2024-03-18T14:51:07.059 INFO:tasks.workunit.client.0.smithi012.stdout:1/796: dwrite d8/d1e/f2f [0,4194304] 0 2024-03-18T14:51:07.059 INFO:tasks.workunit.client.0.smithi012.stdout:1/797: chown d8/db/d10/d31/d43/d5e/df6 6 1 2024-03-18T14:51:07.060 INFO:tasks.workunit.client.0.smithi012.stdout:2/873: dread d0/d3c/fb3 [0,4194304] 0 2024-03-18T14:51:07.060 INFO:tasks.workunit.client.1.smithi100.stdout:1/499: mkdir d8/d1e/d11/d6d/d59/d93 0 2024-03-18T14:51:07.060 INFO:tasks.workunit.client.1.smithi100.stdout:1/500: chown d8/d2a 36089558 1 2024-03-18T14:51:07.066 INFO:tasks.workunit.client.0.smithi012.stdout:4/777: dread d6/f25 [0,4194304] 0 2024-03-18T14:51:07.068 INFO:tasks.workunit.client.0.smithi012.stdout:3/804: creat dc/d3c/d59/f101 x:0 0 0 2024-03-18T14:51:07.072 INFO:tasks.workunit.client.1.smithi100.stdout:5/467: dwrite d29/d2d/d39/f46 [0,4194304] 0 2024-03-18T14:51:07.072 INFO:tasks.workunit.client.1.smithi100.stdout:5/468: write d29/d2d/d39/d61/f89 [1323772,107884] 0 2024-03-18T14:51:07.072 INFO:tasks.workunit.client.1.smithi100.stdout:2/493: dwrite d3/da/f43 [4194304,4194304] 0 2024-03-18T14:51:07.072 INFO:tasks.workunit.client.1.smithi100.stdout:8/554: rename d4/dc/d1f/d59/fac to d4/dc/d1f/d59/fbf 0 2024-03-18T14:51:07.072 INFO:tasks.workunit.client.1.smithi100.stdout:8/555: truncate d4/dc/d1f/d59/d6c/f98 472446 0 2024-03-18T14:51:07.073 INFO:tasks.workunit.client.0.smithi012.stdout:6/791: sync 2024-03-18T14:51:07.073 INFO:tasks.workunit.client.0.smithi012.stdout:7/859: sync 2024-03-18T14:51:07.073 INFO:tasks.workunit.client.0.smithi012.stdout:0/874: sync 2024-03-18T14:51:07.073 INFO:tasks.workunit.client.0.smithi012.stdout:5/835: symlink d3/d15/d42/dc6/d86/l118 0 2024-03-18T14:51:07.076 INFO:tasks.workunit.client.0.smithi012.stdout:7/860: dread dd/d13/d23/d76/d109/d129/dd5/f11e [0,4194304] 0 2024-03-18T14:51:07.076 INFO:tasks.workunit.client.0.smithi012.stdout:7/861: fdatasync dd/d13/d23/d2f/d53/f8e 0 2024-03-18T14:51:07.076 INFO:tasks.workunit.client.0.smithi012.stdout:7/862: chown dd/d4f/lac 887683123 1 2024-03-18T14:51:07.076 INFO:tasks.workunit.client.1.smithi100.stdout:1/501: dread d8/d30/f35 [0,4194304] 0 2024-03-18T14:51:07.078 INFO:tasks.workunit.client.1.smithi100.stdout:3/554: rename d3/d16/d13/d17/d7d/c93 to d3/dd/d49/cc4 0 2024-03-18T14:51:07.083 INFO:tasks.workunit.client.1.smithi100.stdout:4/504: rmdir d9/d13/d5a/d5b 39 2024-03-18T14:51:07.085 INFO:tasks.workunit.client.0.smithi012.stdout:9/915: creat d3/de6/dbf/f129 x:0 0 0 2024-03-18T14:51:07.087 INFO:tasks.workunit.client.1.smithi100.stdout:6/515: getdents d6/d7/df/d26/d4e 0 2024-03-18T14:51:07.098 INFO:tasks.workunit.client.0.smithi012.stdout:2/874: creat d0/d6b/dbf/d33/d71/de5/d108/f11a x:0 0 0 2024-03-18T14:51:07.098 INFO:tasks.workunit.client.0.smithi012.stdout:2/875: getdents d0/da2/deb 0 2024-03-18T14:51:07.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/556: dwrite d4/dc/dd/de/f93 [4194304,4194304] 0 2024-03-18T14:51:07.099 INFO:tasks.workunit.client.0.smithi012.stdout:9/916: read d3/d9/fd [2290342,90361] 0 2024-03-18T14:51:07.099 INFO:tasks.workunit.client.0.smithi012.stdout:9/917: chown d3/d9/f6c 45996 1 2024-03-18T14:51:07.100 INFO:tasks.workunit.client.0.smithi012.stdout:3/805: mkdir dc/d30/d3a/da0/dc5/deb/d102 0 2024-03-18T14:51:07.101 INFO:tasks.workunit.client.0.smithi012.stdout:3/806: fsync dc/d30/d35/f65 0 2024-03-18T14:51:07.105 INFO:tasks.workunit.client.1.smithi100.stdout:5/469: mkdir d29/d2d/d39/d71/d96 0 2024-03-18T14:51:07.106 INFO:tasks.workunit.client.1.smithi100.stdout:5/470: write d29/d2d/d39/f4c [3930840,79762] 0 2024-03-18T14:51:07.106 INFO:tasks.workunit.client.1.smithi100.stdout:2/494: symlink d3/lb0 0 2024-03-18T14:51:07.107 INFO:tasks.workunit.client.1.smithi100.stdout:2/495: stat d3/d13/d40/d53/f66 0 2024-03-18T14:51:07.109 INFO:tasks.workunit.client.0.smithi012.stdout:0/875: rename d1/d20/d23/f120 to d1/d20/d4e/d54/d8a/d101/f121 0 2024-03-18T14:51:07.110 INFO:tasks.workunit.client.1.smithi100.stdout:9/517: rename da/dd/d3b/f9c to da/d46/fa3 0 2024-03-18T14:51:07.111 INFO:tasks.workunit.client.0.smithi012.stdout:5/836: creat d3/d15/d42/dc6/d65/da5/f119 x:0 0 0 2024-03-18T14:51:07.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/555: link d3/d16/d13/d1f/d3f/l6f d3/d16/d13/d71/d74/lc5 0 2024-03-18T14:51:07.113 INFO:tasks.workunit.client.1.smithi100.stdout:3/556: dread - d3/d16/d13/d1f/d7c/f89 zero size 2024-03-18T14:51:07.113 INFO:tasks.workunit.client.0.smithi012.stdout:4/778: symlink d6/d1a/l108 0 2024-03-18T14:51:07.114 INFO:tasks.workunit.client.1.smithi100.stdout:4/505: creat d9/d81/f96 x:0 0 0 2024-03-18T14:51:07.115 INFO:tasks.workunit.client.0.smithi012.stdout:9/918: creat d3/de6/dbf/d66/d69/d7f/f12a x:0 0 0 2024-03-18T14:51:07.117 INFO:tasks.workunit.client.1.smithi100.stdout:8/557: mkdir d4/dc/dd/de/d19/dc0 0 2024-03-18T14:51:07.119 INFO:tasks.workunit.client.1.smithi100.stdout:2/496: dread d3/d6/d5f/d63/f72 [0,4194304] 0 2024-03-18T14:51:07.119 INFO:tasks.workunit.client.0.smithi012.stdout:0/876: rename d1/d20/d23/d3b/f2c to d1/d20/d23/d3b/d33/f122 0 2024-03-18T14:51:07.120 INFO:tasks.workunit.client.1.smithi100.stdout:1/502: dwrite d8/f29 [0,4194304] 0 2024-03-18T14:51:07.121 INFO:tasks.workunit.client.0.smithi012.stdout:6/792: dwrite d17/d19/d27/d58/f9f [0,4194304] 0 2024-03-18T14:51:07.121 INFO:tasks.workunit.client.0.smithi012.stdout:6/793: chown d17/d26/fb3 12291 1 2024-03-18T14:51:07.121 INFO:tasks.workunit.client.0.smithi012.stdout:6/794: fdatasync d17/d26/d36/d4d/d82/dcd/fda 0 2024-03-18T14:51:07.121 INFO:tasks.workunit.client.0.smithi012.stdout:7/863: dwrite dd/d13/d23/d105/f127 [0,4194304] 0 2024-03-18T14:51:07.121 INFO:tasks.workunit.client.0.smithi012.stdout:7/864: stat dd/d13/d23/d76/d109/d129/d48/l50 0 2024-03-18T14:51:07.122 INFO:tasks.workunit.client.0.smithi012.stdout:5/837: rmdir d3/d15/d42/dc6 39 2024-03-18T14:51:07.122 INFO:tasks.workunit.client.0.smithi012.stdout:7/865: dread dd/d13/d23/d2f/d53/dc6/df1/ff4 [0,4194304] 0 2024-03-18T14:51:07.123 INFO:tasks.workunit.client.1.smithi100.stdout:5/471: truncate f7 570447 0 2024-03-18T14:51:07.128 INFO:tasks.workunit.client.1.smithi100.stdout:9/518: unlink da/d51/l61 0 2024-03-18T14:51:07.128 INFO:tasks.workunit.client.1.smithi100.stdout:9/519: chown da/dd/d29/d71/d72/f9b 2010457 1 2024-03-18T14:51:07.128 INFO:tasks.workunit.client.1.smithi100.stdout:8/558: dread d4/dc/dd/d48/f5f [0,4194304] 0 2024-03-18T14:51:07.128 INFO:tasks.workunit.client.1.smithi100.stdout:8/559: chown d4/f9 12312949 1 2024-03-18T14:51:07.129 INFO:tasks.workunit.client.1.smithi100.stdout:6/516: dwrite d6/d12/f5c [0,4194304] 0 2024-03-18T14:51:07.129 INFO:tasks.workunit.client.0.smithi012.stdout:2/876: dwrite d0/d3c/f115 [0,4194304] 0 2024-03-18T14:51:07.130 INFO:tasks.workunit.client.0.smithi012.stdout:4/779: creat d6/d1a/d98/daf/db5/f109 x:0 0 0 2024-03-18T14:51:07.132 INFO:tasks.workunit.client.1.smithi100.stdout:3/557: mknod d3/d16/d13/d17/cc6 0 2024-03-18T14:51:07.138 INFO:tasks.workunit.client.0.smithi012.stdout:9/919: truncate d3/de6/dbf/d66/d69/d7f/fe0 726025 0 2024-03-18T14:51:07.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/506: creat d9/d13/d39/d79/f97 x:0 0 0 2024-03-18T14:51:07.140 INFO:tasks.workunit.client.0.smithi012.stdout:5/838: dread d3/d15/d42/dc6/d73/d6b/df3/f7c [4194304,4194304] 0 2024-03-18T14:51:07.153 INFO:tasks.workunit.client.1.smithi100.stdout:1/503: rename d8/d2a/d37/f7d to d8/d1e/d11/d27/d3e/f94 0 2024-03-18T14:51:07.153 INFO:tasks.workunit.client.1.smithi100.stdout:1/504: creat d8/d30/d61/d8d/f95 x:0 0 0 2024-03-18T14:51:07.157 INFO:tasks.workunit.client.1.smithi100.stdout:5/472: creat d29/d2d/f97 x:0 0 0 2024-03-18T14:51:07.164 INFO:tasks.workunit.client.1.smithi100.stdout:8/560: dwrite d4/fa [0,4194304] 0 2024-03-18T14:51:07.166 INFO:tasks.workunit.client.0.smithi012.stdout:4/780: dwrite d6/d19/d6c/d96/df0/ff1 [0,4194304] 0 2024-03-18T14:51:07.166 INFO:tasks.workunit.client.0.smithi012.stdout:0/877: dread d1/d16/d19/f47 [0,4194304] 0 2024-03-18T14:51:07.167 INFO:tasks.workunit.client.1.smithi100.stdout:3/558: dwrite d3/d16/d13/d17/dab/fbc [0,4194304] 0 2024-03-18T14:51:07.171 INFO:tasks.workunit.client.1.smithi100.stdout:5/473: write d29/f37 [2848657,119110] 0 2024-03-18T14:51:07.171 INFO:tasks.workunit.client.1.smithi100.stdout:5/474: read - d29/d2d/d32/f7c zero size 2024-03-18T14:51:07.172 INFO:tasks.workunit.client.0.smithi012.stdout:7/866: rmdir dd/d13/d23/d76/d6f/db3 39 2024-03-18T14:51:07.174 INFO:tasks.workunit.client.1.smithi100.stdout:1/505: dwrite d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:07.179 INFO:tasks.workunit.client.0.smithi012.stdout:2/877: creat d0/d6b/dbf/d33/f11b x:0 0 0 2024-03-18T14:51:07.179 INFO:tasks.workunit.client.0.smithi012.stdout:2/878: chown d0/d6b 783642 1 2024-03-18T14:51:07.183 INFO:tasks.workunit.client.0.smithi012.stdout:5/839: mknod d3/d15/d42/dc6/d65/c11a 0 2024-03-18T14:51:07.185 INFO:tasks.workunit.client.0.smithi012.stdout:4/781: dwrite d6/da/d89/d95/fd4 [0,4194304] 0 2024-03-18T14:51:07.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/520: rename da/dd/f94 to da/fa4 0 2024-03-18T14:51:07.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/521: dread - da/dd/f42 zero size 2024-03-18T14:51:07.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/522: chown da/d46/d4d/f9f 34 1 2024-03-18T14:51:07.186 INFO:tasks.workunit.client.1.smithi100.stdout:9/523: readlink da/l1e 0 2024-03-18T14:51:07.190 INFO:tasks.workunit.client.1.smithi100.stdout:9/524: dread da/dd/d29/d71/d72/f78 [0,4194304] 0 2024-03-18T14:51:07.192 INFO:tasks.workunit.client.1.smithi100.stdout:8/561: truncate d4/f43 579239 0 2024-03-18T14:51:07.194 INFO:tasks.workunit.client.0.smithi012.stdout:3/807: rename dc/d3c/d59/d86 to dc/d3c/d4c/d57/d103 0 2024-03-18T14:51:07.194 INFO:tasks.workunit.client.0.smithi012.stdout:3/808: dread - dc/d91/fea zero size 2024-03-18T14:51:07.208 INFO:tasks.workunit.client.0.smithi012.stdout:5/840: dwrite d3/d29/d35/fd4 [0,4194304] 0 2024-03-18T14:51:07.211 INFO:tasks.workunit.client.1.smithi100.stdout:3/559: creat d3/da5/fc7 x:0 0 0 2024-03-18T14:51:07.211 INFO:tasks.workunit.client.1.smithi100.stdout:3/560: creat d3/d16/d13/d1f/d3f/d4f/d65/fc8 x:0 0 0 2024-03-18T14:51:07.211 INFO:tasks.workunit.client.1.smithi100.stdout:3/561: chown d3/d16/d13/d17/f7a 60635 1 2024-03-18T14:51:07.213 INFO:tasks.workunit.client.1.smithi100.stdout:5/475: rmdir d29/d2d/d32/d38 39 2024-03-18T14:51:07.215 INFO:tasks.workunit.client.0.smithi012.stdout:7/867: mkdir dd/d13/d103/d12b 0 2024-03-18T14:51:07.217 INFO:tasks.workunit.client.0.smithi012.stdout:4/782: dwrite d6/d1a/d30/d46/d6f/d7c/faa [0,4194304] 0 2024-03-18T14:51:07.219 INFO:tasks.workunit.client.0.smithi012.stdout:2/879: mkdir d0/d6b/dbf/d33/d77/da0/d11c 0 2024-03-18T14:51:07.219 INFO:tasks.workunit.client.0.smithi012.stdout:2/880: chown d0/d6b/dbf/d33/d71/c82 887 1 2024-03-18T14:51:07.220 INFO:tasks.workunit.client.1.smithi100.stdout:8/562: dwrite d4/dc/dd/de/d19/d37/f7f [0,4194304] 0 2024-03-18T14:51:07.220 INFO:tasks.workunit.client.1.smithi100.stdout:8/563: fsync d4/dc/d1f/d59/fbd 0 2024-03-18T14:51:07.221 INFO:tasks.workunit.client.0.smithi012.stdout:5/841: dread d3/d29/d35/f48 [0,4194304] 0 2024-03-18T14:51:07.222 INFO:tasks.workunit.client.0.smithi012.stdout:5/842: chown d3/d1e/d33/f74 30 1 2024-03-18T14:51:07.225 INFO:tasks.workunit.client.1.smithi100.stdout:1/506: mknod d8/d2a/d37/d50/d6f/d81/d91/c96 0 2024-03-18T14:51:07.228 INFO:tasks.workunit.client.1.smithi100.stdout:6/517: getdents d6/d7/df/d72/d8b 0 2024-03-18T14:51:07.228 INFO:tasks.workunit.client.1.smithi100.stdout:6/518: chown d6/d7/df/d72/d48/c77 111 1 2024-03-18T14:51:07.232 INFO:tasks.workunit.client.1.smithi100.stdout:0/589: sync 2024-03-18T14:51:07.233 INFO:tasks.workunit.client.0.smithi012.stdout:5/843: read d3/d29/d35/fbc [2314318,11214] 0 2024-03-18T14:51:07.234 INFO:tasks.workunit.client.1.smithi100.stdout:0/590: dread dd/d20/db5/f6d [0,4194304] 0 2024-03-18T14:51:07.235 INFO:tasks.workunit.client.1.smithi100.stdout:0/591: dread - dd/d2b/d95/d79/d7d/da9/db0/fb6 zero size 2024-03-18T14:51:07.235 INFO:tasks.workunit.client.1.smithi100.stdout:0/592: write dd/fb1 [103106,115869] 0 2024-03-18T14:51:07.236 INFO:tasks.workunit.client.1.smithi100.stdout:4/507: rename d9/d13/d25/d51/f61 to d9/d27/f98 0 2024-03-18T14:51:07.236 INFO:tasks.workunit.client.1.smithi100.stdout:0/593: rename dd/d20 to dd/d20/db5/dd5 22 2024-03-18T14:51:07.239 INFO:tasks.workunit.client.1.smithi100.stdout:9/525: mknod da/dd/da0/ca5 0 2024-03-18T14:51:07.239 INFO:tasks.workunit.client.1.smithi100.stdout:9/526: fsync da/dd/d29/f87 0 2024-03-18T14:51:07.239 INFO:tasks.workunit.client.1.smithi100.stdout:9/527: chown da/dd/d3b 410043 1 2024-03-18T14:51:07.240 INFO:tasks.workunit.client.0.smithi012.stdout:6/795: dwrite d17/d18/f94 [0,4194304] 0 2024-03-18T14:51:07.240 INFO:tasks.workunit.client.0.smithi012.stdout:6/796: chown d17/f1a 124 1 2024-03-18T14:51:07.241 INFO:tasks.workunit.client.0.smithi012.stdout:3/809: creat dc/d3c/d4c/d5f/f104 x:0 0 0 2024-03-18T14:51:07.252 INFO:tasks.workunit.client.1.smithi100.stdout:3/562: mkdir d3/dd/d21/dae/dc9 0 2024-03-18T14:51:07.254 INFO:tasks.workunit.client.0.smithi012.stdout:0/878: mknod d1/d20/d7b/c123 0 2024-03-18T14:51:07.254 INFO:tasks.workunit.client.0.smithi012.stdout:0/879: chown d1/d20/d23/d6f/d93/dba 937664 1 2024-03-18T14:51:07.254 INFO:tasks.workunit.client.0.smithi012.stdout:0/880: chown d1/d20/d23/f105 3 1 2024-03-18T14:51:07.257 INFO:tasks.workunit.client.1.smithi100.stdout:5/476: link d29/d2d/d39/d60/f70 d29/d2d/d39/d65/f98 0 2024-03-18T14:51:07.259 INFO:tasks.workunit.client.0.smithi012.stdout:7/868: mkdir dd/d13/d23/d76/d109/d129/dd5/d12c 0 2024-03-18T14:51:07.259 INFO:tasks.workunit.client.0.smithi012.stdout:7/869: read dd/d13/d23/d76/f10a [934765,96691] 0 2024-03-18T14:51:07.259 INFO:tasks.workunit.client.0.smithi012.stdout:7/870: fsync dd/d13/d23/d76/d109/d129/d37/f126 0 2024-03-18T14:51:07.266 INFO:tasks.workunit.client.1.smithi100.stdout:8/564: creat d4/dc/dd/de/d19/db0/fc1 x:0 0 0 2024-03-18T14:51:07.267 INFO:tasks.workunit.client.1.smithi100.stdout:7/518: sync 2024-03-18T14:51:07.267 INFO:tasks.workunit.client.1.smithi100.stdout:5/477: dread d29/d2d/d32/d38/f5d [0,4194304] 0 2024-03-18T14:51:07.267 INFO:tasks.workunit.client.0.smithi012.stdout:5/844: dwrite d3/d15/d42/dc6/d65/fa1 [0,4194304] 0 2024-03-18T14:51:07.268 INFO:tasks.workunit.client.0.smithi012.stdout:4/783: mknod d6/da/d89/da1/dbb/c10a 0 2024-03-18T14:51:07.268 INFO:tasks.workunit.client.1.smithi100.stdout:7/519: write d5/d14/d16/d44/fa7 [987215,118294] 0 2024-03-18T14:51:07.268 INFO:tasks.workunit.client.1.smithi100.stdout:7/520: chown d5/d22/d24/f91 5 1 2024-03-18T14:51:07.268 INFO:tasks.workunit.client.1.smithi100.stdout:7/521: chown d5/d30/d4d/l58 445085 1 2024-03-18T14:51:07.270 INFO:tasks.workunit.client.1.smithi100.stdout:6/519: mkdir d6/d9f 0 2024-03-18T14:51:07.271 INFO:tasks.workunit.client.1.smithi100.stdout:1/507: dwrite d8/d30/f35 [0,4194304] 0 2024-03-18T14:51:07.271 INFO:tasks.workunit.client.0.smithi012.stdout:3/810: dwrite dc/d3c/d4c/d5f/f104 [0,4194304] 0 2024-03-18T14:51:07.272 INFO:tasks.workunit.client.0.smithi012.stdout:2/881: rename d0/d6b/d9f/fd4 to d0/d6b/dbf/d33/d77/d10a/f11d 0 2024-03-18T14:51:07.272 INFO:tasks.workunit.client.0.smithi012.stdout:2/882: getdents d0/d6b/dbf/d33/d71/d90/dea/dfe 0 2024-03-18T14:51:07.276 INFO:tasks.workunit.client.1.smithi100.stdout:0/594: truncate dd/f3a 1124060 0 2024-03-18T14:51:07.276 INFO:tasks.workunit.client.1.smithi100.stdout:0/595: dread - dd/d20/db5/fab zero size 2024-03-18T14:51:07.280 INFO:tasks.workunit.client.1.smithi100.stdout:8/565: write d4/dc/d1f/d33/f4e [4174817,76775] 0 2024-03-18T14:51:07.280 INFO:tasks.workunit.client.1.smithi100.stdout:4/508: rename d9/d27/d42/f50 to d9/d3a/f99 0 2024-03-18T14:51:07.280 INFO:tasks.workunit.client.1.smithi100.stdout:4/509: creat d9/d27/d62/f9a x:0 0 0 2024-03-18T14:51:07.281 INFO:tasks.workunit.client.1.smithi100.stdout:7/522: read d5/d14/d16/d2c/d86/f50 [2462668,96390] 0 2024-03-18T14:51:07.282 INFO:tasks.workunit.client.0.smithi012.stdout:6/797: unlink d17/d19/d27/d6e/fad 0 2024-03-18T14:51:07.283 INFO:tasks.workunit.client.1.smithi100.stdout:9/528: creat da/dd/da0/fa6 x:0 0 0 2024-03-18T14:51:07.286 INFO:tasks.workunit.client.1.smithi100.stdout:3/563: mkdir d3/d16/dca 0 2024-03-18T14:51:07.288 INFO:tasks.workunit.client.0.smithi012.stdout:7/871: creat dd/d13/d23/d76/d109/d129/dd5/f12d x:0 0 0 2024-03-18T14:51:07.289 INFO:tasks.workunit.client.1.smithi100.stdout:7/523: dread d5/f35 [0,4194304] 0 2024-03-18T14:51:07.290 INFO:tasks.workunit.client.1.smithi100.stdout:5/478: mkdir d29/d2d/d39/d65/d99 0 2024-03-18T14:51:07.290 INFO:tasks.workunit.client.1.smithi100.stdout:5/479: dread - d29/d2d/d39/d61/f6b zero size 2024-03-18T14:51:07.293 INFO:tasks.workunit.client.1.smithi100.stdout:6/520: unlink d6/c16 0 2024-03-18T14:51:07.298 INFO:tasks.workunit.client.0.smithi012.stdout:5/845: mknod d3/d15/c11b 0 2024-03-18T14:51:07.298 INFO:tasks.workunit.client.0.smithi012.stdout:5/846: fsync d3/d15/d42/dc6/d73/d6b/df3/fa8 0 2024-03-18T14:51:07.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/508: creat d8/d1e/d11/d8a/f97 x:0 0 0 2024-03-18T14:51:07.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/509: chown d8/d1e/d11/d27/d3e/l6a 275035098 1 2024-03-18T14:51:07.310 INFO:tasks.workunit.client.0.smithi012.stdout:4/784: rename d6/da/d58/l77 to d6/d1a/d30/d46/d6f/d7c/l10b 0 2024-03-18T14:51:07.313 INFO:tasks.workunit.client.0.smithi012.stdout:2/883: rmdir d0/d6b/dbf 39 2024-03-18T14:51:07.314 INFO:tasks.workunit.client.1.smithi100.stdout:0/596: rmdir dd/d2b/d95 39 2024-03-18T14:51:07.318 INFO:tasks.workunit.client.0.smithi012.stdout:6/798: mkdir d17/d19/d27/dea 0 2024-03-18T14:51:07.320 INFO:tasks.workunit.client.0.smithi012.stdout:0/881: mkdir d1/d20/d7b/d124 0 2024-03-18T14:51:07.320 INFO:tasks.workunit.client.0.smithi012.stdout:0/882: creat d1/d20/d23/d88/f125 x:0 0 0 2024-03-18T14:51:07.327 INFO:tasks.workunit.client.0.smithi012.stdout:2/884: dwrite d0/d6b/dbf/d33/d71/de5/f117 [0,4194304] 0 2024-03-18T14:51:07.327 INFO:tasks.workunit.client.0.smithi012.stdout:2/885: creat d0/d6b/dbf/d33/d80/f11e x:0 0 0 2024-03-18T14:51:07.327 INFO:tasks.workunit.client.0.smithi012.stdout:2/886: stat d0/d6b/dbf/d33/fd5 0 2024-03-18T14:51:07.328 INFO:tasks.workunit.client.1.smithi100.stdout:2/497: sync 2024-03-18T14:51:07.329 INFO:tasks.workunit.client.0.smithi012.stdout:7/872: unlink dd/d4f/c71 0 2024-03-18T14:51:07.329 INFO:tasks.workunit.client.0.smithi012.stdout:5/847: creat d3/d84/f11c x:0 0 0 2024-03-18T14:51:07.330 INFO:tasks.workunit.client.1.smithi100.stdout:4/510: mknod d9/d3a/c9b 0 2024-03-18T14:51:07.330 INFO:tasks.workunit.client.1.smithi100.stdout:4/511: fsync d9/d27/f29 0 2024-03-18T14:51:07.332 INFO:tasks.workunit.client.0.smithi012.stdout:4/785: rename d6/d1a/d30/d46/d6f/f71 to d6/d19/f10c 0 2024-03-18T14:51:07.336 INFO:tasks.workunit.client.1.smithi100.stdout:9/529: rename da/dd/d29/d71/d72/f81 to da/dd/d29/d9e/fa7 0 2024-03-18T14:51:07.337 INFO:tasks.workunit.client.0.smithi012.stdout:0/883: dwrite d1/d20/d23/d3b/dc1/ffd [0,4194304] 0 2024-03-18T14:51:07.339 INFO:tasks.workunit.client.1.smithi100.stdout:3/564: creat d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb x:0 0 0 2024-03-18T14:51:07.340 INFO:tasks.workunit.client.1.smithi100.stdout:3/565: creat d3/dd/d21/d3e/fcc x:0 0 0 2024-03-18T14:51:07.341 INFO:tasks.workunit.client.1.smithi100.stdout:7/524: mkdir d5/d22/d24/d6e/d7a/dc2 0 2024-03-18T14:51:07.342 INFO:tasks.workunit.client.1.smithi100.stdout:5/480: creat d29/d2d/d32/d92/f9a x:0 0 0 2024-03-18T14:51:07.343 INFO:tasks.workunit.client.1.smithi100.stdout:5/481: write d29/d2d/d32/d38/f53 [279013,96172] 0 2024-03-18T14:51:07.343 INFO:tasks.workunit.client.1.smithi100.stdout:5/482: chown d29/d2d/d39/f84 93 1 2024-03-18T14:51:07.344 INFO:tasks.workunit.client.1.smithi100.stdout:6/521: creat d6/d7/df/d26/d4e/d66/d36/fa0 x:0 0 0 2024-03-18T14:51:07.344 INFO:tasks.workunit.client.1.smithi100.stdout:6/522: chown d6/d12/f43 120916250 1 2024-03-18T14:51:07.346 INFO:tasks.workunit.client.0.smithi012.stdout:2/887: mknod d0/d6c/de2/c11f 0 2024-03-18T14:51:07.348 INFO:tasks.workunit.client.1.smithi100.stdout:0/597: link dd/f85 dd/d2b/d95/d79/fd6 0 2024-03-18T14:51:07.351 INFO:tasks.workunit.client.0.smithi012.stdout:6/799: dread d17/d38/f5a [0,4194304] 0 2024-03-18T14:51:07.353 INFO:tasks.workunit.client.1.smithi100.stdout:4/512: dwrite d9/d27/d62/f6c [0,4194304] 0 2024-03-18T14:51:07.353 INFO:tasks.workunit.client.1.smithi100.stdout:4/513: chown d9/c40 97738718 1 2024-03-18T14:51:07.353 INFO:tasks.workunit.client.1.smithi100.stdout:8/566: mkdir d4/dc2 0 2024-03-18T14:51:07.354 INFO:tasks.workunit.client.0.smithi012.stdout:5/848: creat d3/d15/d42/dc6/d73/d6b/d8f/d113/f11d x:0 0 0 2024-03-18T14:51:07.356 INFO:tasks.workunit.client.1.smithi100.stdout:2/498: creat d3/d1a/d76/fb1 x:0 0 0 2024-03-18T14:51:07.357 INFO:tasks.workunit.client.0.smithi012.stdout:7/873: dwrite dd/d13/d23/fce [0,4194304] 0 2024-03-18T14:51:07.357 INFO:tasks.workunit.client.0.smithi012.stdout:7/874: getdents dd/d13/d23/d2f/d7a/db6 0 2024-03-18T14:51:07.359 INFO:tasks.workunit.client.0.smithi012.stdout:3/811: rename dc/le to dc/d30/d3a/db7/dd3/l105 0 2024-03-18T14:51:07.364 INFO:tasks.workunit.client.0.smithi012.stdout:4/786: mkdir d6/da/d89/d94/d10d 0 2024-03-18T14:51:07.364 INFO:tasks.workunit.client.0.smithi012.stdout:5/849: write d3/d29/d9d/f10d [3552472,15262] 0 2024-03-18T14:51:07.364 INFO:tasks.workunit.client.0.smithi012.stdout:5/850: dread - d3/d29/d36/d9b/fbd zero size 2024-03-18T14:51:07.368 INFO:tasks.workunit.client.0.smithi012.stdout:2/888: dwrite d0/d6b/dbf/d33/d99/fce [0,4194304] 0 2024-03-18T14:51:07.369 INFO:tasks.workunit.client.1.smithi100.stdout:5/483: dwrite d29/d2d/d32/f7c [0,4194304] 0 2024-03-18T14:51:07.369 INFO:tasks.workunit.client.1.smithi100.stdout:5/484: write d29/d2d/d39/d61/f6b [131694,57459] 0 2024-03-18T14:51:07.369 INFO:tasks.workunit.client.1.smithi100.stdout:5/485: write d29/d2d/d39/f4e [998992,11646] 0 2024-03-18T14:51:07.370 INFO:tasks.workunit.client.1.smithi100.stdout:5/486: read - d29/d2d/f97 zero size 2024-03-18T14:51:07.370 INFO:tasks.workunit.client.1.smithi100.stdout:9/530: truncate f2 4031404 0 2024-03-18T14:51:07.373 INFO:tasks.workunit.client.0.smithi012.stdout:0/884: creat d1/d20/d4e/d54/d67/dbc/f126 x:0 0 0 2024-03-18T14:51:07.373 INFO:tasks.workunit.client.0.smithi012.stdout:0/885: fsync d1/d20/d4e/d54/d8a/fb9 0 2024-03-18T14:51:07.373 INFO:tasks.workunit.client.0.smithi012.stdout:0/886: creat d1/d20/d4e/d54/f127 x:0 0 0 2024-03-18T14:51:07.374 INFO:tasks.workunit.client.1.smithi100.stdout:3/566: mkdir d3/dd/d21/dcd 0 2024-03-18T14:51:07.376 INFO:tasks.workunit.client.0.smithi012.stdout:6/800: creat d17/d38/d51/d72/feb x:0 0 0 2024-03-18T14:51:07.378 INFO:tasks.workunit.client.1.smithi100.stdout:5/487: dread d29/d2d/d32/f7c [0,4194304] 0 2024-03-18T14:51:07.378 INFO:tasks.workunit.client.1.smithi100.stdout:5/488: chown d29/d2d/d32 5 1 2024-03-18T14:51:07.378 INFO:tasks.workunit.client.1.smithi100.stdout:7/525: truncate d5/d14/d16/d2c/f5b 2845506 0 2024-03-18T14:51:07.380 INFO:tasks.workunit.client.1.smithi100.stdout:5/489: write d29/d2d/d39/d61/f89 [512413,124363] 0 2024-03-18T14:51:07.380 INFO:tasks.workunit.client.1.smithi100.stdout:5/490: chown d29/d2d/d39/d60/f70 66640229 1 2024-03-18T14:51:07.381 INFO:tasks.workunit.client.1.smithi100.stdout:6/523: mkdir d6/da1 0 2024-03-18T14:51:07.384 INFO:tasks.workunit.client.0.smithi012.stdout:0/887: read d1/d16/d6e/f9b [540596,125624] 0 2024-03-18T14:51:07.385 INFO:tasks.workunit.client.1.smithi100.stdout:0/598: mknod dd/d31/d91/d56/d62/cd7 0 2024-03-18T14:51:07.387 INFO:tasks.workunit.client.1.smithi100.stdout:6/524: dread d6/d7/df/d72/d48/f4a [4194304,4194304] 0 2024-03-18T14:51:07.390 INFO:tasks.workunit.client.1.smithi100.stdout:9/531: dwrite da/dd/d3c/d6b/f84 [0,4194304] 0 2024-03-18T14:51:07.390 INFO:tasks.workunit.client.1.smithi100.stdout:9/532: chown da/dd/fa1 151 1 2024-03-18T14:51:07.393 INFO:tasks.workunit.client.1.smithi100.stdout:4/514: link d9/f22 d9/d3a/f9c 0 2024-03-18T14:51:07.395 INFO:tasks.workunit.client.0.smithi012.stdout:3/812: creat dc/d30/d35/f106 x:0 0 0 2024-03-18T14:51:07.397 INFO:tasks.workunit.client.1.smithi100.stdout:9/533: write da/dd/d3c/d4b/f58 [78072,129710] 0 2024-03-18T14:51:07.397 INFO:tasks.workunit.client.0.smithi012.stdout:4/787: rename d6/da/d89/d95/ld0 to d6/da/d89/l10e 0 2024-03-18T14:51:07.399 INFO:tasks.workunit.client.1.smithi100.stdout:2/499: rename d3/da/de/d20/d82/d95/f70 to d3/d6/d5f/d6a/fb2 0 2024-03-18T14:51:07.400 INFO:tasks.workunit.client.0.smithi012.stdout:5/851: rmdir d3/d15/d42/dc6/d65 39 2024-03-18T14:51:07.411 INFO:tasks.workunit.client.1.smithi100.stdout:3/567: mknod d3/d16/d13/d1f/d3f/d4f/d60/cce 0 2024-03-18T14:51:07.414 INFO:tasks.workunit.client.0.smithi012.stdout:0/888: creat d1/d20/d23/d6f/f128 x:0 0 0 2024-03-18T14:51:07.415 INFO:tasks.workunit.client.1.smithi100.stdout:7/526: mkdir d5/d14/d16/d2e/d62/dc3 0 2024-03-18T14:51:07.416 INFO:tasks.workunit.client.0.smithi012.stdout:0/889: read d1/d16/d19/f7d [565565,13537] 0 2024-03-18T14:51:07.416 INFO:tasks.workunit.client.0.smithi012.stdout:0/890: fsync d1/d16/d19/f21 0 2024-03-18T14:51:07.419 INFO:tasks.workunit.client.1.smithi100.stdout:5/491: rename d29/f5a to d29/f9b 0 2024-03-18T14:51:07.419 INFO:tasks.workunit.client.1.smithi100.stdout:5/492: readlink d29/d2d/d39/l51 0 2024-03-18T14:51:07.419 INFO:tasks.workunit.client.1.smithi100.stdout:5/493: creat d29/d4a/f9c x:0 0 0 2024-03-18T14:51:07.421 INFO:tasks.workunit.client.1.smithi100.stdout:0/599: truncate fa 561520 0 2024-03-18T14:51:07.421 INFO:tasks.workunit.client.1.smithi100.stdout:0/600: chown dd/d20/c51 823181 1 2024-03-18T14:51:07.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/525: stat d6/d7/df/d26/d4e/d66/d36/d4f/c5a 0 2024-03-18T14:51:07.423 INFO:tasks.workunit.client.1.smithi100.stdout:6/526: truncate d6/d7/df/d26/d4e/f82 903848 0 2024-03-18T14:51:07.426 INFO:tasks.workunit.client.1.smithi100.stdout:1/510: sync 2024-03-18T14:51:07.426 INFO:tasks.workunit.client.1.smithi100.stdout:1/511: chown d8/d1e/f28 6231010 1 2024-03-18T14:51:07.426 INFO:tasks.workunit.client.1.smithi100.stdout:1/512: chown d8/d1e/d11/f77 83791 1 2024-03-18T14:51:07.473 INFO:tasks.workunit.client.0.smithi012.stdout:1/798: sync 2024-03-18T14:51:07.478 INFO:tasks.workunit.client.0.smithi012.stdout:9/920: sync 2024-03-18T14:51:07.479 INFO:tasks.workunit.client.1.smithi100.stdout:4/515: link d9/d17/l49 d9/d27/d62/d7b/l9d 0 2024-03-18T14:51:07.479 INFO:tasks.workunit.client.1.smithi100.stdout:4/516: read - d9/d27/d62/d7b/f93 zero size 2024-03-18T14:51:07.479 INFO:tasks.workunit.client.1.smithi100.stdout:4/517: fdatasync d9/d27/d62/f73 0 2024-03-18T14:51:07.479 INFO:tasks.workunit.client.1.smithi100.stdout:1/513: dread d8/f69 [0,4194304] 0 2024-03-18T14:51:07.480 INFO:tasks.workunit.client.1.smithi100.stdout:9/534: symlink da/d46/d4d/d67/la8 0 2024-03-18T14:51:07.481 INFO:tasks.workunit.client.0.smithi012.stdout:7/875: rmdir dd/d13 39 2024-03-18T14:51:07.482 INFO:tasks.workunit.client.1.smithi100.stdout:2/500: mknod d3/d6/d29/cb3 0 2024-03-18T14:51:07.482 INFO:tasks.workunit.client.1.smithi100.stdout:2/501: write d3/d6/f9 [1663982,62439] 0 2024-03-18T14:51:07.547 INFO:tasks.workunit.client.1.smithi100.stdout:3/568: dwrite d3/d16/d13/d1f/f59 [4194304,4194304] 0 2024-03-18T14:51:07.547 INFO:tasks.workunit.client.1.smithi100.stdout:8/567: dwrite d4/dc/d1f/d59/d6c/f98 [0,4194304] 0 2024-03-18T14:51:07.547 INFO:tasks.workunit.client.1.smithi100.stdout:8/568: creat d4/dc/d31/d66/fc3 x:0 0 0 2024-03-18T14:51:07.547 INFO:tasks.workunit.client.0.smithi012.stdout:3/813: creat dc/d3c/d4c/d57/d103/d9a/dd7/f107 x:0 0 0 2024-03-18T14:51:07.548 INFO:tasks.workunit.client.1.smithi100.stdout:7/527: symlink d5/d14/d16/d44/lc4 0 2024-03-18T14:51:07.551 INFO:tasks.workunit.client.0.smithi012.stdout:5/852: rename d3/d15/d42/dc6/d73/d6b/df3/la3 to d3/dad/df2/l11e 0 2024-03-18T14:51:07.551 INFO:tasks.workunit.client.0.smithi012.stdout:5/853: write d3/d15/d42/dc6/d73/d6b/ff4 [4661280,118459] 0 2024-03-18T14:51:07.551 INFO:tasks.workunit.client.0.smithi012.stdout:5/854: chown d3/d15/d42/dc6/d65/fa1 1962485214 1 2024-03-18T14:51:07.552 INFO:tasks.workunit.client.1.smithi100.stdout:5/494: mknod d29/d2d/d39/d71/d96/c9d 0 2024-03-18T14:51:07.552 INFO:tasks.workunit.client.0.smithi012.stdout:2/889: getdents d0/d3c/d40 0 2024-03-18T14:51:07.554 INFO:tasks.workunit.client.1.smithi100.stdout:0/601: getdents dd/d31/d91 0 2024-03-18T14:51:07.554 INFO:tasks.workunit.client.1.smithi100.stdout:0/602: chown dd/d20 5885365 1 2024-03-18T14:51:07.554 INFO:tasks.workunit.client.1.smithi100.stdout:6/527: dwrite d6/d7/df/d26/d4e/d66/f68 [0,4194304] 0 2024-03-18T14:51:07.554 INFO:tasks.workunit.client.1.smithi100.stdout:6/528: chown d6/d7/df/d26/d4e/d66/d36/l49 14942 1 2024-03-18T14:51:07.554 INFO:tasks.workunit.client.1.smithi100.stdout:6/529: write d6/d7/df/d72/d48/f4a [2344925,57752] 0 2024-03-18T14:51:07.555 INFO:tasks.workunit.client.0.smithi012.stdout:6/801: getdents d17/d26/d36/d4d/d82/dcd 0 2024-03-18T14:51:07.557 INFO:tasks.workunit.client.1.smithi100.stdout:3/569: write d3/dd/f28 [2168195,85814] 0 2024-03-18T14:51:07.557 INFO:tasks.workunit.client.1.smithi100.stdout:3/570: write d3/d16/f84 [555133,91375] 0 2024-03-18T14:51:07.559 INFO:tasks.workunit.client.0.smithi012.stdout:9/921: dwrite d3/d49/fd4 [0,4194304] 0 2024-03-18T14:51:07.559 INFO:tasks.workunit.client.0.smithi012.stdout:9/922: truncate d3/de6/dbf/d66/d21/d35/fb9 1637728 0 2024-03-18T14:51:07.561 INFO:tasks.workunit.client.0.smithi012.stdout:7/876: rmdir dd/d13/d103 39 2024-03-18T14:51:07.563 INFO:tasks.workunit.client.1.smithi100.stdout:1/514: symlink d8/d1e/d11/d8a/l98 0 2024-03-18T14:51:07.563 INFO:tasks.workunit.client.1.smithi100.stdout:1/515: chown d8/d1e/d11/d6d/f87 11685933 1 2024-03-18T14:51:07.564 INFO:tasks.workunit.client.0.smithi012.stdout:3/814: dwrite f6 [4194304,4194304] 0 2024-03-18T14:51:07.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/815: chown dc/d3c/d59 4126962 1 2024-03-18T14:51:07.565 INFO:tasks.workunit.client.0.smithi012.stdout:3/816: chown dc/d3c/d4c/d57/d103/d9a/d64/fd4 4008 1 2024-03-18T14:51:07.565 INFO:tasks.workunit.client.1.smithi100.stdout:5/495: dread d29/d2d/d39/d61/f89 [0,4194304] 0 2024-03-18T14:51:07.567 INFO:tasks.workunit.client.1.smithi100.stdout:9/535: mkdir da/d88/da9 0 2024-03-18T14:51:07.567 INFO:tasks.workunit.client.1.smithi100.stdout:9/536: creat da/dd/d29/d71/d72/faa x:0 0 0 2024-03-18T14:51:07.570 INFO:tasks.workunit.client.1.smithi100.stdout:2/502: link d3/d6/f67 d3/d6/d29/d36/d49/fb4 0 2024-03-18T14:51:07.570 INFO:tasks.workunit.client.1.smithi100.stdout:2/503: readlink d3/d1a/d76/lae 0 2024-03-18T14:51:07.570 INFO:tasks.workunit.client.1.smithi100.stdout:2/504: write d3/d13/d40/fa7 [381947,59970] 0 2024-03-18T14:51:07.573 INFO:tasks.workunit.client.0.smithi012.stdout:2/890: truncate d0/d3c/d40/f5e 1248249 0 2024-03-18T14:51:07.573 INFO:tasks.workunit.client.0.smithi012.stdout:2/891: readlink d0/l30 0 2024-03-18T14:51:07.577 INFO:tasks.workunit.client.0.smithi012.stdout:6/802: rename d17/d19/c25 to d17/d38/d51/d72/cec 0 2024-03-18T14:51:07.577 INFO:tasks.workunit.client.0.smithi012.stdout:4/788: write d6/d1a/d30/d46/f48 [236421,66743] 0 2024-03-18T14:51:07.584 INFO:tasks.workunit.client.0.smithi012.stdout:9/923: creat d3/de6/dbf/d66/d69/f12b x:0 0 0 2024-03-18T14:51:07.584 INFO:tasks.workunit.client.0.smithi012.stdout:9/924: dread - d3/de6/dbf/d66/d21/f8e zero size 2024-03-18T14:51:07.585 INFO:tasks.workunit.client.1.smithi100.stdout:8/569: truncate d4/dc/dd/d48/fa6 684867 0 2024-03-18T14:51:07.586 INFO:tasks.workunit.client.0.smithi012.stdout:1/799: getdents d8/db/d10/d31/d43/d5e/d65/d70/dc7 0 2024-03-18T14:51:07.586 INFO:tasks.workunit.client.0.smithi012.stdout:1/800: fdatasync f5 0 2024-03-18T14:51:07.589 INFO:tasks.workunit.client.1.smithi100.stdout:7/528: mkdir d5/d14/d16/d2c/d86/d85/d87/dc5 0 2024-03-18T14:51:07.589 INFO:tasks.workunit.client.1.smithi100.stdout:7/529: dread - d5/d22/d24/dac/fb6 zero size 2024-03-18T14:51:07.590 INFO:tasks.workunit.client.1.smithi100.stdout:7/530: chown d5/d22/d3b 3501 1 2024-03-18T14:51:07.590 INFO:tasks.workunit.client.1.smithi100.stdout:7/531: chown d5/d22/d24/d6e/f84 1 1 2024-03-18T14:51:07.590 INFO:tasks.workunit.client.1.smithi100.stdout:7/532: getdents d5/d14/d16/d2e/d62/dc3 0 2024-03-18T14:51:07.590 INFO:tasks.workunit.client.1.smithi100.stdout:7/533: getdents d5/d14/d16/d2e/d62/dc3 0 2024-03-18T14:51:07.591 INFO:tasks.workunit.client.0.smithi012.stdout:7/877: creat dd/d13/d23/d76/d6f/db3/dc7/f12e x:0 0 0 2024-03-18T14:51:07.592 INFO:tasks.workunit.client.0.smithi012.stdout:2/892: dwrite d0/da2/d59/f69 [4194304,4194304] 0 2024-03-18T14:51:07.595 INFO:tasks.workunit.client.0.smithi012.stdout:3/817: rename dc/d3c/d4c/d57/d103/cbd to dc/d3c/d4c/d57/d103/d9a/c108 0 2024-03-18T14:51:07.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/603: link dd/d31/d91/d7b/fad dd/d31/d91/d56/d62/fd8 0 2024-03-18T14:51:07.600 INFO:tasks.workunit.client.0.smithi012.stdout:6/803: getdents d17 0 2024-03-18T14:51:07.600 INFO:tasks.workunit.client.0.smithi012.stdout:6/804: read - d17/d19/d27/d58/f86 zero size 2024-03-18T14:51:07.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/530: mkdir d6/d12/da2 0 2024-03-18T14:51:07.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/531: write d6/d12/f89 [614977,121847] 0 2024-03-18T14:51:07.604 INFO:tasks.workunit.client.0.smithi012.stdout:4/789: link d6/d27/ldf d6/d19/d6c/d96/l10f 0 2024-03-18T14:51:07.606 INFO:tasks.workunit.client.1.smithi100.stdout:3/571: unlink d3/d16/d13/d17/f2b 0 2024-03-18T14:51:07.607 INFO:tasks.workunit.client.0.smithi012.stdout:9/925: mkdir d3/de6/dbf/d77/d12c 0 2024-03-18T14:51:07.608 INFO:tasks.workunit.client.1.smithi100.stdout:4/518: getdents d9/d13/d39/d76 0 2024-03-18T14:51:07.612 INFO:tasks.workunit.client.0.smithi012.stdout:2/893: creat d0/f120 x:0 0 0 2024-03-18T14:51:07.612 INFO:tasks.workunit.client.0.smithi012.stdout:2/894: write d0/da2/f10e [876009,43739] 0 2024-03-18T14:51:07.612 INFO:tasks.workunit.client.0.smithi012.stdout:2/895: chown d0/l4a 4438278 1 2024-03-18T14:51:07.613 INFO:tasks.workunit.client.0.smithi012.stdout:2/896: fdatasync d0/d6b/dbf/d33/d80/de1/f10b 0 2024-03-18T14:51:07.613 INFO:tasks.workunit.client.1.smithi100.stdout:7/534: dwrite d5/d14/d16/d44/fa7 [0,4194304] 0 2024-03-18T14:51:07.613 INFO:tasks.workunit.client.1.smithi100.stdout:7/535: truncate d5/d22/d24/f91 1090973 0 2024-03-18T14:51:07.613 INFO:tasks.workunit.client.1.smithi100.stdout:7/536: creat d5/d22/d3b/d9f/da0/fc6 x:0 0 0 2024-03-18T14:51:07.615 INFO:tasks.workunit.client.1.smithi100.stdout:1/516: rename d8/d2a/d37/d50/d6f/d71/c7b to d8/d1e/d11/d6d/d59/d93/c99 0 2024-03-18T14:51:07.615 INFO:tasks.workunit.client.1.smithi100.stdout:1/517: stat d8/d2a/d37/d50/d6f/d71 0 2024-03-18T14:51:07.615 INFO:tasks.workunit.client.1.smithi100.stdout:1/518: write d8/d1e/d11/d6d/f87 [780798,12545] 0 2024-03-18T14:51:07.617 INFO:tasks.workunit.client.0.smithi012.stdout:1/801: dwrite d8/db/d10/d31/d43/d5e/d65/d84/fde [0,4194304] 0 2024-03-18T14:51:07.617 INFO:tasks.workunit.client.0.smithi012.stdout:3/818: mkdir dc/d3c/d72/df3/d109 0 2024-03-18T14:51:07.619 INFO:tasks.workunit.client.1.smithi100.stdout:5/496: mkdir d29/d2d/d32/d38/d9e 0 2024-03-18T14:51:07.619 INFO:tasks.workunit.client.1.smithi100.stdout:5/497: chown d29/d4a/f8e 0 1 2024-03-18T14:51:07.621 INFO:tasks.workunit.client.0.smithi012.stdout:4/790: mknod d6/d1a/d30/d46/d6f/c110 0 2024-03-18T14:51:07.622 INFO:tasks.workunit.client.1.smithi100.stdout:9/537: rmdir da/d46/d8b 39 2024-03-18T14:51:07.622 INFO:tasks.workunit.client.1.smithi100.stdout:3/572: write d3/dd/d49/f70 [2192242,62129] 0 2024-03-18T14:51:07.623 INFO:tasks.workunit.client.0.smithi012.stdout:2/897: mknod d0/d107/c121 0 2024-03-18T14:51:07.626 INFO:tasks.workunit.client.1.smithi100.stdout:2/505: creat d3/d13/d4f/fb5 x:0 0 0 2024-03-18T14:51:07.628 INFO:tasks.workunit.client.0.smithi012.stdout:1/802: creat d8/db/d10/d16/da1/d2e/f100 x:0 0 0 2024-03-18T14:51:07.628 INFO:tasks.workunit.client.0.smithi012.stdout:1/803: truncate d8/db/d10/d16/da1/d2e/d9c/fea 86497 0 2024-03-18T14:51:07.630 INFO:tasks.workunit.client.1.smithi100.stdout:8/570: symlink d4/dc/dd/de/d21/d23/d47/d54/lc4 0 2024-03-18T14:51:07.632 INFO:tasks.workunit.client.0.smithi012.stdout:3/819: creat dc/d3c/d4c/d57/d103/d9a/d54/f10a x:0 0 0 2024-03-18T14:51:07.632 INFO:tasks.workunit.client.0.smithi012.stdout:3/820: read - dc/d30/d3a/fd9 zero size 2024-03-18T14:51:07.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/538: dread da/f16 [0,4194304] 0 2024-03-18T14:51:07.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/539: chown da/d51/d55/f5c 561929 1 2024-03-18T14:51:07.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/540: write da/fc [3281126,17499] 0 2024-03-18T14:51:07.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/541: truncate da/d2c/d2f/f5b 914790 0 2024-03-18T14:51:07.634 INFO:tasks.workunit.client.1.smithi100.stdout:0/604: symlink dd/d2b/d95/d79/d7d/da9/dac/ld9 0 2024-03-18T14:51:07.636 INFO:tasks.workunit.client.1.smithi100.stdout:9/542: dread da/dd/d29/f2e [0,4194304] 0 2024-03-18T14:51:07.637 INFO:tasks.workunit.client.1.smithi100.stdout:9/543: creat da/d46/d4d/d67/fab x:0 0 0 2024-03-18T14:51:07.642 INFO:tasks.workunit.client.1.smithi100.stdout:4/519: truncate d9/d13/d25/f48 647636 0 2024-03-18T14:51:07.642 INFO:tasks.workunit.client.1.smithi100.stdout:4/520: write d9/d13/d25/f4d [154444,101965] 0 2024-03-18T14:51:07.644 INFO:tasks.workunit.client.0.smithi012.stdout:1/804: mknod d8/db/dbe/c101 0 2024-03-18T14:51:07.647 INFO:tasks.workunit.client.0.smithi012.stdout:4/791: rmdir d6/d1a/d30/d46/d6f/d8b 39 2024-03-18T14:51:07.648 INFO:tasks.workunit.client.1.smithi100.stdout:6/532: dwrite d6/f2f [4194304,4194304] 0 2024-03-18T14:51:07.650 INFO:tasks.workunit.client.0.smithi012.stdout:2/898: getdents d0/d6b/dbf/d33 0 2024-03-18T14:51:07.653 INFO:tasks.workunit.client.0.smithi012.stdout:1/805: truncate d8/d1e/d27/f38 3061289 0 2024-03-18T14:51:07.656 INFO:tasks.workunit.client.0.smithi012.stdout:2/899: getdents d0/d6b/dbf/d33/d99/daa 0 2024-03-18T14:51:07.659 INFO:tasks.workunit.client.0.smithi012.stdout:2/900: mkdir d0/d6b/dbf/d33/d77/d10a/d122 0 2024-03-18T14:51:07.661 INFO:tasks.workunit.client.0.smithi012.stdout:1/806: dread d8/d1e/f7c [0,4194304] 0 2024-03-18T14:51:07.661 INFO:tasks.workunit.client.1.smithi100.stdout:3/573: link d3/dd/d21/d3e/fcc d3/d16/dca/fcf 0 2024-03-18T14:51:07.663 INFO:tasks.workunit.client.1.smithi100.stdout:3/574: dread d3/dd/d21/d3e/fbb [0,4194304] 0 2024-03-18T14:51:07.663 INFO:tasks.workunit.client.1.smithi100.stdout:3/575: dread - d3/fc2 zero size 2024-03-18T14:51:07.663 INFO:tasks.workunit.client.1.smithi100.stdout:3/576: dread - d3/dd/d21/d3e/f80 zero size 2024-03-18T14:51:07.663 INFO:tasks.workunit.client.1.smithi100.stdout:2/506: rename d3/da/de/d20/l34 to d3/da/de/d48/lb6 0 2024-03-18T14:51:07.664 INFO:tasks.workunit.client.0.smithi012.stdout:2/901: symlink d0/d6b/d9f/l123 0 2024-03-18T14:51:07.665 INFO:tasks.workunit.client.1.smithi100.stdout:8/571: symlink d4/dc/d31/d85/d8e/lc5 0 2024-03-18T14:51:07.665 INFO:tasks.workunit.client.1.smithi100.stdout:8/572: truncate d4/dc/dd/de/d19/d37/f61 87679 0 2024-03-18T14:51:07.667 INFO:tasks.workunit.client.1.smithi100.stdout:7/537: dwrite d5/d14/d16/d2e/d62/d66/f54 [0,4194304] 0 2024-03-18T14:51:07.667 INFO:tasks.workunit.client.0.smithi012.stdout:9/926: dwrite d3/de6/dbf/d66/d69/d7f/d8c/dac/fcb [0,4194304] 0 2024-03-18T14:51:07.668 INFO:tasks.workunit.client.1.smithi100.stdout:5/498: dwrite d29/f91 [0,4194304] 0 2024-03-18T14:51:07.668 INFO:tasks.workunit.client.1.smithi100.stdout:5/499: write d29/f37 [4303270,16297] 0 2024-03-18T14:51:07.668 INFO:tasks.workunit.client.1.smithi100.stdout:5/500: creat d29/d2d/d39/d83/f9f x:0 0 0 2024-03-18T14:51:07.669 INFO:tasks.workunit.client.0.smithi012.stdout:2/902: creat d0/d6b/dbf/d33/dac/dd0/f124 x:0 0 0 2024-03-18T14:51:07.671 INFO:tasks.workunit.client.0.smithi012.stdout:3/821: dwrite dc/d3c/f74 [0,4194304] 0 2024-03-18T14:51:07.672 INFO:tasks.workunit.client.0.smithi012.stdout:4/792: dwrite d6/d1a/d30/d46/f48 [0,4194304] 0 2024-03-18T14:51:07.675 INFO:tasks.workunit.client.1.smithi100.stdout:4/521: creat d9/d13/d5a/d5b/f9e x:0 0 0 2024-03-18T14:51:07.676 INFO:tasks.workunit.client.1.smithi100.stdout:2/507: dread d3/d6/fa9 [0,4194304] 0 2024-03-18T14:51:07.676 INFO:tasks.workunit.client.1.smithi100.stdout:2/508: stat d3/fc 0 2024-03-18T14:51:07.681 INFO:tasks.workunit.client.1.smithi100.stdout:1/519: symlink d8/d1e/d11/d27/d3f/d76/l9a 0 2024-03-18T14:51:07.686 INFO:tasks.workunit.client.1.smithi100.stdout:3/577: unlink d3/d16/d13/d17/la4 0 2024-03-18T14:51:07.689 INFO:tasks.workunit.client.1.smithi100.stdout:0/605: rename dd/d31/d91/d7b/l81 to dd/d20/db5/lda 0 2024-03-18T14:51:07.690 INFO:tasks.workunit.client.1.smithi100.stdout:5/501: dwrite d29/d4a/f9c [0,4194304] 0 2024-03-18T14:51:07.690 INFO:tasks.workunit.client.1.smithi100.stdout:5/502: creat d29/fa0 x:0 0 0 2024-03-18T14:51:07.691 INFO:tasks.workunit.client.1.smithi100.stdout:8/573: mkdir d4/dc/dc6 0 2024-03-18T14:51:07.693 INFO:tasks.workunit.client.0.smithi012.stdout:2/903: write d0/d6b/dbf/d33/fd5 [3257979,67744] 0 2024-03-18T14:51:07.693 INFO:tasks.workunit.client.0.smithi012.stdout:2/904: write d0/d6b/d9f/fc5 [531474,92273] 0 2024-03-18T14:51:07.699 INFO:tasks.workunit.client.1.smithi100.stdout:5/503: write f23 [573459,28279] 0 2024-03-18T14:51:07.699 INFO:tasks.workunit.client.1.smithi100.stdout:5/504: write f12 [5185216,77433] 0 2024-03-18T14:51:07.701 INFO:tasks.workunit.client.1.smithi100.stdout:0/606: dread dd/f55 [0,4194304] 0 2024-03-18T14:51:07.703 INFO:tasks.workunit.client.0.smithi012.stdout:9/927: dwrite d3/de6/dbf/d77/d52/d99/da0/ddf/dee/f116 [0,4194304] 0 2024-03-18T14:51:07.704 INFO:tasks.workunit.client.1.smithi100.stdout:2/509: dwrite d3/da/de/d20/f90 [0,4194304] 0 2024-03-18T14:51:07.704 INFO:tasks.workunit.client.0.smithi012.stdout:1/807: truncate d8/db/d10/d31/d43/d5e/d65/d84/fde 1791605 0 2024-03-18T14:51:07.707 INFO:tasks.workunit.client.0.smithi012.stdout:2/905: dread d0/d6b/dbf/d33/d99/fce [0,4194304] 0 2024-03-18T14:51:07.709 INFO:tasks.workunit.client.0.smithi012.stdout:3/822: dwrite dc/fa4 [0,4194304] 0 2024-03-18T14:51:07.723 INFO:tasks.workunit.client.0.smithi012.stdout:4/793: rmdir d6/db6 39 2024-03-18T14:51:07.730 INFO:tasks.workunit.client.0.smithi012.stdout:1/808: mknod d8/db/d10/d16/da1/c102 0 2024-03-18T14:51:07.732 INFO:tasks.workunit.client.1.smithi100.stdout:4/522: symlink d9/d17/l9f 0 2024-03-18T14:51:07.732 INFO:tasks.workunit.client.1.smithi100.stdout:4/523: readlink d9/d27/d62/d7b/l7e 0 2024-03-18T14:51:07.732 INFO:tasks.workunit.client.1.smithi100.stdout:4/524: fdatasync d9/d27/d62/f6c 0 2024-03-18T14:51:07.733 INFO:tasks.workunit.client.0.smithi012.stdout:9/928: dwrite d3/de6/dbf/d77/d52/fc0 [0,4194304] 0 2024-03-18T14:51:07.733 INFO:tasks.workunit.client.0.smithi012.stdout:2/906: dwrite d0/d6b/dbf/d33/d77/df5/d103/fc4 [0,4194304] 0 2024-03-18T14:51:07.734 INFO:tasks.workunit.client.0.smithi012.stdout:3/823: dwrite dc/d30/d35/f66 [0,4194304] 0 2024-03-18T14:51:07.736 INFO:tasks.workunit.client.0.smithi012.stdout:1/809: mknod d8/db/d10/d16/da1/d2e/d9c/d9d/c103 0 2024-03-18T14:51:07.736 INFO:tasks.workunit.client.0.smithi012.stdout:1/810: stat d8/d1e/d27/d44/cb5 0 2024-03-18T14:51:07.740 INFO:tasks.workunit.client.0.smithi012.stdout:4/794: mknod d6/d1a/c111 0 2024-03-18T14:51:07.740 INFO:tasks.workunit.client.0.smithi012.stdout:4/795: write d6/d19/d6c/f101 [628234,87747] 0 2024-03-18T14:51:07.740 INFO:tasks.workunit.client.0.smithi012.stdout:4/796: write d6/d1a/d98/daf/db5/fe4 [356707,6963] 0 2024-03-18T14:51:07.740 INFO:tasks.workunit.client.0.smithi012.stdout:4/797: chown d6 229 1 2024-03-18T14:51:07.743 INFO:tasks.workunit.client.0.smithi012.stdout:9/929: creat d3/de6/dbf/d77/d52/d95/da3/f12d x:0 0 0 2024-03-18T14:51:07.746 INFO:tasks.workunit.client.0.smithi012.stdout:1/811: dread d8/db/d10/d16/d6b/d7e/fd6 [0,4194304] 0 2024-03-18T14:51:07.754 INFO:tasks.workunit.client.0.smithi012.stdout:4/798: creat d6/d1a/d30/d46/d6f/d8b/f112 x:0 0 0 2024-03-18T14:51:07.755 INFO:tasks.workunit.client.1.smithi100.stdout:1/520: mkdir d8/d2a/d9b 0 2024-03-18T14:51:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:3/824: dwrite dc/d3c/d59/db2/fb5 [0,4194304] 0 2024-03-18T14:51:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:9/930: creat d3/d9/d5c/f12e x:0 0 0 2024-03-18T14:51:07.759 INFO:tasks.workunit.client.1.smithi100.stdout:3/578: mkdir d3/dd/dd0 0 2024-03-18T14:51:07.759 INFO:tasks.workunit.client.1.smithi100.stdout:3/579: truncate d3/d16/d13/f3b 130601 0 2024-03-18T14:51:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:1/812: symlink d8/db/d10/d31/d81/l104 0 2024-03-18T14:51:07.764 INFO:tasks.workunit.client.1.smithi100.stdout:7/538: rename d5/d14/d16/d2c/d86/d85/c8f to d5/d22/d3b/d9b/cc7 0 2024-03-18T14:51:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:9/931: mkdir d3/de6/dbf/d77/d52/d78/d12f 0 2024-03-18T14:51:07.771 INFO:tasks.workunit.client.1.smithi100.stdout:8/574: symlink d4/d3c/lc7 0 2024-03-18T14:51:07.771 INFO:tasks.workunit.client.1.smithi100.stdout:8/575: chown c2 1 1 2024-03-18T14:51:07.776 INFO:tasks.workunit.client.1.smithi100.stdout:5/505: link d29/d4a/f80 d29/d2d/d39/d65/d99/fa1 0 2024-03-18T14:51:07.777 INFO:tasks.workunit.client.1.smithi100.stdout:5/506: chown d29/d2d/d39/f46 263833000 1 2024-03-18T14:51:07.778 INFO:tasks.workunit.client.0.smithi012.stdout:3/825: dwrite dc/d3c/d4c/d57/d103/d9a/d54/f10a [0,4194304] 0 2024-03-18T14:51:07.782 INFO:tasks.workunit.client.0.smithi012.stdout:9/932: truncate d3/d9/fa2 3831276 0 2024-03-18T14:51:07.782 INFO:tasks.workunit.client.1.smithi100.stdout:2/510: creat d3/d13/d40/d7d/fb7 x:0 0 0 2024-03-18T14:51:07.784 INFO:tasks.workunit.client.0.smithi012.stdout:4/799: dwrite d6/d1a/f31 [0,4194304] 0 2024-03-18T14:51:07.785 INFO:tasks.workunit.client.1.smithi100.stdout:4/525: truncate d9/d17/f34 6796330 0 2024-03-18T14:51:07.785 INFO:tasks.workunit.client.1.smithi100.stdout:4/526: chown d9/d17/f45 368 1 2024-03-18T14:51:07.788 INFO:tasks.workunit.client.0.smithi012.stdout:3/826: rename dc/d3c/d4c/d57/d103/d9a/fd5 to dc/de6/f10b 0 2024-03-18T14:51:07.789 INFO:tasks.workunit.client.1.smithi100.stdout:5/507: write d29/d2d/d32/f5c [488329,93076] 0 2024-03-18T14:51:07.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/933: symlink d3/de6/dbf/d77/da8/l130 0 2024-03-18T14:51:07.792 INFO:tasks.workunit.client.0.smithi012.stdout:9/934: stat d3/de6/dbf/d77/da8/le2 0 2024-03-18T14:51:07.794 INFO:tasks.workunit.client.1.smithi100.stdout:0/607: rename f2 to dd/d20/fdb 0 2024-03-18T14:51:07.795 INFO:tasks.workunit.client.0.smithi012.stdout:9/935: dread d3/de6/dbf/d66/d6e/dc6/ff2 [0,4194304] 0 2024-03-18T14:51:07.795 INFO:tasks.workunit.client.0.smithi012.stdout:9/936: truncate d3/d9/f1c 385454 0 2024-03-18T14:51:07.797 INFO:tasks.workunit.client.1.smithi100.stdout:8/576: mknod d4/d5a/cc8 0 2024-03-18T14:51:07.797 INFO:tasks.workunit.client.0.smithi012.stdout:3/827: dread dc/d30/d35/f66 [0,4194304] 0 2024-03-18T14:51:07.803 INFO:tasks.workunit.client.1.smithi100.stdout:8/577: dread d4/dc/dd/de/d21/d23/f62 [0,4194304] 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/937: creat d3/de6/f131 x:0 0 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/938: creat d3/de6/dbf/d77/d52/d99/da0/ddf/dee/f132 x:0 0 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/939: fdatasync d3/f12 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/940: fdatasync d3/de6/dbf/d66/d69/d7f/d8c/f10e 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/941: write d3/de6/dbf/d66/d69/d7f/d8c/f10e [15623,35945] 0 2024-03-18T14:51:07.804 INFO:tasks.workunit.client.0.smithi012.stdout:9/942: chown d3/de6/dbf 4226978 1 2024-03-18T14:51:07.806 INFO:tasks.workunit.client.1.smithi100.stdout:2/511: getdents d3/da 0 2024-03-18T14:51:07.808 INFO:tasks.workunit.client.1.smithi100.stdout:3/580: getdents d3/d16/d13/d1f/d37 0 2024-03-18T14:51:07.815 INFO:tasks.workunit.client.1.smithi100.stdout:5/508: unlink f20 0 2024-03-18T14:51:07.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/608: mkdir dd/d20/ddc 0 2024-03-18T14:51:07.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/609: chown dd/d31/d91/d56/d71/cd1 985129591 1 2024-03-18T14:51:07.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/610: write dd/f42 [676748,33678] 0 2024-03-18T14:51:07.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/611: dread - dd/d2b/d95/d79/d7d/f9d zero size 2024-03-18T14:51:07.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/612: stat dd/d31/d91/d56/d59/c96 0 2024-03-18T14:51:07.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/539: rename d5/d30/f3c to d5/d22/d3b/d9f/fc8 0 2024-03-18T14:51:07.822 INFO:tasks.workunit.client.1.smithi100.stdout:8/578: rmdir d4/d5a/dbb 39 2024-03-18T14:51:07.825 INFO:tasks.workunit.client.1.smithi100.stdout:7/540: dread d5/d22/d24/d6e/d7a/f89 [0,4194304] 0 2024-03-18T14:51:07.825 INFO:tasks.workunit.client.1.smithi100.stdout:7/541: dread - d5/d30/d4d/d5a/f82 zero size 2024-03-18T14:51:07.827 INFO:tasks.workunit.client.1.smithi100.stdout:2/512: mkdir d3/da/db8 0 2024-03-18T14:51:07.827 INFO:tasks.workunit.client.1.smithi100.stdout:2/513: fdatasync d3/d6/d29/d36/d49/f51 0 2024-03-18T14:51:07.827 INFO:tasks.workunit.client.1.smithi100.stdout:2/514: fdatasync d3/d6/f2f 0 2024-03-18T14:51:07.827 INFO:tasks.workunit.client.1.smithi100.stdout:5/509: symlink d29/d2d/d39/d65/d99/la2 0 2024-03-18T14:51:07.828 INFO:tasks.workunit.client.1.smithi100.stdout:5/510: dread - d29/d2d/d32/d38/f5f zero size 2024-03-18T14:51:07.828 INFO:tasks.workunit.client.1.smithi100.stdout:5/511: write d29/f34 [721000,39544] 0 2024-03-18T14:51:07.829 INFO:tasks.workunit.client.1.smithi100.stdout:4/527: rename d9/d13/d39/d59 to d9/d13/d5a/d5b/da0 0 2024-03-18T14:51:07.830 INFO:tasks.workunit.client.1.smithi100.stdout:2/515: creat d3/d6/d29/d36/d4a/fb9 x:0 0 0 2024-03-18T14:51:07.832 INFO:tasks.workunit.client.1.smithi100.stdout:5/512: unlink d29/c63 0 2024-03-18T14:51:07.832 INFO:tasks.workunit.client.1.smithi100.stdout:7/542: dread d5/fd [0,4194304] 0 2024-03-18T14:51:07.833 INFO:tasks.workunit.client.1.smithi100.stdout:8/579: dread d4/dc/dd/de/d21/d23/f27 [0,4194304] 0 2024-03-18T14:51:07.834 INFO:tasks.workunit.client.1.smithi100.stdout:4/528: truncate d9/f11 949610 0 2024-03-18T14:51:07.834 INFO:tasks.workunit.client.1.smithi100.stdout:4/529: write d9/d3a/f3c [1606749,50746] 0 2024-03-18T14:51:07.835 INFO:tasks.workunit.client.0.smithi012.stdout:9/943: dwrite d3/de6/dbf/f129 [0,4194304] 0 2024-03-18T14:51:07.835 INFO:tasks.workunit.client.1.smithi100.stdout:2/516: truncate d3/d6/f44 7171482 0 2024-03-18T14:51:07.836 INFO:tasks.workunit.client.1.smithi100.stdout:7/543: creat d5/d14/d16/d44/fc9 x:0 0 0 2024-03-18T14:51:07.838 INFO:tasks.workunit.client.1.smithi100.stdout:8/580: rename d4/dc/dd/de/d19/l4c to d4/dc/d1f/d33/lc9 0 2024-03-18T14:51:07.842 INFO:tasks.workunit.client.1.smithi100.stdout:2/517: unlink d3/d13/d40/d53/l84 0 2024-03-18T14:51:07.842 INFO:tasks.workunit.client.1.smithi100.stdout:2/518: chown d3/d6/d5f/f6d 0 1 2024-03-18T14:51:07.842 INFO:tasks.workunit.client.1.smithi100.stdout:3/581: dwrite d3/dd/d21/d3e/f32 [4194304,4194304] 0 2024-03-18T14:51:07.843 INFO:tasks.workunit.client.1.smithi100.stdout:3/582: write d3/d16/d13/f5f [174601,6112] 0 2024-03-18T14:51:07.843 INFO:tasks.workunit.client.1.smithi100.stdout:3/583: getdents d3/dd/d21/dae/dc9 0 2024-03-18T14:51:07.843 INFO:tasks.workunit.client.1.smithi100.stdout:7/544: rename d5/d22/d24/d39/fb0 to d5/d22/d24/dac/fca 0 2024-03-18T14:51:07.843 INFO:tasks.workunit.client.1.smithi100.stdout:7/545: read - d5/d22/d3b/d9f/da0/fa8 zero size 2024-03-18T14:51:07.844 INFO:tasks.workunit.client.1.smithi100.stdout:8/581: mknod d4/d5a/dbb/cca 0 2024-03-18T14:51:07.844 INFO:tasks.workunit.client.1.smithi100.stdout:8/582: creat d4/dc/d31/d85/fcb x:0 0 0 2024-03-18T14:51:07.844 INFO:tasks.workunit.client.1.smithi100.stdout:8/583: write d4/fa8 [96219,12476] 0 2024-03-18T14:51:07.844 INFO:tasks.workunit.client.1.smithi100.stdout:0/613: dwrite fa [0,4194304] 0 2024-03-18T14:51:07.844 INFO:tasks.workunit.client.1.smithi100.stdout:5/513: write d29/d2d/d39/d60/f6d [61833,44693] 0 2024-03-18T14:51:07.845 INFO:tasks.workunit.client.1.smithi100.stdout:2/519: creat d3/fba x:0 0 0 2024-03-18T14:51:07.845 INFO:tasks.workunit.client.1.smithi100.stdout:2/520: chown d3/d1a/l41 74067474 1 2024-03-18T14:51:07.846 INFO:tasks.workunit.client.1.smithi100.stdout:4/530: dread d9/f2b [0,4194304] 0 2024-03-18T14:51:07.846 INFO:tasks.workunit.client.0.smithi012.stdout:9/944: dread d3/de6/dbf/d66/f45 [0,4194304] 0 2024-03-18T14:51:07.847 INFO:tasks.workunit.client.0.smithi012.stdout:9/945: write d3/d9/f17 [217094,125042] 0 2024-03-18T14:51:07.850 INFO:tasks.workunit.client.1.smithi100.stdout:3/584: rename d3/d16/d13/d1f/d3f/d4f/d65/fc8 to d3/dd/d21/dae/fd1 0 2024-03-18T14:51:07.850 INFO:tasks.workunit.client.1.smithi100.stdout:7/546: mkdir d5/d14/d16/d2e/d62/d66/d4f/dcb 0 2024-03-18T14:51:07.850 INFO:tasks.workunit.client.1.smithi100.stdout:9/544: sync 2024-03-18T14:51:07.850 INFO:tasks.workunit.client.1.smithi100.stdout:9/545: truncate da/d46/f63 1628499 0 2024-03-18T14:51:07.851 INFO:tasks.workunit.client.0.smithi012.stdout:9/946: symlink d3/de6/dbf/d77/d52/d95/l133 0 2024-03-18T14:51:07.851 INFO:tasks.workunit.client.0.smithi012.stdout:3/828: dwrite dc/d30/d35/f2e [0,4194304] 0 2024-03-18T14:51:07.852 INFO:tasks.workunit.client.1.smithi100.stdout:8/584: creat d4/fcc x:0 0 0 2024-03-18T14:51:07.854 INFO:tasks.workunit.client.0.smithi012.stdout:9/947: unlink d3/de6/dbf/d66/d69/d7f/fe0 0 2024-03-18T14:51:07.857 INFO:tasks.workunit.client.1.smithi100.stdout:0/614: mkdir dd/d31/d91/d56/d62/d6e/ddd 0 2024-03-18T14:51:07.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/514: creat d29/d2d/d39/d71/d96/fa3 x:0 0 0 2024-03-18T14:51:07.863 INFO:tasks.workunit.client.1.smithi100.stdout:5/515: fdatasync d29/d4a/f80 0 2024-03-18T14:51:07.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/615: dread dd/d31/d91/d56/d71/fa3 [0,4194304] 0 2024-03-18T14:51:07.864 INFO:tasks.workunit.client.1.smithi100.stdout:0/616: stat dd/d20/d44/dce 0 2024-03-18T14:51:07.866 INFO:tasks.workunit.client.0.smithi012.stdout:9/948: creat d3/de6/dbf/d77/da8/f134 x:0 0 0 2024-03-18T14:51:07.867 INFO:tasks.workunit.client.1.smithi100.stdout:4/531: mknod d9/ca1 0 2024-03-18T14:51:07.868 INFO:tasks.workunit.client.1.smithi100.stdout:4/532: fdatasync d9/d13/d5a/d5b/f70 0 2024-03-18T14:51:07.868 INFO:tasks.workunit.client.1.smithi100.stdout:3/585: stat d3/d16/d13/d1f/d3f/d4f/d60/f63 0 2024-03-18T14:51:07.869 INFO:tasks.workunit.client.1.smithi100.stdout:3/586: chown d3/dd/d21/l53 30478117 1 2024-03-18T14:51:07.870 INFO:tasks.workunit.client.0.smithi012.stdout:9/949: rename d3/de6/dbf/d77/cb3 to d3/de6/dbf/d77/d52/d99/c135 0 2024-03-18T14:51:07.871 INFO:tasks.workunit.client.1.smithi100.stdout:7/547: mknod d5/d14/d16/d2e/d62/ccc 0 2024-03-18T14:51:07.873 INFO:tasks.workunit.client.1.smithi100.stdout:9/546: unlink f2 0 2024-03-18T14:51:07.873 INFO:tasks.workunit.client.1.smithi100.stdout:9/547: write da/d46/f8e [2587304,67398] 0 2024-03-18T14:51:07.873 INFO:tasks.workunit.client.1.smithi100.stdout:9/548: dread - da/dd/d29/d71/f98 zero size 2024-03-18T14:51:07.877 INFO:tasks.workunit.client.0.smithi012.stdout:9/950: dread d3/de6/dbf/d77/d52/d99/da0/ddf/f123 [0,4194304] 0 2024-03-18T14:51:07.880 INFO:tasks.workunit.client.1.smithi100.stdout:5/516: rename f10 to d29/d2d/d32/d38/fa4 0 2024-03-18T14:51:07.880 INFO:tasks.workunit.client.1.smithi100.stdout:5/517: readlink d29/d2d/d32/l90 0 2024-03-18T14:51:07.880 INFO:tasks.workunit.client.1.smithi100.stdout:5/518: write d29/f8c [78705,58472] 0 2024-03-18T14:51:07.880 INFO:tasks.workunit.client.1.smithi100.stdout:5/519: chown d29/d2d/d8f 1770 1 2024-03-18T14:51:07.881 INFO:tasks.workunit.client.0.smithi012.stdout:9/951: unlink d3/d9/d5c/l88 0 2024-03-18T14:51:07.883 INFO:tasks.workunit.client.1.smithi100.stdout:0/617: creat dd/d61/fde x:0 0 0 2024-03-18T14:51:07.883 INFO:tasks.workunit.client.1.smithi100.stdout:0/618: write dd/d31/d91/d65/fc8 [163697,88850] 0 2024-03-18T14:51:07.883 INFO:tasks.workunit.client.1.smithi100.stdout:0/619: chown dd/d2b/f4c 1324207489 1 2024-03-18T14:51:07.884 INFO:tasks.workunit.client.1.smithi100.stdout:0/620: write dd/f85 [858912,75317] 0 2024-03-18T14:51:07.884 INFO:tasks.workunit.client.1.smithi100.stdout:2/521: symlink d3/da/de/d20/lbb 0 2024-03-18T14:51:07.884 INFO:tasks.workunit.client.1.smithi100.stdout:2/522: truncate d3/d1a/d6c/fa4 116798 0 2024-03-18T14:51:07.885 INFO:tasks.workunit.client.0.smithi012.stdout:9/952: creat d3/de6/dbf/d66/d21/f136 x:0 0 0 2024-03-18T14:51:07.885 INFO:tasks.workunit.client.0.smithi012.stdout:9/953: truncate d3/de6/dbf/d77/fd8 449416 0 2024-03-18T14:51:07.886 INFO:tasks.workunit.client.1.smithi100.stdout:3/587: stat d3/d16/d13/d71/db9/lbd 0 2024-03-18T14:51:07.887 INFO:tasks.workunit.client.1.smithi100.stdout:7/548: creat d5/d22/d24/fcd x:0 0 0 2024-03-18T14:51:07.890 INFO:tasks.workunit.client.1.smithi100.stdout:9/549: symlink da/d46/lac 0 2024-03-18T14:51:07.891 INFO:tasks.workunit.client.0.smithi012.stdout:9/954: rename d3/de6/dbf/d66/d6e to d3/de6/dbf/d66/d69/d7f/d113/d137 0 2024-03-18T14:51:07.892 INFO:tasks.workunit.client.1.smithi100.stdout:5/520: creat d29/d2d/d39/d74/fa5 x:0 0 0 2024-03-18T14:51:07.893 INFO:tasks.workunit.client.1.smithi100.stdout:5/521: creat d29/fa6 x:0 0 0 2024-03-18T14:51:07.896 INFO:tasks.workunit.client.1.smithi100.stdout:0/621: link dd/f1d dd/d31/d91/d56/d71/dcc/dcf/fdf 0 2024-03-18T14:51:07.901 INFO:tasks.workunit.client.1.smithi100.stdout:3/588: creat d3/fd2 x:0 0 0 2024-03-18T14:51:07.902 INFO:tasks.workunit.client.1.smithi100.stdout:7/549: mkdir d5/d22/d3b/d9f/dce 0 2024-03-18T14:51:07.905 INFO:tasks.workunit.client.1.smithi100.stdout:9/550: rename da/db/f1a to da/d46/d4d/fad 0 2024-03-18T14:51:07.905 INFO:tasks.workunit.client.1.smithi100.stdout:9/551: read da/d46/d4d/f56 [412170,41520] 0 2024-03-18T14:51:07.907 INFO:tasks.workunit.client.1.smithi100.stdout:2/523: getdents d3/d13/d4f 0 2024-03-18T14:51:07.910 INFO:tasks.workunit.client.1.smithi100.stdout:7/550: creat d5/d22/d3b/d9f/fcf x:0 0 0 2024-03-18T14:51:07.911 INFO:tasks.workunit.client.1.smithi100.stdout:4/533: dread d9/d3a/f3c [0,4194304] 0 2024-03-18T14:51:07.911 INFO:tasks.workunit.client.1.smithi100.stdout:4/534: chown d9/d3a/f3c 172 1 2024-03-18T14:51:07.912 INFO:tasks.workunit.client.1.smithi100.stdout:4/535: write d9/f7f [55091,110657] 0 2024-03-18T14:51:07.912 INFO:tasks.workunit.client.1.smithi100.stdout:4/536: write d9/d13/f33 [2980925,113373] 0 2024-03-18T14:51:07.912 INFO:tasks.workunit.client.1.smithi100.stdout:4/537: fdatasync d9/d27/f36 0 2024-03-18T14:51:07.913 INFO:tasks.workunit.client.1.smithi100.stdout:9/552: mknod da/dd/cae 0 2024-03-18T14:51:07.913 INFO:tasks.workunit.client.1.smithi100.stdout:9/553: readlink da/dd/d3b/l5e 0 2024-03-18T14:51:07.913 INFO:tasks.workunit.client.1.smithi100.stdout:9/554: creat da/d46/d4d/d67/faf x:0 0 0 2024-03-18T14:51:07.913 INFO:tasks.workunit.client.1.smithi100.stdout:9/555: chown da/d88 341 1 2024-03-18T14:51:07.913 INFO:tasks.workunit.client.1.smithi100.stdout:4/538: write d9/d27/f36 [951009,62336] 0 2024-03-18T14:51:07.914 INFO:tasks.workunit.client.1.smithi100.stdout:2/524: rename d3/d6/cd to d3/d13/cbc 0 2024-03-18T14:51:07.914 INFO:tasks.workunit.client.1.smithi100.stdout:2/525: stat d3/d6/d29/d36/d4a 0 2024-03-18T14:51:07.916 INFO:tasks.workunit.client.1.smithi100.stdout:5/522: dwrite d29/d2d/d32/d38/f53 [0,4194304] 0 2024-03-18T14:51:07.916 INFO:tasks.workunit.client.1.smithi100.stdout:5/523: chown d29/d2d/d39/d65/c79 595 1 2024-03-18T14:51:07.916 INFO:tasks.workunit.client.1.smithi100.stdout:5/524: chown d29/d2d/f6f 6 1 2024-03-18T14:51:07.916 INFO:tasks.workunit.client.1.smithi100.stdout:3/589: dread d3/d16/d13/d1f/d3f/d4f/f51 [0,4194304] 0 2024-03-18T14:51:07.916 INFO:tasks.workunit.client.1.smithi100.stdout:3/590: read - d3/d16/d13/d1f/d37/fb3 zero size 2024-03-18T14:51:07.917 INFO:tasks.workunit.client.0.smithi012.stdout:9/955: dwrite d3/de6/dbf/d66/d69/d7f/d8c/dac/fb8 [0,4194304] 0 2024-03-18T14:51:07.919 INFO:tasks.workunit.client.0.smithi012.stdout:9/956: unlink d3/d9/fa2 0 2024-03-18T14:51:07.921 INFO:tasks.workunit.client.1.smithi100.stdout:9/556: mkdir da/db/db0 0 2024-03-18T14:51:07.921 INFO:tasks.workunit.client.1.smithi100.stdout:0/622: dwrite dd/d2b/f89 [0,4194304] 0 2024-03-18T14:51:07.922 INFO:tasks.workunit.client.1.smithi100.stdout:9/557: write da/dd/f60 [68726,54543] 0 2024-03-18T14:51:07.923 INFO:tasks.workunit.client.1.smithi100.stdout:4/539: mkdir d9/d27/da2 0 2024-03-18T14:51:07.927 INFO:tasks.workunit.client.1.smithi100.stdout:7/551: rename d5/f10 to d5/d90/fd0 0 2024-03-18T14:51:07.931 INFO:tasks.workunit.client.0.smithi012.stdout:9/957: dread d3/de6/dbf/d66/d69/d7f/d8c/fcd [0,4194304] 0 2024-03-18T14:51:07.931 INFO:tasks.workunit.client.0.smithi012.stdout:9/958: read d3/de6/dbf/d66/d21/f26 [2679293,61252] 0 2024-03-18T14:51:07.931 INFO:tasks.workunit.client.1.smithi100.stdout:2/526: symlink d3/d13/d40/lbd 0 2024-03-18T14:51:07.932 INFO:tasks.workunit.client.1.smithi100.stdout:0/623: dread dd/d31/d91/d56/d71/f90 [0,4194304] 0 2024-03-18T14:51:07.936 INFO:tasks.workunit.client.1.smithi100.stdout:5/525: dwrite d29/f8c [0,4194304] 0 2024-03-18T14:51:07.936 INFO:tasks.workunit.client.1.smithi100.stdout:5/526: truncate d29/d2d/d32/d38/f5f 498619 0 2024-03-18T14:51:07.937 INFO:tasks.workunit.client.1.smithi100.stdout:5/527: fsync d29/d2d/f41 0 2024-03-18T14:51:07.937 INFO:tasks.workunit.client.1.smithi100.stdout:7/552: read d5/d22/d24/d6e/f84 [2353146,48377] 0 2024-03-18T14:51:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/959: creat d3/de6/dbf/d66/f138 x:0 0 0 2024-03-18T14:51:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/960: fdatasync d3/de6/dbf/d77/d52/d99/da0/ddf/f123 0 2024-03-18T14:51:07.937 INFO:tasks.workunit.client.0.smithi012.stdout:9/961: chown d3/de6/dbf/d66/d21/d35/d48/l2d 15611 1 2024-03-18T14:51:07.939 INFO:tasks.workunit.client.1.smithi100.stdout:3/591: rmdir d3/d16 39 2024-03-18T14:51:07.940 INFO:tasks.workunit.client.0.smithi012.stdout:9/962: truncate d3/de6/dbf/d77/f56 2606318 0 2024-03-18T14:51:07.955 INFO:tasks.workunit.client.1.smithi100.stdout:9/558: mknod da/d88/cb1 0 2024-03-18T14:51:07.956 INFO:tasks.workunit.client.1.smithi100.stdout:4/540: mkdir d9/d13/d39/d76/da3 0 2024-03-18T14:51:07.958 INFO:tasks.workunit.client.1.smithi100.stdout:7/553: dwrite d5/d30/d4d/d5a/f6f [0,4194304] 0 2024-03-18T14:51:07.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/527: getdents d3/da 0 2024-03-18T14:51:07.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/528: write d3/da/de/d20/f9c [484691,51358] 0 2024-03-18T14:51:07.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/529: fsync d3/d1a/f2d 0 2024-03-18T14:51:07.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/530: fsync d3/d13/d40/d53/f6f 0 2024-03-18T14:51:07.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/531: creat d3/d1a/fbe x:0 0 0 2024-03-18T14:51:07.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/532: readlink d3/d13/d40/la3 0 2024-03-18T14:51:07.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/533: truncate d3/d13/d40/f83 322288 0 2024-03-18T14:51:07.961 INFO:tasks.workunit.client.1.smithi100.stdout:5/528: creat d29/d2d/d32/d38/fa7 x:0 0 0 2024-03-18T14:51:07.961 INFO:tasks.workunit.client.1.smithi100.stdout:5/529: write d29/d2d/d32/f5c [1047112,35321] 0 2024-03-18T14:51:07.962 INFO:tasks.workunit.client.0.smithi012.stdout:9/963: dwrite d3/de6/dbf/d77/d52/d99/da0/fd2 [0,4194304] 0 2024-03-18T14:51:07.963 INFO:tasks.workunit.client.0.smithi012.stdout:9/964: readlink d3/de6/dbf/d77/d52/lf7 0 2024-03-18T14:51:07.964 INFO:tasks.workunit.client.1.smithi100.stdout:3/592: link d3/d16/d13/d1f/l77 d3/dd/ld3 0 2024-03-18T14:51:07.968 INFO:tasks.workunit.client.1.smithi100.stdout:9/559: mkdir da/dd/d29/d71/d72/db2 0 2024-03-18T14:51:07.968 INFO:tasks.workunit.client.1.smithi100.stdout:1/521: sync 2024-03-18T14:51:07.968 INFO:tasks.workunit.client.1.smithi100.stdout:6/533: sync 2024-03-18T14:51:07.968 INFO:tasks.workunit.client.1.smithi100.stdout:9/560: truncate da/dd/f27 1748143 0 2024-03-18T14:51:07.968 INFO:tasks.workunit.client.1.smithi100.stdout:6/534: chown d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f88 69894450 1 2024-03-18T14:51:07.969 INFO:tasks.workunit.client.1.smithi100.stdout:2/534: dread d3/da/de/d20/f50 [0,4194304] 0 2024-03-18T14:51:07.969 INFO:tasks.workunit.client.1.smithi100.stdout:2/535: dread - d3/d6/d29/f92 zero size 2024-03-18T14:51:07.972 INFO:tasks.workunit.client.0.smithi012.stdout:9/965: link d3/de6/dbf/d66/d69/d7f/d113/d137/fbc d3/de6/dbf/d66/d21/d35/f139 0 2024-03-18T14:51:07.976 INFO:tasks.workunit.client.1.smithi100.stdout:1/522: symlink d8/d1e/d11/d6d/l9c 0 2024-03-18T14:51:07.976 INFO:tasks.workunit.client.1.smithi100.stdout:1/523: dread - d8/d30/d61/d8d/f95 zero size 2024-03-18T14:51:07.977 INFO:tasks.workunit.client.1.smithi100.stdout:5/530: dwrite d29/f9b [0,4194304] 0 2024-03-18T14:51:07.978 INFO:tasks.workunit.client.1.smithi100.stdout:5/531: write d29/d2d/d39/d60/f6d [2612049,124613] 0 2024-03-18T14:51:07.978 INFO:tasks.workunit.client.1.smithi100.stdout:5/532: truncate d29/d2d/d39/d83/f85 911345 0 2024-03-18T14:51:07.978 INFO:tasks.workunit.client.1.smithi100.stdout:5/533: readlink d29/d2d/d32/d38/l59 0 2024-03-18T14:51:07.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/536: creat d3/d13/d40/dac/fbf x:0 0 0 2024-03-18T14:51:07.980 INFO:tasks.workunit.client.0.smithi012.stdout:9/966: write d3/de6/dbf/d77/d52/d99/da0/fd2 [5135493,39194] 0 2024-03-18T14:51:07.980 INFO:tasks.workunit.client.0.smithi012.stdout:9/967: chown d3/de6/dbf/d66/d69/d7f/d8c/d8b 2 1 2024-03-18T14:51:07.982 INFO:tasks.workunit.client.1.smithi100.stdout:9/561: getdents da/d46/d4d 0 2024-03-18T14:51:07.984 INFO:tasks.workunit.client.0.smithi012.stdout:9/968: getdents d3/de6/dbf/d77/d52/d99/da0/ddf 0 2024-03-18T14:51:07.987 INFO:tasks.workunit.client.1.smithi100.stdout:1/524: rmdir d8/d2a/d84 0 2024-03-18T14:51:07.987 INFO:tasks.workunit.client.1.smithi100.stdout:1/525: write d8/d1e/d11/d27/d3e/f4b [76083,7609] 0 2024-03-18T14:51:07.987 INFO:tasks.workunit.client.1.smithi100.stdout:1/526: chown d8/d1e/f36 46027 1 2024-03-18T14:51:07.988 INFO:tasks.workunit.client.0.smithi012.stdout:9/969: symlink d3/d49/l13a 0 2024-03-18T14:51:07.988 INFO:tasks.workunit.client.0.smithi012.stdout:9/970: chown d3/de6/dbf/d66/d69/d7f/d8c/fcd 5975 1 2024-03-18T14:51:07.988 INFO:tasks.workunit.client.0.smithi012.stdout:9/971: chown d3/d49/fd4 85181 1 2024-03-18T14:51:07.991 INFO:tasks.workunit.client.1.smithi100.stdout:9/562: write da/d46/d4d/d67/f6e [3564990,44586] 0 2024-03-18T14:51:07.994 INFO:tasks.workunit.client.1.smithi100.stdout:5/534: link d29/d2d/d39/d61/f89 d29/d2d/d39/d65/fa8 0 2024-03-18T14:51:07.994 INFO:tasks.workunit.client.1.smithi100.stdout:5/535: stat d29/d2d/d39/d65/d7d 0 2024-03-18T14:51:07.994 INFO:tasks.workunit.client.1.smithi100.stdout:5/536: chown d29/d2d/d39/d71/c82 118787 1 2024-03-18T14:51:07.996 INFO:tasks.workunit.client.1.smithi100.stdout:6/535: rmdir d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27 39 2024-03-18T14:51:07.998 INFO:tasks.workunit.client.1.smithi100.stdout:2/537: mkdir d3/da/de/dc0 0 2024-03-18T14:51:07.998 INFO:tasks.workunit.client.1.smithi100.stdout:2/538: fsync d3/d6/d29/d36/d49/f5b 0 2024-03-18T14:51:07.998 INFO:tasks.workunit.client.1.smithi100.stdout:2/539: chown d3/da/de/d6e 928 1 2024-03-18T14:51:07.998 INFO:tasks.workunit.client.1.smithi100.stdout:2/540: truncate d3/d13/d40/d53/f8f 1272070 0 2024-03-18T14:51:07.999 INFO:tasks.workunit.client.1.smithi100.stdout:2/541: dread - d3/d1a/fbe zero size 2024-03-18T14:51:08.000 INFO:tasks.workunit.client.1.smithi100.stdout:4/541: rename d9/d81 to d9/d13/d5a/da4 0 2024-03-18T14:51:08.003 INFO:tasks.workunit.client.1.smithi100.stdout:1/527: creat d8/d1e/d11/d27/d73/d89/f9d x:0 0 0 2024-03-18T14:51:08.011 INFO:tasks.workunit.client.1.smithi100.stdout:9/563: unlink da/d46/d8b/c97 0 2024-03-18T14:51:08.012 INFO:tasks.workunit.client.1.smithi100.stdout:5/537: mkdir d29/d2d/d39/d71/da9 0 2024-03-18T14:51:08.012 INFO:tasks.workunit.client.1.smithi100.stdout:5/538: truncate d29/d2d/d32/d38/f76 905870 0 2024-03-18T14:51:08.014 INFO:tasks.workunit.client.1.smithi100.stdout:6/536: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa3 x:0 0 0 2024-03-18T14:51:08.014 INFO:tasks.workunit.client.1.smithi100.stdout:6/537: dread - d6/f91 zero size 2024-03-18T14:51:08.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/542: mknod d3/d1a/d6c/cc1 0 2024-03-18T14:51:08.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/543: chown d3/da 11265 1 2024-03-18T14:51:08.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/544: chown d3/l52 317 1 2024-03-18T14:51:08.017 INFO:tasks.workunit.client.1.smithi100.stdout:4/542: creat d9/d13/d25/fa5 x:0 0 0 2024-03-18T14:51:08.019 INFO:tasks.workunit.client.0.smithi012.stdout:9/972: dwrite d3/d9/d5c/f8f [0,4194304] 0 2024-03-18T14:51:08.019 INFO:tasks.workunit.client.0.smithi012.stdout:9/973: creat d3/de6/dbf/d66/d69/d7f/f13b x:0 0 0 2024-03-18T14:51:08.019 INFO:tasks.workunit.client.1.smithi100.stdout:1/528: link d8/d1e/d11/d27/d3e/f94 d8/d1e/d11/d6d/d59/d93/f9e 0 2024-03-18T14:51:08.028 INFO:tasks.workunit.client.1.smithi100.stdout:6/538: mknod d6/d7/df/d26/d4e/ca4 0 2024-03-18T14:51:08.029 INFO:tasks.workunit.client.1.smithi100.stdout:6/539: read d6/d12/f89 [387471,68463] 0 2024-03-18T14:51:08.030 INFO:tasks.workunit.client.1.smithi100.stdout:2/545: creat d3/d13/d4f/fc2 x:0 0 0 2024-03-18T14:51:08.030 INFO:tasks.workunit.client.1.smithi100.stdout:2/546: write d3/d6/d29/d36/d4a/fb9 [110293,14606] 0 2024-03-18T14:51:08.034 INFO:tasks.workunit.client.1.smithi100.stdout:9/564: dwrite da/dd/d3c/d6b/f84 [4194304,4194304] 0 2024-03-18T14:51:08.034 INFO:tasks.workunit.client.1.smithi100.stdout:9/565: truncate da/d46/fa3 1016627 0 2024-03-18T14:51:08.035 INFO:tasks.workunit.client.1.smithi100.stdout:2/547: unlink d3/d6/f2f 0 2024-03-18T14:51:08.036 INFO:tasks.workunit.client.1.smithi100.stdout:7/554: rename d5/d14/d16/d2c to d5/d14/d16/d2e/d62/dd1 0 2024-03-18T14:51:08.036 INFO:tasks.workunit.client.1.smithi100.stdout:7/555: chown d5/d14/d16/d2e/d62/ccc 15 1 2024-03-18T14:51:08.036 INFO:tasks.workunit.client.1.smithi100.stdout:7/556: dread - d5/d14/d16/d2e/d7b/fa6 zero size 2024-03-18T14:51:08.037 INFO:tasks.workunit.client.1.smithi100.stdout:1/529: dread d8/d1e/d11/d2d/f18 [0,4194304] 0 2024-03-18T14:51:08.037 INFO:tasks.workunit.client.1.smithi100.stdout:1/530: write d8/d1e/d11/d27/d73/d89/f9d [274087,34139] 0 2024-03-18T14:51:08.037 INFO:tasks.workunit.client.1.smithi100.stdout:1/531: dread - d8/d30/d61/f63 zero size 2024-03-18T14:51:08.039 INFO:tasks.workunit.client.1.smithi100.stdout:2/548: mkdir d3/dc3 0 2024-03-18T14:51:08.039 INFO:tasks.workunit.client.1.smithi100.stdout:2/549: dread - d3/d1a/fbe zero size 2024-03-18T14:51:08.041 INFO:tasks.workunit.client.1.smithi100.stdout:3/593: rename d3/d16/d13/d1f/d37/f54 to d3/d16/d13/d17/d7d/fd4 0 2024-03-18T14:51:08.042 INFO:tasks.workunit.client.0.smithi012.stdout:9/974: dread d3/de6/dbf/d77/d52/d65/fc4 [0,4194304] 0 2024-03-18T14:51:08.042 INFO:tasks.workunit.client.0.smithi012.stdout:9/975: write d3/f41 [370989,49287] 0 2024-03-18T14:51:08.042 INFO:tasks.workunit.client.1.smithi100.stdout:1/532: symlink d8/d2a/d9b/l9f 0 2024-03-18T14:51:08.044 INFO:tasks.workunit.client.1.smithi100.stdout:4/543: rename d9/d13/d25/d92 to d9/da6 0 2024-03-18T14:51:08.044 INFO:tasks.workunit.client.1.smithi100.stdout:4/544: fdatasync d9/d27/f36 0 2024-03-18T14:51:08.044 INFO:tasks.workunit.client.1.smithi100.stdout:4/545: chown d9/d27/f4b 144667201 1 2024-03-18T14:51:08.044 INFO:tasks.workunit.client.1.smithi100.stdout:9/566: read da/d2c/d2f/f31 [2234994,117214] 0 2024-03-18T14:51:08.045 INFO:tasks.workunit.client.0.smithi012.stdout:9/976: link d3/de6/dbf/d66/d69/d7f/d8c/d8b/lb7 d3/de6/dbf/d77/d52/l13c 0 2024-03-18T14:51:08.045 INFO:tasks.workunit.client.0.smithi012.stdout:9/977: dread - d3/de6/dbf/d66/d69/d7f/f12a zero size 2024-03-18T14:51:08.046 INFO:tasks.workunit.client.1.smithi100.stdout:3/594: creat d3/fd5 x:0 0 0 2024-03-18T14:51:08.046 INFO:tasks.workunit.client.1.smithi100.stdout:9/567: dread da/d46/d8b/f3f [0,4194304] 0 2024-03-18T14:51:08.048 INFO:tasks.workunit.client.1.smithi100.stdout:7/557: write d5/d14/d16/d2e/d62/f97 [1692314,60993] 0 2024-03-18T14:51:08.048 INFO:tasks.workunit.client.1.smithi100.stdout:6/540: dwrite d6/f5e [4194304,4194304] 0 2024-03-18T14:51:08.049 INFO:tasks.workunit.client.1.smithi100.stdout:1/533: link d8/d1e/d11/d27/f64 d8/d1e/d11/d6d/fa0 0 2024-03-18T14:51:08.054 INFO:tasks.workunit.client.1.smithi100.stdout:2/550: rename d3/da/de/d20/d82/d95/c73 to d3/da/de/d48/cc4 0 2024-03-18T14:51:08.054 INFO:tasks.workunit.client.1.smithi100.stdout:4/546: rmdir d9/d27 39 2024-03-18T14:51:08.055 INFO:tasks.workunit.client.0.smithi012.stdout:0/891: sync 2024-03-18T14:51:08.055 INFO:tasks.workunit.client.0.smithi012.stdout:7/878: sync 2024-03-18T14:51:08.055 INFO:tasks.workunit.client.0.smithi012.stdout:5/855: sync 2024-03-18T14:51:08.055 INFO:tasks.workunit.client.0.smithi012.stdout:5/856: write d3/dad/df2/d102/f115 [352579,2354] 0 2024-03-18T14:51:08.057 INFO:tasks.workunit.client.1.smithi100.stdout:3/595: unlink d3/c12 0 2024-03-18T14:51:08.058 INFO:tasks.workunit.client.1.smithi100.stdout:9/568: read da/d46/d4d/fad [2210049,50161] 0 2024-03-18T14:51:08.064 INFO:tasks.workunit.client.0.smithi012.stdout:7/879: getdents dd/d13/d23/d60 0 2024-03-18T14:51:08.067 INFO:tasks.workunit.client.0.smithi012.stdout:9/978: dwrite d3/de6/dbf/d66/d21/f136 [0,4194304] 0 2024-03-18T14:51:08.067 INFO:tasks.workunit.client.1.smithi100.stdout:2/551: mkdir d3/da/de/d20/d82/dad/dc5 0 2024-03-18T14:51:08.068 INFO:tasks.workunit.client.1.smithi100.stdout:3/596: creat d3/d16/d13/d1f/d3f/d4f/d60/da9/fd6 x:0 0 0 2024-03-18T14:51:08.071 INFO:tasks.workunit.client.1.smithi100.stdout:6/541: mkdir d6/d7/df/d72/d65/d7a/da5 0 2024-03-18T14:51:08.073 INFO:tasks.workunit.client.1.smithi100.stdout:2/552: unlink d3/d6/d29/c3f 0 2024-03-18T14:51:08.073 INFO:tasks.workunit.client.1.smithi100.stdout:2/553: creat d3/d6/d5f/fc6 x:0 0 0 2024-03-18T14:51:08.074 INFO:tasks.workunit.client.0.smithi012.stdout:0/892: dread d1/d20/d7b/fe3 [0,4194304] 0 2024-03-18T14:51:08.074 INFO:tasks.workunit.client.0.smithi012.stdout:9/979: creat d3/de6/dbf/d77/d52/d78/d12f/f13d x:0 0 0 2024-03-18T14:51:08.075 INFO:tasks.workunit.client.1.smithi100.stdout:3/597: creat d3/dd/d21/fd7 x:0 0 0 2024-03-18T14:51:08.075 INFO:tasks.workunit.client.1.smithi100.stdout:3/598: truncate d3/d16/d13/f9c 674367 0 2024-03-18T14:51:08.075 INFO:tasks.workunit.client.0.smithi012.stdout:0/893: truncate d1/d16/d31/f35 1437788 0 2024-03-18T14:51:08.076 INFO:tasks.workunit.client.1.smithi100.stdout:9/569: getdents da/d46/d8b 0 2024-03-18T14:51:08.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/554: rename d3/d13/d40/d53/f66 to d3/d1a/d76/fc7 0 2024-03-18T14:51:08.080 INFO:tasks.workunit.client.1.smithi100.stdout:3/599: link d3/d16/d13/d1f/d7c/c8c d3/d16/cd8 0 2024-03-18T14:51:08.082 INFO:tasks.workunit.client.1.smithi100.stdout:8/585: sync 2024-03-18T14:51:08.083 INFO:tasks.workunit.client.1.smithi100.stdout:0/624: sync 2024-03-18T14:51:08.085 INFO:tasks.workunit.client.1.smithi100.stdout:3/600: rename d3/dd/f24 to d3/dd/d21/dcd/fd9 0 2024-03-18T14:51:08.086 INFO:tasks.workunit.client.1.smithi100.stdout:6/542: write d6/d7/df/d26/f45 [2782467,35736] 0 2024-03-18T14:51:08.086 INFO:tasks.workunit.client.1.smithi100.stdout:6/543: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/f96 [526951,109759] 0 2024-03-18T14:51:08.087 INFO:tasks.workunit.client.1.smithi100.stdout:8/586: symlink d4/d3c/lcd 0 2024-03-18T14:51:08.087 INFO:tasks.workunit.client.1.smithi100.stdout:8/587: chown d4/dc/dd/de/f93 0 1 2024-03-18T14:51:08.088 INFO:tasks.workunit.client.1.smithi100.stdout:3/601: creat d3/d16/d13/d1f/fda x:0 0 0 2024-03-18T14:51:08.088 INFO:tasks.workunit.client.1.smithi100.stdout:3/602: write d3/d16/d13/d17/f33 [2421165,94232] 0 2024-03-18T14:51:08.088 INFO:tasks.workunit.client.1.smithi100.stdout:3/603: truncate d3/d16/d13/d17/f7a 1883119 0 2024-03-18T14:51:08.090 INFO:tasks.workunit.client.1.smithi100.stdout:3/604: link d3/fc2 d3/d16/d13/d17/dab/fdb 0 2024-03-18T14:51:08.091 INFO:tasks.workunit.client.1.smithi100.stdout:8/588: getdents d4/dc/dd/de/d21/d23/d25 0 2024-03-18T14:51:08.091 INFO:tasks.workunit.client.1.smithi100.stdout:8/589: write d4/fcc [939097,68816] 0 2024-03-18T14:51:08.093 INFO:tasks.workunit.client.1.smithi100.stdout:1/534: dwrite d8/d1e/d11/d27/d73/d89/f9d [0,4194304] 0 2024-03-18T14:51:08.094 INFO:tasks.workunit.client.1.smithi100.stdout:3/605: mknod d3/d16/d13/d71/db9/cdc 0 2024-03-18T14:51:08.094 INFO:tasks.workunit.client.1.smithi100.stdout:3/606: chown d3/d16/d13/d17/d7d/fa3 49900393 1 2024-03-18T14:51:08.094 INFO:tasks.workunit.client.1.smithi100.stdout:3/607: chown d3/dd/d49/c6e 126402 1 2024-03-18T14:51:08.100 INFO:tasks.workunit.client.1.smithi100.stdout:1/535: creat d8/d1e/d11/d27/d83/fa1 x:0 0 0 2024-03-18T14:51:08.100 INFO:tasks.workunit.client.1.smithi100.stdout:7/558: dwrite d5/f72 [0,4194304] 0 2024-03-18T14:51:08.101 INFO:tasks.workunit.client.1.smithi100.stdout:3/608: mknod d3/d16/d13/d1f/d7c/cdd 0 2024-03-18T14:51:08.103 INFO:tasks.workunit.client.0.smithi012.stdout:9/980: rename d3/de6/dbf/d66/d69/d7f to d3/d9/d5c/dbd/d13e 0 2024-03-18T14:51:08.103 INFO:tasks.workunit.client.0.smithi012.stdout:9/981: chown d3/de6/dbf/d66/d21/d35/l3d 17 1 2024-03-18T14:51:08.103 INFO:tasks.workunit.client.0.smithi012.stdout:9/982: creat d3/de6/dbf/d77/d52/d99/da0/ddf/dee/f13f x:0 0 0 2024-03-18T14:51:08.103 INFO:tasks.workunit.client.0.smithi012.stdout:9/983: write d3/de6/dbf/d66/d21/f26 [4833473,110691] 0 2024-03-18T14:51:08.109 INFO:tasks.workunit.client.1.smithi100.stdout:4/547: dwrite d9/d17/f24 [4194304,4194304] 0 2024-03-18T14:51:08.110 INFO:tasks.workunit.client.0.smithi012.stdout:5/857: dwrite d3/f1f [8388608,4194304] 0 2024-03-18T14:51:08.110 INFO:tasks.workunit.client.1.smithi100.stdout:7/559: symlink d5/d22/d3b/d9f/da0/ld2 0 2024-03-18T14:51:08.110 INFO:tasks.workunit.client.1.smithi100.stdout:7/560: getdents d5/d22/d3b 0 2024-03-18T14:51:08.111 INFO:tasks.workunit.client.1.smithi100.stdout:7/561: write d5/d22/d24/f91 [832423,121789] 0 2024-03-18T14:51:08.111 INFO:tasks.workunit.client.1.smithi100.stdout:7/562: chown d5/c29 16 1 2024-03-18T14:51:08.112 INFO:tasks.workunit.client.0.smithi012.stdout:7/880: dwrite dd/d13/d23/d2f/d53/fa5 [0,4194304] 0 2024-03-18T14:51:08.114 INFO:tasks.workunit.client.0.smithi012.stdout:9/984: mknod d3/d9/d5c/dbd/d13e/d113/d137/dc6/d111/c140 0 2024-03-18T14:51:08.119 INFO:tasks.workunit.client.1.smithi100.stdout:9/570: dwrite da/d2c/d2f/f4a [0,4194304] 0 2024-03-18T14:51:08.119 INFO:tasks.workunit.client.1.smithi100.stdout:2/555: dwrite d3/da/de/f3c [0,4194304] 0 2024-03-18T14:51:08.119 INFO:tasks.workunit.client.1.smithi100.stdout:2/556: read d3/d1a/d6c/f8b [1639670,3623] 0 2024-03-18T14:51:08.119 INFO:tasks.workunit.client.1.smithi100.stdout:0/625: dwrite dd/d31/d91/d7b/fad [0,4194304] 0 2024-03-18T14:51:08.120 INFO:tasks.workunit.client.1.smithi100.stdout:4/548: unlink d9/d13/f3f 0 2024-03-18T14:51:08.120 INFO:tasks.workunit.client.1.smithi100.stdout:3/609: dread d3/d16/d13/d1f/d3f/d4f/d60/f94 [0,4194304] 0 2024-03-18T14:51:08.121 INFO:tasks.workunit.client.1.smithi100.stdout:3/610: read - d3/d16/d13/d17/d7d/fad zero size 2024-03-18T14:51:08.121 INFO:tasks.workunit.client.1.smithi100.stdout:1/536: dwrite d8/d1e/d11/f77 [4194304,4194304] 0 2024-03-18T14:51:08.125 INFO:tasks.workunit.client.1.smithi100.stdout:7/563: mkdir d5/d22/d3b/d9f/da0/db1/dd3 0 2024-03-18T14:51:08.136 INFO:tasks.workunit.client.1.smithi100.stdout:0/626: write dd/d20/f32 [2083623,28950] 0 2024-03-18T14:51:08.137 INFO:tasks.workunit.client.1.smithi100.stdout:4/549: write d9/d17/f45 [860109,41979] 0 2024-03-18T14:51:08.137 INFO:tasks.workunit.client.1.smithi100.stdout:9/571: truncate da/d46/f63 499665 0 2024-03-18T14:51:08.139 INFO:tasks.workunit.client.1.smithi100.stdout:3/611: dread d3/f10 [0,4194304] 0 2024-03-18T14:51:08.139 INFO:tasks.workunit.client.1.smithi100.stdout:3/612: dread - d3/dd/d49/f87 zero size 2024-03-18T14:51:08.150 INFO:tasks.workunit.client.0.smithi012.stdout:5/858: dwrite d3/d15/d42/dc6/d73/fee [0,4194304] 0 2024-03-18T14:51:08.152 INFO:tasks.workunit.client.1.smithi100.stdout:1/537: rmdir d8/d1e/d11/d27/d3e 39 2024-03-18T14:51:08.160 INFO:tasks.workunit.client.0.smithi012.stdout:9/985: dwrite d3/f3b [0,4194304] 0 2024-03-18T14:51:08.163 INFO:tasks.workunit.client.0.smithi012.stdout:0/894: rename d1/d20/d4e/d54 to d1/d16/d19/dd0/d129 0 2024-03-18T14:51:08.164 INFO:tasks.workunit.client.0.smithi012.stdout:9/986: rename d3/de6 to d3/de6/dc1/d141 22 2024-03-18T14:51:08.164 INFO:tasks.workunit.client.0.smithi012.stdout:9/987: chown d3/le 213 1 2024-03-18T14:51:08.164 INFO:tasks.workunit.client.0.smithi012.stdout:9/988: readlink d3/de6/dbf/d66/l37 0 2024-03-18T14:51:08.164 INFO:tasks.workunit.client.1.smithi100.stdout:2/557: dwrite d3/d6/d5f/f6d [0,4194304] 0 2024-03-18T14:51:08.170 INFO:tasks.workunit.client.0.smithi012.stdout:7/881: getdents dd/d4f 0 2024-03-18T14:51:08.170 INFO:tasks.workunit.client.0.smithi012.stdout:7/882: readlink dd/d13/d23/d2f/d7a/dab/dbc/lc1 0 2024-03-18T14:51:08.174 INFO:tasks.workunit.client.0.smithi012.stdout:9/989: dread d3/d49/f57 [0,4194304] 0 2024-03-18T14:51:08.177 INFO:tasks.workunit.client.0.smithi012.stdout:0/895: creat d1/d16/d19/f12a x:0 0 0 2024-03-18T14:51:08.179 INFO:tasks.workunit.client.0.smithi012.stdout:7/883: creat dd/d13/d23/d2f/de1/f12f x:0 0 0 2024-03-18T14:51:08.181 INFO:tasks.workunit.client.0.smithi012.stdout:9/990: creat d3/f142 x:0 0 0 2024-03-18T14:51:08.181 INFO:tasks.workunit.client.0.smithi012.stdout:9/991: write d3/d9/fd [15092,68140] 0 2024-03-18T14:51:08.184 INFO:tasks.workunit.client.0.smithi012.stdout:0/896: unlink d1/d16/d19/dd0/d129/d8a/f111 0 2024-03-18T14:51:08.184 INFO:tasks.workunit.client.0.smithi012.stdout:0/897: write d1/d16/d19/dd0/d129/d67/f9a [3071598,39532] 0 2024-03-18T14:51:08.184 INFO:tasks.workunit.client.0.smithi012.stdout:0/898: stat d1/d2d/d113 0 2024-03-18T14:51:08.186 INFO:tasks.workunit.client.0.smithi012.stdout:7/884: link dd/d13/d23/ldb dd/d13/d23/d76/d6f/db3/l130 0 2024-03-18T14:51:08.188 INFO:tasks.workunit.client.0.smithi012.stdout:0/899: dread d1/d16/d79/ff5 [0,4194304] 0 2024-03-18T14:51:08.188 INFO:tasks.workunit.client.0.smithi012.stdout:0/900: creat d1/d20/d23/d6f/f12b x:0 0 0 2024-03-18T14:51:08.188 INFO:tasks.workunit.client.0.smithi012.stdout:9/992: mknod d3/d9/d5c/dbd/d13e/d8c/dac/dd1/c143 0 2024-03-18T14:51:08.189 INFO:tasks.workunit.client.0.smithi012.stdout:9/993: readlink d3/d9/d5c/dbd/d13e/d8c/l75 0 2024-03-18T14:51:08.189 INFO:tasks.workunit.client.0.smithi012.stdout:9/994: write d3/de6/dbf/d66/d21/f46 [4254429,101542] 0 2024-03-18T14:51:08.193 INFO:tasks.workunit.client.1.smithi100.stdout:1/538: unlink d8/d1e/d11/c23 0 2024-03-18T14:51:08.194 INFO:tasks.workunit.client.0.smithi012.stdout:7/885: mkdir dd/d13/d23/d60/d131 0 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.1.smithi100.stdout:0/627: dwrite dd/d2b/d95/d79/fd6 [0,4194304] 0 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.1.smithi100.stdout:0/628: read - dd/d20/db5/fab zero size 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.1.smithi100.stdout:7/564: dwrite d5/d22/d24/f91 [0,4194304] 0 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.1.smithi100.stdout:7/565: chown d5/d14/l6b 13059702 1 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.1.smithi100.stdout:8/590: dwrite d4/f43 [0,4194304] 0 2024-03-18T14:51:08.198 INFO:tasks.workunit.client.0.smithi012.stdout:9/995: mknod d3/d9/d5c/dbd/c144 0 2024-03-18T14:51:08.200 INFO:tasks.workunit.client.1.smithi100.stdout:3/613: dwrite d3/d16/d13/d17/dab/fbc [0,4194304] 0 2024-03-18T14:51:08.200 INFO:tasks.workunit.client.1.smithi100.stdout:3/614: chown d3/d16/d13/d17/f33 2233966 1 2024-03-18T14:51:08.200 INFO:tasks.workunit.client.1.smithi100.stdout:3/615: write d3/f10 [9055865,16268] 0 2024-03-18T14:51:08.201 INFO:tasks.workunit.client.0.smithi012.stdout:7/886: mknod dd/d13/d23/d76/d109/dfa/c132 0 2024-03-18T14:51:08.201 INFO:tasks.workunit.client.0.smithi012.stdout:5/859: dwrite d3/d1e/d33/faa [0,4194304] 0 2024-03-18T14:51:08.201 INFO:tasks.workunit.client.0.smithi012.stdout:5/860: dread - d3/d29/d35/fea zero size 2024-03-18T14:51:08.201 INFO:tasks.workunit.client.0.smithi012.stdout:5/861: chown d3/d15/d42/dc6/d73/f92 3 1 2024-03-18T14:51:08.202 INFO:tasks.workunit.client.1.smithi100.stdout:1/539: getdents d8/d2a/d37/d50/d6f/d71 0 2024-03-18T14:51:08.205 INFO:tasks.workunit.client.0.smithi012.stdout:9/996: rename d3/d9/d5c/dbd/d13e/d8c/d8b to d3/d9/d5c/dbd/d145 0 2024-03-18T14:51:08.205 INFO:tasks.workunit.client.0.smithi012.stdout:9/997: fdatasync d3/f4 0 2024-03-18T14:51:08.207 INFO:tasks.workunit.client.1.smithi100.stdout:2/558: getdents d3/d6/d29 0 2024-03-18T14:51:08.216 INFO:tasks.workunit.client.1.smithi100.stdout:4/550: truncate d9/d17/f24 5709929 0 2024-03-18T14:51:08.216 INFO:tasks.workunit.client.1.smithi100.stdout:4/551: readlink d9/d27/d62/l68 0 2024-03-18T14:51:08.220 INFO:tasks.workunit.client.0.smithi012.stdout:5/862: write d3/d15/d5f/f70 [850057,117955] 0 2024-03-18T14:51:08.221 INFO:tasks.workunit.client.1.smithi100.stdout:7/566: symlink d5/d14/d16/d2e/d62/d66/ld4 0 2024-03-18T14:51:08.223 INFO:tasks.workunit.client.1.smithi100.stdout:8/591: link d4/l6e d4/d3c/lce 0 2024-03-18T14:51:08.223 INFO:tasks.workunit.client.1.smithi100.stdout:8/592: creat d4/dc/d1f/d59/d6c/fcf x:0 0 0 2024-03-18T14:51:08.223 INFO:tasks.workunit.client.1.smithi100.stdout:8/593: chown d4/dc/dd/de/d19/c4d 150516 1 2024-03-18T14:51:08.224 INFO:tasks.workunit.client.0.smithi012.stdout:7/887: write dd/d13/d23/d105/f127 [1317571,3863] 0 2024-03-18T14:51:08.224 INFO:tasks.workunit.client.0.smithi012.stdout:5/863: mknod d3/d15/d42/dc6/d73/dd3/c11f 0 2024-03-18T14:51:08.229 INFO:tasks.workunit.client.1.smithi100.stdout:4/552: read d9/fb [2664478,101826] 0 2024-03-18T14:51:08.229 INFO:tasks.workunit.client.0.smithi012.stdout:7/888: creat dd/d13/d23/d76/d6f/f133 x:0 0 0 2024-03-18T14:51:08.231 INFO:tasks.workunit.client.1.smithi100.stdout:1/540: mkdir d8/d30/d61/da2 0 2024-03-18T14:51:08.231 INFO:tasks.workunit.client.1.smithi100.stdout:1/541: chown d8/d2a/d37/f66 0 1 2024-03-18T14:51:08.231 INFO:tasks.workunit.client.0.smithi012.stdout:0/901: dwrite d1/d16/d19/d8d/fab [4194304,4194304] 0 2024-03-18T14:51:08.232 INFO:tasks.workunit.client.0.smithi012.stdout:5/864: symlink d3/d84/l120 0 2024-03-18T14:51:08.236 INFO:tasks.workunit.client.0.smithi012.stdout:7/889: truncate fa 2609100 0 2024-03-18T14:51:08.238 INFO:tasks.workunit.client.1.smithi100.stdout:8/594: creat d4/dc/dd/de/d21/fd0 x:0 0 0 2024-03-18T14:51:08.238 INFO:tasks.workunit.client.1.smithi100.stdout:8/595: dread - d4/f60 zero size 2024-03-18T14:51:08.254 INFO:tasks.workunit.client.1.smithi100.stdout:1/542: write d8/d2a/d37/f39 [2397618,86767] 0 2024-03-18T14:51:08.258 INFO:tasks.workunit.client.1.smithi100.stdout:0/629: dwrite dd/f49 [0,4194304] 0 2024-03-18T14:51:08.260 INFO:tasks.workunit.client.1.smithi100.stdout:2/559: dwrite d3/d13/d4f/fc2 [0,4194304] 0 2024-03-18T14:51:08.260 INFO:tasks.workunit.client.1.smithi100.stdout:2/560: creat d3/d6/d5f/d6a/fc8 x:0 0 0 2024-03-18T14:51:08.261 INFO:tasks.workunit.client.0.smithi012.stdout:9/998: dwrite d3/de6/dbf/d66/d69/f124 [0,4194304] 0 2024-03-18T14:51:08.317 INFO:tasks.workunit.client.0.smithi012.stdout:7/890: truncate dd/d13/d23/d76/d6f/f91 2361831 0 2024-03-18T14:51:08.319 INFO:tasks.workunit.client.1.smithi100.stdout:3/616: dwrite d3/da5/fc7 [0,4194304] 0 2024-03-18T14:51:08.320 INFO:tasks.workunit.client.0.smithi012.stdout:2/907: sync 2024-03-18T14:51:08.320 INFO:tasks.workunit.client.0.smithi012.stdout:6/805: sync 2024-03-18T14:51:08.320 INFO:tasks.workunit.client.0.smithi012.stdout:6/806: rename d17 to d17/d38/d51/d66/ded 22 2024-03-18T14:51:08.320 INFO:tasks.workunit.client.0.smithi012.stdout:2/908: dread - d0/d6b/dbf/d4f/fe6 zero size 2024-03-18T14:51:08.326 INFO:tasks.workunit.client.0.smithi012.stdout:3/829: sync 2024-03-18T14:51:08.327 INFO:tasks.workunit.client.0.smithi012.stdout:4/800: sync 2024-03-18T14:51:08.327 INFO:tasks.workunit.client.0.smithi012.stdout:1/813: sync 2024-03-18T14:51:08.330 INFO:tasks.workunit.client.0.smithi012.stdout:6/807: write d17/d19/f3e [810498,128520] 0 2024-03-18T14:51:08.388 INFO:tasks.workunit.client.1.smithi100.stdout:1/543: stat d8/d1e/d11/d27/d3f/f49 0 2024-03-18T14:51:08.388 INFO:tasks.workunit.client.1.smithi100.stdout:1/544: chown d8/d1e/d11 95235803 1 2024-03-18T14:51:08.389 INFO:tasks.workunit.client.0.smithi012.stdout:1/814: dread d8/db/f2d [0,4194304] 0 2024-03-18T14:51:08.390 INFO:tasks.workunit.client.1.smithi100.stdout:0/630: mkdir dd/d31/d91/d56/de0 0 2024-03-18T14:51:08.393 INFO:tasks.workunit.client.1.smithi100.stdout:4/553: dwrite d9/d13/d25/f4d [0,4194304] 0 2024-03-18T14:51:08.394 INFO:tasks.workunit.client.1.smithi100.stdout:4/554: truncate d9/d13/d25/f48 1514513 0 2024-03-18T14:51:08.394 INFO:tasks.workunit.client.1.smithi100.stdout:7/567: dwrite d5/d14/d16/d2e/d62/f97 [0,4194304] 0 2024-03-18T14:51:08.394 INFO:tasks.workunit.client.1.smithi100.stdout:7/568: truncate d5/d14/f40 5196176 0 2024-03-18T14:51:08.394 INFO:tasks.workunit.client.0.smithi012.stdout:9/999: dwrite d3/de6/dbf/fe3 [0,4194304] 0 2024-03-18T14:51:08.394 INFO:tasks.workunit.client.1.smithi100.stdout:2/561: dwrite d3/d1a/d6c/fa4 [0,4194304] 0 2024-03-18T14:51:08.395 INFO:tasks.workunit.client.0.smithi012.stdout:7/891: creat dd/d13/d103/d12b/f134 x:0 0 0 2024-03-18T14:51:08.395 INFO:tasks.workunit.client.0.smithi012.stdout:7/892: fsync dd/d13/d23/d60/fed 0 2024-03-18T14:51:08.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/909: symlink d0/d6b/dbf/d33/d77/l125 0 2024-03-18T14:51:08.401 INFO:tasks.workunit.client.0.smithi012.stdout:2/910: write d0/d6b/dbf/ff3 [138763,100298] 0 2024-03-18T14:51:08.405 INFO:tasks.workunit.client.0.smithi012.stdout:3/830: unlink dc/de6/f10b 0 2024-03-18T14:51:08.405 INFO:tasks.workunit.client.0.smithi012.stdout:3/831: dread - dc/d3c/d4c/d57/d103/d9a/dd7/f107 zero size 2024-03-18T14:51:08.408 INFO:tasks.workunit.client.0.smithi012.stdout:4/801: mkdir d6/da/d58/d113 0 2024-03-18T14:51:08.414 INFO:tasks.workunit.client.1.smithi100.stdout:3/617: rename d3/d16/d13/d1f/d3f/d4f/d65/lb2 to d3/dd/dd0/lde 0 2024-03-18T14:51:08.414 INFO:tasks.workunit.client.1.smithi100.stdout:3/618: write d3/dd/d49/f4a [2593770,95950] 0 2024-03-18T14:51:08.416 INFO:tasks.workunit.client.0.smithi012.stdout:6/808: mkdir d17/d38/d51/d66/dee 0 2024-03-18T14:51:08.416 INFO:tasks.workunit.client.0.smithi012.stdout:3/832: dread dc/d3c/d59/f5a [0,4194304] 0 2024-03-18T14:51:08.418 INFO:tasks.workunit.client.1.smithi100.stdout:1/545: symlink d8/d1e/d11/d27/la3 0 2024-03-18T14:51:08.418 INFO:tasks.workunit.client.1.smithi100.stdout:1/546: creat d8/d1e/d11/d27/d73/d89/fa4 x:0 0 0 2024-03-18T14:51:08.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/815: creat d8/db/d10/d16/da1/db9/f105 x:0 0 0 2024-03-18T14:51:08.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/816: readlink d8/db/d10/d31/d43/d5e/d65/d70/lc9 0 2024-03-18T14:51:08.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/817: write d8/db/d10/d16/da1/d2e/d9c/fea [342187,124243] 0 2024-03-18T14:51:08.420 INFO:tasks.workunit.client.1.smithi100.stdout:0/631: creat dd/d31/d91/fe1 x:0 0 0 2024-03-18T14:51:08.426 INFO:tasks.workunit.client.0.smithi012.stdout:7/893: creat dd/d13/d23/d60/d9c/f135 x:0 0 0 2024-03-18T14:51:08.429 INFO:tasks.workunit.client.1.smithi100.stdout:4/555: link d9/d13/d39/l58 d9/d13/d39/d4c/la7 0 2024-03-18T14:51:08.432 INFO:tasks.workunit.client.0.smithi012.stdout:2/911: rename d0/d10/l4d to d0/d6c/l126 0 2024-03-18T14:51:08.436 INFO:tasks.workunit.client.0.smithi012.stdout:4/802: creat d6/da/d89/d97/dc5/dd3/f114 x:0 0 0 2024-03-18T14:51:08.439 INFO:tasks.workunit.client.0.smithi012.stdout:0/902: write d1/d16/d19/f47 [2394743,75443] 0 2024-03-18T14:51:08.439 INFO:tasks.workunit.client.0.smithi012.stdout:7/894: getdents dd/d13/d23/d60/d9c/dfd 0 2024-03-18T14:51:08.440 INFO:tasks.workunit.client.0.smithi012.stdout:2/912: rmdir d0/d6c 39 2024-03-18T14:51:08.443 INFO:tasks.workunit.client.0.smithi012.stdout:0/903: dread d1/d16/f17 [0,4194304] 0 2024-03-18T14:51:08.444 INFO:tasks.workunit.client.1.smithi100.stdout:7/569: creat d5/d22/d3b/d47/fd5 x:0 0 0 2024-03-18T14:51:08.444 INFO:tasks.workunit.client.1.smithi100.stdout:7/570: chown d5/d22/d3b/d9f/dce 14820 1 2024-03-18T14:51:08.444 INFO:tasks.workunit.client.1.smithi100.stdout:7/571: dread - d5/d30/d4d/d5a/f82 zero size 2024-03-18T14:51:08.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/809: dwrite d17/f1b [0,4194304] 0 2024-03-18T14:51:08.446 INFO:tasks.workunit.client.0.smithi012.stdout:3/833: dwrite dc/d3c/d59/fbc [0,4194304] 0 2024-03-18T14:51:08.446 INFO:tasks.workunit.client.0.smithi012.stdout:4/803: rename d6/da/d89/cea to d6/da/d58/dd2/c115 0 2024-03-18T14:51:08.449 INFO:tasks.workunit.client.0.smithi012.stdout:7/895: getdents dd/d13/d23/d76/d109/d129/dcc 0 2024-03-18T14:51:08.449 INFO:tasks.workunit.client.0.smithi012.stdout:7/896: write dd/d13/d23/d76/d109/d129/fff [3088231,123071] 0 2024-03-18T14:51:08.451 INFO:tasks.workunit.client.0.smithi012.stdout:0/904: stat d1/d16/d19/dd0/d129/c11f 0 2024-03-18T14:51:08.453 INFO:tasks.workunit.client.1.smithi100.stdout:2/562: getdents d3/da/de/d20/d82/dad 0 2024-03-18T14:51:08.453 INFO:tasks.workunit.client.1.smithi100.stdout:2/563: chown d3/da/de/d20/f90 7730808 1 2024-03-18T14:51:08.457 INFO:tasks.workunit.client.0.smithi012.stdout:6/810: mknod d17/d18/dc1/cef 0 2024-03-18T14:51:08.458 INFO:tasks.workunit.client.1.smithi100.stdout:7/572: read d5/d22/d24/f2d [596629,50835] 0 2024-03-18T14:51:08.458 INFO:tasks.workunit.client.1.smithi100.stdout:7/573: truncate d5/f35 5202676 0 2024-03-18T14:51:08.462 INFO:tasks.workunit.client.1.smithi100.stdout:3/619: mknod d3/d16/d13/d17/dab/cdf 0 2024-03-18T14:51:08.462 INFO:tasks.workunit.client.0.smithi012.stdout:4/804: stat d6/d19/d6c/d96/l10f 0 2024-03-18T14:51:08.463 INFO:tasks.workunit.client.0.smithi012.stdout:4/805: fsync d6/d1a/d30/fd5 0 2024-03-18T14:51:08.463 INFO:tasks.workunit.client.0.smithi012.stdout:4/806: chown d6/d1a/d30/d46/d6f/d8b/c9b 0 1 2024-03-18T14:51:08.463 INFO:tasks.workunit.client.0.smithi012.stdout:4/807: truncate d6/d1a/d30/d46/d6f/d8b/f112 383000 0 2024-03-18T14:51:08.464 INFO:tasks.workunit.client.1.smithi100.stdout:5/539: sync 2024-03-18T14:51:08.464 INFO:tasks.workunit.client.1.smithi100.stdout:5/540: chown d29/f91 102 1 2024-03-18T14:51:08.464 INFO:tasks.workunit.client.1.smithi100.stdout:1/547: truncate d8/d30/f35 7235840 0 2024-03-18T14:51:08.465 INFO:tasks.workunit.client.1.smithi100.stdout:2/564: dread d3/da/de/f3c [0,4194304] 0 2024-03-18T14:51:08.465 INFO:tasks.workunit.client.1.smithi100.stdout:2/565: creat d3/d6/d5f/d6a/fc9 x:0 0 0 2024-03-18T14:51:08.466 INFO:tasks.workunit.client.0.smithi012.stdout:7/897: truncate dd/d13/d23/d76/d6b/f10e 300509 0 2024-03-18T14:51:08.467 INFO:tasks.workunit.client.1.smithi100.stdout:0/632: truncate dd/d20/f28 3511414 0 2024-03-18T14:51:08.468 INFO:tasks.workunit.client.1.smithi100.stdout:0/633: truncate dd/d31/f52 339991 0 2024-03-18T14:51:08.469 INFO:tasks.workunit.client.0.smithi012.stdout:2/913: dwrite d0/d3c/d40/d47/d85/fba [0,4194304] 0 2024-03-18T14:51:08.469 INFO:tasks.workunit.client.0.smithi012.stdout:6/811: creat d17/d38/d5b/ff0 x:0 0 0 2024-03-18T14:51:08.470 INFO:tasks.workunit.client.0.smithi012.stdout:3/834: dwrite f6 [4194304,4194304] 0 2024-03-18T14:51:08.470 INFO:tasks.workunit.client.0.smithi012.stdout:3/835: dread - dc/d3c/d4c/d57/d103/fcd zero size 2024-03-18T14:51:08.471 INFO:tasks.workunit.client.1.smithi100.stdout:4/556: truncate d9/d3a/f69 3950883 0 2024-03-18T14:51:08.474 INFO:tasks.workunit.client.0.smithi012.stdout:4/808: creat d6/d19/d6c/d96/f116 x:0 0 0 2024-03-18T14:51:08.475 INFO:tasks.workunit.client.1.smithi100.stdout:1/548: dread d8/d2a/d37/f39 [4194304,4194304] 0 2024-03-18T14:51:08.475 INFO:tasks.workunit.client.0.smithi012.stdout:4/809: read d6/d1a/d92/f5d [174617,1704] 0 2024-03-18T14:51:08.479 INFO:tasks.workunit.client.0.smithi012.stdout:3/836: dread dc/d30/d3a/f4b [0,4194304] 0 2024-03-18T14:51:08.480 INFO:tasks.workunit.client.0.smithi012.stdout:3/837: chown dc/d3c/d4c/d57/d103/d9a/d64 2960651 1 2024-03-18T14:51:08.481 INFO:tasks.workunit.client.1.smithi100.stdout:5/541: rename d29/d2d/d39/d65/d99 to d29/d2d/d32/d92/daa 0 2024-03-18T14:51:08.481 INFO:tasks.workunit.client.1.smithi100.stdout:5/542: write d29/d4a/f80 [471012,13377] 0 2024-03-18T14:51:08.481 INFO:tasks.workunit.client.1.smithi100.stdout:5/543: read - d29/d2d/d39/d65/f98 zero size 2024-03-18T14:51:08.483 INFO:tasks.workunit.client.1.smithi100.stdout:7/574: dwrite d5/f35 [0,4194304] 0 2024-03-18T14:51:08.483 INFO:tasks.workunit.client.1.smithi100.stdout:4/557: dread d9/f3b [0,4194304] 0 2024-03-18T14:51:08.487 INFO:tasks.workunit.client.1.smithi100.stdout:2/566: creat d3/da/de/d20/d82/dad/fca x:0 0 0 2024-03-18T14:51:08.489 INFO:tasks.workunit.client.0.smithi012.stdout:2/914: unlink d0/d6b/dbf/d4f/f105 0 2024-03-18T14:51:08.490 INFO:tasks.workunit.client.0.smithi012.stdout:2/915: chown d0/d6b/dbf/d33/d80/c83 1138810226 1 2024-03-18T14:51:08.490 INFO:tasks.workunit.client.0.smithi012.stdout:2/916: truncate d0/d6b/fe7 671384 0 2024-03-18T14:51:08.492 INFO:tasks.workunit.client.1.smithi100.stdout:0/634: creat dd/d31/d91/d56/d62/fe2 x:0 0 0 2024-03-18T14:51:08.496 INFO:tasks.workunit.client.0.smithi012.stdout:6/812: rmdir d17/d26/d36/d4d/dc6 0 2024-03-18T14:51:08.500 INFO:tasks.workunit.client.1.smithi100.stdout:5/544: rename d29/l3a to d29/d2d/d39/lab 0 2024-03-18T14:51:08.500 INFO:tasks.workunit.client.1.smithi100.stdout:5/545: fdatasync d29/d2d/d39/d71/f77 0 2024-03-18T14:51:08.500 INFO:tasks.workunit.client.1.smithi100.stdout:5/546: read - d29/d2d/d39/d71/d96/fa3 zero size 2024-03-18T14:51:08.501 INFO:tasks.workunit.client.0.smithi012.stdout:4/810: mkdir d6/da/d89/d94/da2/d117 0 2024-03-18T14:51:08.501 INFO:tasks.workunit.client.1.smithi100.stdout:7/575: creat d5/d14/d16/d63/fd6 x:0 0 0 2024-03-18T14:51:08.503 INFO:tasks.workunit.client.0.smithi012.stdout:3/838: creat dc/d3c/d72/df3/d109/f10c x:0 0 0 2024-03-18T14:51:08.503 INFO:tasks.workunit.client.0.smithi012.stdout:3/839: fdatasync f6 0 2024-03-18T14:51:08.503 INFO:tasks.workunit.client.0.smithi012.stdout:3/840: readlink dc/d3c/d4c/d57/d103/ldb 0 2024-03-18T14:51:08.504 INFO:tasks.workunit.client.1.smithi100.stdout:4/558: symlink d9/d27/d62/d7b/d82/la8 0 2024-03-18T14:51:08.505 INFO:tasks.workunit.client.0.smithi012.stdout:2/917: symlink d0/d6b/dbf/d33/dac/dd0/l127 0 2024-03-18T14:51:08.505 INFO:tasks.workunit.client.0.smithi012.stdout:2/918: dread - d0/d107/f116 zero size 2024-03-18T14:51:08.506 INFO:tasks.workunit.client.1.smithi100.stdout:0/635: mknod dd/d31/d91/d7b/ce3 0 2024-03-18T14:51:08.506 INFO:tasks.workunit.client.1.smithi100.stdout:0/636: creat dd/d2b/d95/d79/d7d/da9/db0/fe4 x:0 0 0 2024-03-18T14:51:08.507 INFO:tasks.workunit.client.0.smithi012.stdout:6/813: mknod d17/d38/d51/cf1 0 2024-03-18T14:51:08.508 INFO:tasks.workunit.client.1.smithi100.stdout:5/547: truncate d29/d2d/d39/f4c 3454746 0 2024-03-18T14:51:08.508 INFO:tasks.workunit.client.1.smithi100.stdout:5/548: chown d29/d2d/d39/lab 576549 1 2024-03-18T14:51:08.508 INFO:tasks.workunit.client.1.smithi100.stdout:0/637: dread dd/d31/d91/d56/d71/fa3 [0,4194304] 0 2024-03-18T14:51:08.510 INFO:tasks.workunit.client.1.smithi100.stdout:1/549: dwrite d8/d1e/d11/d2d/f25 [0,4194304] 0 2024-03-18T14:51:08.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/567: dwrite d3/d6/d29/d36/f9e [0,4194304] 0 2024-03-18T14:51:08.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/568: write d3/d6/d29/d36/f4e [4298872,72666] 0 2024-03-18T14:51:08.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/569: write d3/d1a/d76/f9f [550688,60950] 0 2024-03-18T14:51:08.510 INFO:tasks.workunit.client.1.smithi100.stdout:2/570: truncate d3/d1a/d76/fb1 1007762 0 2024-03-18T14:51:08.511 INFO:tasks.workunit.client.0.smithi012.stdout:7/898: dwrite dd/d13/d23/d2f/f9f [4194304,4194304] 0 2024-03-18T14:51:08.511 INFO:tasks.workunit.client.0.smithi012.stdout:4/811: link d6/d19/f2f d6/d1a/d98/f118 0 2024-03-18T14:51:08.511 INFO:tasks.workunit.client.1.smithi100.stdout:7/576: symlink d5/d90/da1/ld7 0 2024-03-18T14:51:08.514 INFO:tasks.workunit.client.0.smithi012.stdout:2/919: rename d0/d6b/dbf/d33/d71/de5/d108/f11a to d0/d6b/dbf/d33/d77/d10a/f128 0 2024-03-18T14:51:08.514 INFO:tasks.workunit.client.0.smithi012.stdout:2/920: readlink d0/d6b/d9f/l123 0 2024-03-18T14:51:08.515 INFO:tasks.workunit.client.0.smithi012.stdout:3/841: write dc/d3c/d4c/d57/d103/fa8 [1573147,118880] 0 2024-03-18T14:51:08.515 INFO:tasks.workunit.client.1.smithi100.stdout:4/559: creat d9/d27/fa9 x:0 0 0 2024-03-18T14:51:08.516 INFO:tasks.workunit.client.0.smithi012.stdout:6/814: getdents d17/d19/d27/d6e/d84 0 2024-03-18T14:51:08.517 INFO:tasks.workunit.client.1.smithi100.stdout:1/550: dread d8/d2a/f44 [0,4194304] 0 2024-03-18T14:51:08.519 INFO:tasks.workunit.client.1.smithi100.stdout:5/549: symlink d29/d2d/d32/d38/d9e/lac 0 2024-03-18T14:51:08.521 INFO:tasks.workunit.client.0.smithi012.stdout:4/812: write d6/da/d89/d95/fd4 [3349166,115333] 0 2024-03-18T14:51:08.521 INFO:tasks.workunit.client.1.smithi100.stdout:0/638: symlink dd/d31/d91/d56/d59/le5 0 2024-03-18T14:51:08.523 INFO:tasks.workunit.client.0.smithi012.stdout:2/921: link d0/d6b/dbf/f5b d0/d6b/dbf/d33/d99/daa/f129 0 2024-03-18T14:51:08.530 INFO:tasks.workunit.client.0.smithi012.stdout:3/842: rmdir dc/d30/d3a/db7 39 2024-03-18T14:51:08.530 INFO:tasks.workunit.client.0.smithi012.stdout:3/843: readlink dc/d3c/d4c/d57/d103/d9a/d54/l71 0 2024-03-18T14:51:08.535 INFO:tasks.workunit.client.0.smithi012.stdout:7/899: dwrite dd/d13/d23/f2b [0,4194304] 0 2024-03-18T14:51:08.535 INFO:tasks.workunit.client.0.smithi012.stdout:7/900: fsync dd/d13/d23/d2f/d53/dc6/df1/ff2 0 2024-03-18T14:51:08.535 INFO:tasks.workunit.client.0.smithi012.stdout:6/815: mknod d17/d19/d98/cf2 0 2024-03-18T14:51:08.537 INFO:tasks.workunit.client.1.smithi100.stdout:2/571: rename d3/d13/c1e to d3/d13/d40/ccb 0 2024-03-18T14:51:08.537 INFO:tasks.workunit.client.1.smithi100.stdout:2/572: write d3/d1a/d6c/f8b [5229703,35643] 0 2024-03-18T14:51:08.538 INFO:tasks.workunit.client.1.smithi100.stdout:2/573: readlink d3/d13/l14 0 2024-03-18T14:51:08.541 INFO:tasks.workunit.client.1.smithi100.stdout:4/560: link d9/d17/l4f d9/d13/d25/laa 0 2024-03-18T14:51:08.545 INFO:tasks.workunit.client.1.smithi100.stdout:5/550: unlink f23 0 2024-03-18T14:51:08.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/639: creat dd/d31/fe6 x:0 0 0 2024-03-18T14:51:08.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/640: creat dd/d2b/d95/d79/d7d/da9/db0/fe7 x:0 0 0 2024-03-18T14:51:08.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/641: write dd/f1c [1149950,95989] 0 2024-03-18T14:51:08.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/642: creat dd/d61/fe8 x:0 0 0 2024-03-18T14:51:08.548 INFO:tasks.workunit.client.1.smithi100.stdout:0/643: fdatasync dd/d31/d91/d56/d62/d6b/fc5 0 2024-03-18T14:51:08.549 INFO:tasks.workunit.client.0.smithi012.stdout:7/901: dread dd/d13/d23/f99 [0,4194304] 0 2024-03-18T14:51:08.550 INFO:tasks.workunit.client.0.smithi012.stdout:3/844: getdents dc/d30/d3a/da0/dc5 0 2024-03-18T14:51:08.550 INFO:tasks.workunit.client.0.smithi012.stdout:3/845: dread - dc/d30/dab/fff zero size 2024-03-18T14:51:08.550 INFO:tasks.workunit.client.0.smithi012.stdout:3/846: dread - dc/d3c/d72/df3/d109/f10c zero size 2024-03-18T14:51:08.550 INFO:tasks.workunit.client.0.smithi012.stdout:3/847: creat dc/d91/f10d x:0 0 0 2024-03-18T14:51:08.552 INFO:tasks.workunit.client.1.smithi100.stdout:7/577: rename d5/d14/d16/d2e/d62/dd1/d86/f50 to d5/d14/d16/d2e/d62/d66/d4f/fd8 0 2024-03-18T14:51:08.552 INFO:tasks.workunit.client.1.smithi100.stdout:7/578: getdents d5/d30/d4d/db4 0 2024-03-18T14:51:08.553 INFO:tasks.workunit.client.0.smithi012.stdout:7/902: mkdir dd/d13/d23/d76/d6f/db3/dc7/d136 0 2024-03-18T14:51:08.553 INFO:tasks.workunit.client.0.smithi012.stdout:7/903: fdatasync dd/d13/d23/d2f/d53/fa5 0 2024-03-18T14:51:08.555 INFO:tasks.workunit.client.1.smithi100.stdout:5/551: mkdir d29/d2d/d32/dad 0 2024-03-18T14:51:08.556 INFO:tasks.workunit.client.0.smithi012.stdout:3/848: symlink dc/d3c/d4c/d57/d103/d9a/l10e 0 2024-03-18T14:51:08.556 INFO:tasks.workunit.client.0.smithi012.stdout:3/849: chown dc/d3c/d4c/d57/d103/d9a/d54/f10a 22801 1 2024-03-18T14:51:08.558 INFO:tasks.workunit.client.0.smithi012.stdout:7/904: mknod dd/d13/d103/d12b/c137 0 2024-03-18T14:51:08.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/922: dwrite d0/d6b/dbf/d33/d80/de1/f10b [0,4194304] 0 2024-03-18T14:51:08.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/923: write d0/da2/fd6 [5185226,19761] 0 2024-03-18T14:51:08.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/924: fdatasync d0/d6b/dbf/f101 0 2024-03-18T14:51:08.561 INFO:tasks.workunit.client.0.smithi012.stdout:2/925: write d0/d6b/dbf/f1b [2359986,127277] 0 2024-03-18T14:51:08.561 INFO:tasks.workunit.client.0.smithi012.stdout:4/813: dwrite d6/d1a/d30/d46/d6f/d7c/fed [0,4194304] 0 2024-03-18T14:51:08.562 INFO:tasks.workunit.client.0.smithi012.stdout:4/814: chown d6/da/d58 187 1 2024-03-18T14:51:08.562 INFO:tasks.workunit.client.0.smithi012.stdout:4/815: chown d6/d1a/d30/c61 20 1 2024-03-18T14:51:08.562 INFO:tasks.workunit.client.1.smithi100.stdout:2/574: dwrite d3/d6/d29/d36/d49/f54 [0,4194304] 0 2024-03-18T14:51:08.563 INFO:tasks.workunit.client.0.smithi012.stdout:6/816: dwrite d17/f22 [0,4194304] 0 2024-03-18T14:51:08.565 INFO:tasks.workunit.client.0.smithi012.stdout:7/905: mkdir dd/d13/d23/d76/d109/d129/d48/d138 0 2024-03-18T14:51:08.565 INFO:tasks.workunit.client.0.smithi012.stdout:7/906: chown dd/d13/d23/d76/d6b/l7d 368000879 1 2024-03-18T14:51:08.568 INFO:tasks.workunit.client.1.smithi100.stdout:0/644: symlink dd/le9 0 2024-03-18T14:51:08.570 INFO:tasks.workunit.client.0.smithi012.stdout:2/926: dread d0/d10/f16 [0,4194304] 0 2024-03-18T14:51:08.571 INFO:tasks.workunit.client.0.smithi012.stdout:3/850: getdents dc/d30/d35 0 2024-03-18T14:51:08.574 INFO:tasks.workunit.client.1.smithi100.stdout:9/572: sync 2024-03-18T14:51:08.574 INFO:tasks.workunit.client.1.smithi100.stdout:6/544: sync 2024-03-18T14:51:08.576 INFO:tasks.workunit.client.1.smithi100.stdout:4/561: rename l3 to d9/d13/d39/d76/lab 0 2024-03-18T14:51:08.581 INFO:tasks.workunit.client.1.smithi100.stdout:7/579: link d5/d14/d16/d2e/d62/dd1/f74 d5/d22/d24/d6e/fd9 0 2024-03-18T14:51:08.585 INFO:tasks.workunit.client.1.smithi100.stdout:5/552: creat d29/d2d/d39/fae x:0 0 0 2024-03-18T14:51:08.585 INFO:tasks.workunit.client.1.smithi100.stdout:1/551: write d8/d30/f35 [52840,104395] 0 2024-03-18T14:51:08.586 INFO:tasks.workunit.client.0.smithi012.stdout:4/816: getdents d6/da/d89/d94/dbe/dd6 0 2024-03-18T14:51:08.593 INFO:tasks.workunit.client.0.smithi012.stdout:6/817: symlink d17/d19/lf3 0 2024-03-18T14:51:08.595 INFO:tasks.workunit.client.1.smithi100.stdout:2/575: rmdir d3/d9a 0 2024-03-18T14:51:08.596 INFO:tasks.workunit.client.1.smithi100.stdout:8/596: sync 2024-03-18T14:51:08.596 INFO:tasks.workunit.client.1.smithi100.stdout:8/597: truncate d4/dc/f7b 4817185 0 2024-03-18T14:51:08.596 INFO:tasks.workunit.client.1.smithi100.stdout:8/598: fsync d4/dc/d1f/d59/fbd 0 2024-03-18T14:51:08.597 INFO:tasks.workunit.client.0.smithi012.stdout:7/907: symlink dd/d13/d23/d2f/d7a/dab/dbc/l139 0 2024-03-18T14:51:08.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/645: dwrite dd/d31/d91/d56/fd2 [0,4194304] 0 2024-03-18T14:51:08.598 INFO:tasks.workunit.client.1.smithi100.stdout:0/646: creat dd/d61/fea x:0 0 0 2024-03-18T14:51:08.598 INFO:tasks.workunit.client.1.smithi100.stdout:9/573: link da/d2c/d2f/l3d da/d46/lb3 0 2024-03-18T14:51:08.602 INFO:tasks.workunit.client.0.smithi012.stdout:2/927: unlink d0/d6b/dbf/d33/d71/de5/f117 0 2024-03-18T14:51:08.603 INFO:tasks.workunit.client.1.smithi100.stdout:4/562: truncate d9/f3b 479011 0 2024-03-18T14:51:08.603 INFO:tasks.workunit.client.1.smithi100.stdout:4/563: dread - d9/d13/d25/fa5 zero size 2024-03-18T14:51:08.607 INFO:tasks.workunit.client.0.smithi012.stdout:3/851: dwrite dc/d30/f51 [0,4194304] 0 2024-03-18T14:51:08.607 INFO:tasks.workunit.client.0.smithi012.stdout:3/852: stat dc/d30/fd6 0 2024-03-18T14:51:08.608 INFO:tasks.workunit.client.0.smithi012.stdout:4/817: mknod d6/d19/c119 0 2024-03-18T14:51:08.608 INFO:tasks.workunit.client.0.smithi012.stdout:4/818: creat d6/da/d89/d97/dc5/f11a x:0 0 0 2024-03-18T14:51:08.609 INFO:tasks.workunit.client.1.smithi100.stdout:1/552: dwrite d8/d1e/f45 [0,4194304] 0 2024-03-18T14:51:08.609 INFO:tasks.workunit.client.1.smithi100.stdout:1/553: readlink d8/d1e/d11/d8a/l98 0 2024-03-18T14:51:08.609 INFO:tasks.workunit.client.1.smithi100.stdout:7/580: truncate d5/d14/d16/f1d 781577 0 2024-03-18T14:51:08.612 INFO:tasks.workunit.client.0.smithi012.stdout:6/818: getdents d17/d26/d36 0 2024-03-18T14:51:08.614 INFO:tasks.workunit.client.0.smithi012.stdout:7/908: rmdir dd/d13/d23/d2f/de1/d122 0 2024-03-18T14:51:08.626 INFO:tasks.workunit.client.1.smithi100.stdout:5/553: mknod d29/d2d/d8f/caf 0 2024-03-18T14:51:08.627 INFO:tasks.workunit.client.1.smithi100.stdout:5/554: dread d29/d2d/d39/d83/f85 [0,4194304] 0 2024-03-18T14:51:08.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/576: rename c2 to d3/d1a/ccc 0 2024-03-18T14:51:08.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/577: truncate d3/d6/d5f/d6a/fb2 783428 0 2024-03-18T14:51:08.631 INFO:tasks.workunit.client.1.smithi100.stdout:2/578: chown d3/d13/d40/d53 1279 1 2024-03-18T14:51:08.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/599: dwrite d4/dc/dd/de/d21/d23/f28 [0,4194304] 0 2024-03-18T14:51:08.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/600: chown d4/dc/dd/de/d21/ca7 7782 1 2024-03-18T14:51:08.636 INFO:tasks.workunit.client.1.smithi100.stdout:8/601: read - d4/dc/d1f/d33/f8d zero size 2024-03-18T14:51:08.637 INFO:tasks.workunit.client.0.smithi012.stdout:6/819: creat d17/d18/ff4 x:0 0 0 2024-03-18T14:51:08.637 INFO:tasks.workunit.client.0.smithi012.stdout:6/820: dread - d17/d19/d27/db1/dd7/fe1 zero size 2024-03-18T14:51:08.637 INFO:tasks.workunit.client.0.smithi012.stdout:4/819: dwrite d6/d1a/d30/d46/d6f/d8b/fee [0,4194304] 0 2024-03-18T14:51:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:6/821: unlink d17/d19/d27/d2f/c59 0 2024-03-18T14:51:08.643 INFO:tasks.workunit.client.1.smithi100.stdout:9/574: symlink da/dd/d29/d71/d72/db2/lb4 0 2024-03-18T14:51:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:7/909: dwrite dd/d13/d23/d2f/d53/f5d [0,4194304] 0 2024-03-18T14:51:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:7/910: write dd/d13/d23/d60/d9c/f135 [997659,20229] 0 2024-03-18T14:51:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:2/928: dwrite d0/d6b/dbf/d33/d77/df5/d103/d61/f6d [0,4194304] 0 2024-03-18T14:51:08.646 INFO:tasks.workunit.client.1.smithi100.stdout:4/564: truncate d9/d13/d25/f48 1162166 0 2024-03-18T14:51:08.646 INFO:tasks.workunit.client.1.smithi100.stdout:4/565: truncate d9/d27/d62/f9a 833005 0 2024-03-18T14:51:08.648 INFO:tasks.workunit.client.0.smithi012.stdout:4/820: mknod d6/da/d89/d97/dc5/c11b 0 2024-03-18T14:51:08.650 INFO:tasks.workunit.client.0.smithi012.stdout:6/822: mknod d17/d19/d27/d58/cf5 0 2024-03-18T14:51:08.650 INFO:tasks.workunit.client.0.smithi012.stdout:6/823: chown d17/d26/c31 30 1 2024-03-18T14:51:08.651 INFO:tasks.workunit.client.1.smithi100.stdout:1/554: symlink d8/d30/d61/da2/la5 0 2024-03-18T14:51:08.651 INFO:tasks.workunit.client.1.smithi100.stdout:1/555: chown d8/d1e/c58 5115 1 2024-03-18T14:51:08.651 INFO:tasks.workunit.client.1.smithi100.stdout:1/556: chown d8/d1e/d11/d27 576 1 2024-03-18T14:51:08.651 INFO:tasks.workunit.client.1.smithi100.stdout:1/557: chown d8/d1e/c58 1 1 2024-03-18T14:51:08.653 INFO:tasks.workunit.client.1.smithi100.stdout:5/555: dwrite d29/d2d/d32/d38/fa4 [0,4194304] 0 2024-03-18T14:51:08.653 INFO:tasks.workunit.client.1.smithi100.stdout:7/581: creat d5/d22/d3b/d9f/da0/db1/fda x:0 0 0 2024-03-18T14:51:08.655 INFO:tasks.workunit.client.0.smithi012.stdout:7/911: truncate dd/d13/f73 1946007 0 2024-03-18T14:51:08.655 INFO:tasks.workunit.client.0.smithi012.stdout:7/912: readlink dd/d13/d23/d76/d109/d129/la0 0 2024-03-18T14:51:08.655 INFO:tasks.workunit.client.0.smithi012.stdout:7/913: fsync dd/d13/d23/d76/d6f/db3/dc7/fea 0 2024-03-18T14:51:08.664 INFO:tasks.workunit.client.0.smithi012.stdout:4/821: mkdir d6/d19/d6c/d96/df0/d11c 0 2024-03-18T14:51:08.665 INFO:tasks.workunit.client.1.smithi100.stdout:2/579: truncate d3/d6/f44 3951971 0 2024-03-18T14:51:08.665 INFO:tasks.workunit.client.1.smithi100.stdout:2/580: truncate d3/da/de/fa5 1582723 0 2024-03-18T14:51:08.666 INFO:tasks.workunit.client.1.smithi100.stdout:2/581: truncate d3/d13/d40/fa7 1424357 0 2024-03-18T14:51:08.666 INFO:tasks.workunit.client.1.smithi100.stdout:2/582: write d3/d1a/f60 [714424,56385] 0 2024-03-18T14:51:08.666 INFO:tasks.workunit.client.0.smithi012.stdout:6/824: getdents d17/d26 0 2024-03-18T14:51:08.674 INFO:tasks.workunit.client.0.smithi012.stdout:6/825: dread d17/d19/d27/fd6 [0,4194304] 0 2024-03-18T14:51:08.674 INFO:tasks.workunit.client.0.smithi012.stdout:6/826: chown d17/f2c 31475342 1 2024-03-18T14:51:08.684 INFO:tasks.workunit.client.0.smithi012.stdout:6/827: read d17/d26/f4a [3047777,65598] 0 2024-03-18T14:51:08.684 INFO:tasks.workunit.client.0.smithi012.stdout:6/828: fsync d17/d18/f45 0 2024-03-18T14:51:08.684 INFO:tasks.workunit.client.0.smithi012.stdout:6/829: write d17/d26/d36/d4d/d82/fe3 [291115,114563] 0 2024-03-18T14:51:08.687 INFO:tasks.workunit.client.0.smithi012.stdout:7/914: mknod dd/d13/d23/d76/d109/d129/c13a 0 2024-03-18T14:51:08.690 INFO:tasks.workunit.client.1.smithi100.stdout:8/602: link d4/dc/dd/de/d19/d37/f65 d4/dc/d1f/d59/d6c/d75/fd1 0 2024-03-18T14:51:08.691 INFO:tasks.workunit.client.1.smithi100.stdout:2/583: dwrite d3/d13/d40/d53/f65 [0,4194304] 0 2024-03-18T14:51:08.693 INFO:tasks.workunit.client.1.smithi100.stdout:9/575: rename da/d46/f8e to da/db/db0/fb5 0 2024-03-18T14:51:08.693 INFO:tasks.workunit.client.1.smithi100.stdout:9/576: creat da/dd/d29/d71/d72/fb6 x:0 0 0 2024-03-18T14:51:08.693 INFO:tasks.workunit.client.1.smithi100.stdout:9/577: write da/db/f13 [5187797,13289] 0 2024-03-18T14:51:08.693 INFO:tasks.workunit.client.0.smithi012.stdout:2/929: rmdir d0/d6b/dbf/d33/dac 39 2024-03-18T14:51:08.696 INFO:tasks.workunit.client.0.smithi012.stdout:4/822: mkdir d6/da/d89/d94/d10d/d11d 0 2024-03-18T14:51:08.698 INFO:tasks.workunit.client.1.smithi100.stdout:4/566: unlink d9/d13/d39/d76/l83 0 2024-03-18T14:51:08.701 INFO:tasks.workunit.client.0.smithi012.stdout:6/830: rename d17/d18/ff4 to d17/d19/d27/d58/d91/ff6 0 2024-03-18T14:51:08.703 INFO:tasks.workunit.client.1.smithi100.stdout:7/582: rmdir d5/d14/d16/d2e/d62/dd1/d86/d85/d87/dc5 0 2024-03-18T14:51:08.709 INFO:tasks.workunit.client.1.smithi100.stdout:8/603: dwrite d4/dc/d1f/d33/f8d [0,4194304] 0 2024-03-18T14:51:08.710 INFO:tasks.workunit.client.0.smithi012.stdout:2/930: dwrite d0/d3c/d40/f5e [0,4194304] 0 2024-03-18T14:51:08.716 INFO:tasks.workunit.client.1.smithi100.stdout:2/584: creat d3/da/de/d6e/fcd x:0 0 0 2024-03-18T14:51:08.718 INFO:tasks.workunit.client.0.smithi012.stdout:2/931: dread d0/d3c/f115 [0,4194304] 0 2024-03-18T14:51:08.720 INFO:tasks.workunit.client.0.smithi012.stdout:3/853: getdents dc/d3c/d4c/d57/d103/d9a 0 2024-03-18T14:51:08.725 INFO:tasks.workunit.client.1.smithi100.stdout:4/567: mkdir d9/d27/d62/d7b/d82/dac 0 2024-03-18T14:51:08.725 INFO:tasks.workunit.client.1.smithi100.stdout:4/568: creat d9/d13/d39/fad x:0 0 0 2024-03-18T14:51:08.727 INFO:tasks.workunit.client.1.smithi100.stdout:7/583: creat d5/d30/d4d/db4/fdb x:0 0 0 2024-03-18T14:51:08.728 INFO:tasks.workunit.client.0.smithi012.stdout:2/932: dread d0/d3c/d40/d47/d85/fba [0,4194304] 0 2024-03-18T14:51:08.729 INFO:tasks.workunit.client.0.smithi012.stdout:4/823: unlink d6/da/ce6 0 2024-03-18T14:51:08.729 INFO:tasks.workunit.client.1.smithi100.stdout:4/569: dread d9/f22 [0,4194304] 0 2024-03-18T14:51:08.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/831: mkdir d17/d38/d51/df7 0 2024-03-18T14:51:08.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/832: dread - d17/d38/fdc zero size 2024-03-18T14:51:08.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/833: fsync d17/d26/f4a 0 2024-03-18T14:51:08.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/834: fsync d17/d38/fc7 0 2024-03-18T14:51:08.732 INFO:tasks.workunit.client.0.smithi012.stdout:6/835: creat d17/d19/d27/d2f/ff8 x:0 0 0 2024-03-18T14:51:08.734 INFO:tasks.workunit.client.1.smithi100.stdout:8/604: symlink d4/d96/ld2 0 2024-03-18T14:51:08.735 INFO:tasks.workunit.client.1.smithi100.stdout:8/605: read d4/d5a/dbb/f6b [133667,82588] 0 2024-03-18T14:51:08.735 INFO:tasks.workunit.client.0.smithi012.stdout:3/854: truncate dc/d3c/d4c/d57/d103/fa8 2196812 0 2024-03-18T14:51:08.735 INFO:tasks.workunit.client.0.smithi012.stdout:3/855: creat dc/d3c/d4c/f10f x:0 0 0 2024-03-18T14:51:08.736 INFO:tasks.workunit.client.1.smithi100.stdout:4/570: dread d9/d13/d25/d51/f5d [0,4194304] 0 2024-03-18T14:51:08.738 INFO:tasks.workunit.client.0.smithi012.stdout:2/933: symlink d0/d6c/de2/l12a 0 2024-03-18T14:51:08.738 INFO:tasks.workunit.client.0.smithi012.stdout:2/934: stat d0/d6b/fe7 0 2024-03-18T14:51:08.740 INFO:tasks.workunit.client.0.smithi012.stdout:4/824: creat d6/f11e x:0 0 0 2024-03-18T14:51:08.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: pgmap v8: 65 pgs: 65 active+clean; 2.4 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 157 MiB/s rd, 170 MiB/s wr, 276 op/s 2024-03-18T14:51:08.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: Standby manager daemon smithi012.tmscgt started 2024-03-18T14:51:08.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/crt"}]: dispatch 2024-03-18T14:51:08.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:51:08.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/key"}]: dispatch 2024-03-18T14:51:08.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:08 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:51:08.746 INFO:tasks.workunit.client.0.smithi012.stdout:6/836: rmdir d17/d26 39 2024-03-18T14:51:08.751 INFO:tasks.workunit.client.0.smithi012.stdout:6/837: dread d17/d26/d36/d4d/d82/f8b [0,4194304] 0 2024-03-18T14:51:08.751 INFO:tasks.workunit.client.0.smithi012.stdout:2/935: symlink d0/d6b/dbf/d31/l12b 0 2024-03-18T14:51:08.751 INFO:tasks.workunit.client.0.smithi012.stdout:2/936: truncate d0/d6b/dbf/d33/d77/d10a/f128 635696 0 2024-03-18T14:51:08.751 INFO:tasks.workunit.client.0.smithi012.stdout:2/937: stat d0/d6b/dbf/d33/d71/de5 0 2024-03-18T14:51:08.752 INFO:tasks.workunit.client.1.smithi100.stdout:7/584: symlink d5/d14/d16/d2e/d62/dd1/d86/d85/d87/ldc 0 2024-03-18T14:51:08.753 INFO:tasks.workunit.client.0.smithi012.stdout:4/825: mkdir d6/d1a/d30/d46/d6f/d8b/d11f 0 2024-03-18T14:51:08.754 INFO:tasks.workunit.client.0.smithi012.stdout:3/856: dwrite dc/d30/f51 [0,4194304] 0 2024-03-18T14:51:08.754 INFO:tasks.workunit.client.0.smithi012.stdout:3/857: fsync dc/d30/f99 0 2024-03-18T14:51:08.755 INFO:tasks.workunit.client.1.smithi100.stdout:8/606: rename d4/dc/dd/de/d19/db0/fc1 to d4/dc/dd/de/d21/d23/fd3 0 2024-03-18T14:51:08.756 INFO:tasks.workunit.client.0.smithi012.stdout:6/838: creat d17/d19/d27/d6e/ff9 x:0 0 0 2024-03-18T14:51:08.757 INFO:tasks.workunit.client.1.smithi100.stdout:4/571: unlink d9/l15 0 2024-03-18T14:51:08.760 INFO:tasks.workunit.client.1.smithi100.stdout:7/585: symlink d5/d14/d16/d2e/d62/dc3/ldd 0 2024-03-18T14:51:08.763 INFO:tasks.workunit.client.0.smithi012.stdout:2/938: dread d0/f6e [0,4194304] 0 2024-03-18T14:51:08.764 INFO:tasks.workunit.client.1.smithi100.stdout:8/607: mknod d4/dc/dd/de/d21/d23/d47/d56/cd4 0 2024-03-18T14:51:08.765 INFO:tasks.workunit.client.0.smithi012.stdout:3/858: creat dc/d30/d3a/da0/f110 x:0 0 0 2024-03-18T14:51:08.766 INFO:tasks.workunit.client.1.smithi100.stdout:4/572: mkdir d9/d13/dae 0 2024-03-18T14:51:08.766 INFO:tasks.workunit.client.1.smithi100.stdout:4/573: fdatasync d9/d17/f23 0 2024-03-18T14:51:08.768 INFO:tasks.workunit.client.1.smithi100.stdout:7/586: mknod d5/d30/d4d/d5a/cde 0 2024-03-18T14:51:08.768 INFO:tasks.workunit.client.1.smithi100.stdout:7/587: creat d5/d14/d16/d2e/d62/fdf x:0 0 0 2024-03-18T14:51:08.768 INFO:tasks.workunit.client.1.smithi100.stdout:7/588: truncate d5/fd 1593429 0 2024-03-18T14:51:08.772 INFO:tasks.workunit.client.1.smithi100.stdout:8/608: mkdir d4/dc/dd/de/d19/db0/dd5 0 2024-03-18T14:51:08.774 INFO:tasks.workunit.client.0.smithi012.stdout:6/839: dwrite d17/d38/d51/d66/faf [0,4194304] 0 2024-03-18T14:51:08.775 INFO:tasks.workunit.client.0.smithi012.stdout:2/939: unlink d0/d3c/d40/d47/d85/fba 0 2024-03-18T14:51:08.775 INFO:tasks.workunit.client.0.smithi012.stdout:2/940: write d0/da2/fd6 [1829813,26133] 0 2024-03-18T14:51:08.775 INFO:tasks.workunit.client.0.smithi012.stdout:2/941: write d0/d6b/dbf/d33/d77/df5/d103/fc4 [4292225,90548] 0 2024-03-18T14:51:08.775 INFO:tasks.workunit.client.0.smithi012.stdout:3/859: dread dc/f48 [0,4194304] 0 2024-03-18T14:51:08.778 INFO:tasks.workunit.client.0.smithi012.stdout:4/826: rename d6/da/d89/d97/dc5/dd3 to d6/da/d58/d113/d120 0 2024-03-18T14:51:08.786 INFO:tasks.workunit.client.0.smithi012.stdout:6/840: rmdir d17/d26 39 2024-03-18T14:51:08.786 INFO:tasks.workunit.client.0.smithi012.stdout:6/841: readlink d17/d26/d36/l5f 0 2024-03-18T14:51:08.787 INFO:tasks.workunit.client.1.smithi100.stdout:4/574: dwrite d9/f10 [0,4194304] 0 2024-03-18T14:51:08.789 INFO:tasks.workunit.client.1.smithi100.stdout:8/609: dread d4/fa [0,4194304] 0 2024-03-18T14:51:08.789 INFO:tasks.workunit.client.1.smithi100.stdout:8/610: fsync d4/dc/dd/f44 0 2024-03-18T14:51:08.790 INFO:tasks.workunit.client.1.smithi100.stdout:8/611: chown d4/f6 637 1 2024-03-18T14:51:08.790 INFO:tasks.workunit.client.0.smithi012.stdout:3/860: truncate dc/d3c/d59/fdd 817534 0 2024-03-18T14:51:08.792 INFO:tasks.workunit.client.0.smithi012.stdout:2/942: mknod d0/d6b/dbf/d33/d77/c12c 0 2024-03-18T14:51:08.794 INFO:tasks.workunit.client.1.smithi100.stdout:8/612: unlink d4/f9 0 2024-03-18T14:51:08.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: pgmap v8: 65 pgs: 65 active+clean; 2.4 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 157 MiB/s rd, 170 MiB/s wr, 276 op/s 2024-03-18T14:51:08.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: Standby manager daemon smithi012.tmscgt started 2024-03-18T14:51:08.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/crt"}]: dispatch 2024-03-18T14:51:08.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:51:08.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/key"}]: dispatch 2024-03-18T14:51:08.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:08 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1018101147' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:51:08.796 INFO:tasks.workunit.client.0.smithi012.stdout:3/861: unlink dc/d3c/d4c/lce 0 2024-03-18T14:51:08.796 INFO:tasks.workunit.client.0.smithi012.stdout:3/862: dread - dc/d91/fea zero size 2024-03-18T14:51:08.796 INFO:tasks.workunit.client.0.smithi012.stdout:3/863: creat dc/d30/dab/f111 x:0 0 0 2024-03-18T14:51:08.797 INFO:tasks.workunit.client.1.smithi100.stdout:8/613: mkdir d4/dc/d31/d85/dd6 0 2024-03-18T14:51:08.797 INFO:tasks.workunit.client.1.smithi100.stdout:8/614: creat d4/dc/dd/de/d19/d37/fd7 x:0 0 0 2024-03-18T14:51:08.798 INFO:tasks.workunit.client.1.smithi100.stdout:8/615: dread d4/dc/d31/d85/fb7 [0,4194304] 0 2024-03-18T14:51:08.799 INFO:tasks.workunit.client.0.smithi012.stdout:3/864: mkdir dc/d30/d3a/d112 0 2024-03-18T14:51:08.802 INFO:tasks.workunit.client.0.smithi012.stdout:6/842: write d17/d38/d51/dcc/dd8/f48 [1458163,14988] 0 2024-03-18T14:51:08.802 INFO:tasks.workunit.client.1.smithi100.stdout:8/616: mkdir d4/dc/dd/d48/dd8 0 2024-03-18T14:51:08.810 INFO:tasks.workunit.client.1.smithi100.stdout:7/589: dwrite d5/d22/d24/f91 [0,4194304] 0 2024-03-18T14:51:08.811 INFO:tasks.workunit.client.0.smithi012.stdout:4/827: dwrite d6/d1a/d30/d46/d6f/d8b/fb0 [0,4194304] 0 2024-03-18T14:51:08.811 INFO:tasks.workunit.client.0.smithi012.stdout:4/828: truncate d6/f11e 615412 0 2024-03-18T14:51:08.811 INFO:tasks.workunit.client.0.smithi012.stdout:4/829: chown d6/da/d89/d97 50575 1 2024-03-18T14:51:08.818 INFO:tasks.workunit.client.0.smithi012.stdout:4/830: dread d6/d1a/d30/d46/d6f/d7c/f81 [0,4194304] 0 2024-03-18T14:51:08.818 INFO:tasks.workunit.client.0.smithi012.stdout:4/831: stat d6/d19/d39/lb9 0 2024-03-18T14:51:08.819 INFO:tasks.workunit.client.1.smithi100.stdout:7/590: truncate d5/d14/d16/d44/fa7 338491 0 2024-03-18T14:51:08.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/591: write d5/d22/d24/f2d [90399,112706] 0 2024-03-18T14:51:08.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/592: truncate d5/f72 5215761 0 2024-03-18T14:51:08.820 INFO:tasks.workunit.client.1.smithi100.stdout:7/593: fdatasync d5/d30/d4d/d5a/f82 0 2024-03-18T14:51:08.822 INFO:tasks.workunit.client.1.smithi100.stdout:9/578: truncate da/db/f13 4338570 0 2024-03-18T14:51:08.823 INFO:tasks.workunit.client.1.smithi100.stdout:9/579: creat da/d46/d4d/d62/fb7 x:0 0 0 2024-03-18T14:51:08.825 INFO:tasks.workunit.client.1.smithi100.stdout:9/580: symlink da/dd/d3c/lb8 0 2024-03-18T14:51:08.828 INFO:tasks.workunit.client.0.smithi012.stdout:6/843: dwrite d17/d19/f6d [4194304,4194304] 0 2024-03-18T14:51:08.830 INFO:tasks.workunit.client.0.smithi012.stdout:6/844: write d17/d26/d36/d4d/f81 [44684,1232] 0 2024-03-18T14:51:08.830 INFO:tasks.workunit.client.0.smithi012.stdout:6/845: write d17/d38/d51/d66/fcf [211131,66330] 0 2024-03-18T14:51:08.834 INFO:tasks.workunit.client.1.smithi100.stdout:8/617: dwrite d4/dc/d31/f3d [0,4194304] 0 2024-03-18T14:51:08.834 INFO:tasks.workunit.client.0.smithi012.stdout:3/865: dwrite dc/d91/fea [0,4194304] 0 2024-03-18T14:51:08.835 INFO:tasks.workunit.client.0.smithi012.stdout:4/832: write d6/d27/f56 [1865921,45462] 0 2024-03-18T14:51:08.836 INFO:tasks.workunit.client.1.smithi100.stdout:8/618: rename d4/dc/dd/de/d21/d23/f78 to d4/dc/d31/fd9 0 2024-03-18T14:51:08.836 INFO:tasks.workunit.client.1.smithi100.stdout:8/619: fsync d4/dc/d1f/f5d 0 2024-03-18T14:51:08.839 INFO:tasks.workunit.client.1.smithi100.stdout:8/620: truncate d4/dc/d1f/f5d 3869554 0 2024-03-18T14:51:08.840 INFO:tasks.workunit.client.0.smithi012.stdout:4/833: mknod d6/d19/d6c/d96/df0/d11c/c121 0 2024-03-18T14:51:08.840 INFO:tasks.workunit.client.0.smithi012.stdout:4/834: chown d6/d1a/c111 6685573 1 2024-03-18T14:51:08.840 INFO:tasks.workunit.client.0.smithi012.stdout:4/835: dread - d6/d1a/d30/fd5 zero size 2024-03-18T14:51:08.841 INFO:tasks.workunit.client.1.smithi100.stdout:8/621: rename d4/dc/d97/fa3 to d4/dc/dd/d48/fda 0 2024-03-18T14:51:08.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/836: mkdir d6/da/d89/d94/d10d/d11d/d122 0 2024-03-18T14:51:08.843 INFO:tasks.workunit.client.1.smithi100.stdout:8/622: mkdir d4/dc/d1f/d59/d6c/ddb 0 2024-03-18T14:51:08.844 INFO:tasks.workunit.client.1.smithi100.stdout:8/623: chown d4/dc/dd/de/d21/d23/f28 27 1 2024-03-18T14:51:08.844 INFO:tasks.workunit.client.1.smithi100.stdout:8/624: symlink d4/dc/dd/de/d21/d23/d47/ldc 0 2024-03-18T14:51:08.845 INFO:tasks.workunit.client.1.smithi100.stdout:8/625: creat d4/dc/d1f/d59/d6c/fdd x:0 0 0 2024-03-18T14:51:08.845 INFO:tasks.workunit.client.1.smithi100.stdout:8/626: chown d4/dc/dd/de/f57 502527 1 2024-03-18T14:51:08.845 INFO:tasks.workunit.client.0.smithi012.stdout:4/837: dread d6/da/d89/d94/dbe/dd6/ffa [0,4194304] 0 2024-03-18T14:51:08.845 INFO:tasks.workunit.client.0.smithi012.stdout:4/838: fdatasync d6/f80 0 2024-03-18T14:51:08.853 INFO:tasks.workunit.client.1.smithi100.stdout:8/627: write d4/dc/dd/de/d19/d37/f7f [3588757,22345] 0 2024-03-18T14:51:08.853 INFO:tasks.workunit.client.1.smithi100.stdout:8/628: creat d4/dc/d31/d66/fde x:0 0 0 2024-03-18T14:51:08.853 INFO:tasks.workunit.client.1.smithi100.stdout:8/629: chown f1 4890 1 2024-03-18T14:51:08.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/630: mknod d4/dc/d97/cdf 0 2024-03-18T14:51:08.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/631: write d4/fcc [1631981,34240] 0 2024-03-18T14:51:08.858 INFO:tasks.workunit.client.0.smithi012.stdout:3/866: dread dc/d3c/f74 [0,4194304] 0 2024-03-18T14:51:08.859 INFO:tasks.workunit.client.1.smithi100.stdout:9/581: dwrite da/d46/d8b/f3f [0,4194304] 0 2024-03-18T14:51:08.859 INFO:tasks.workunit.client.1.smithi100.stdout:8/632: getdents d4/d3c 0 2024-03-18T14:51:08.861 INFO:tasks.workunit.client.0.smithi012.stdout:3/867: mknod dc/d3c/d4c/d57/c113 0 2024-03-18T14:51:08.861 INFO:tasks.workunit.client.0.smithi012.stdout:3/868: write dc/d3c/d4c/d57/fa1 [5122260,42736] 0 2024-03-18T14:51:08.865 INFO:tasks.workunit.client.0.smithi012.stdout:3/869: write dc/d30/d35/f70 [385569,36891] 0 2024-03-18T14:51:08.867 INFO:tasks.workunit.client.0.smithi012.stdout:6/846: dwrite d17/d26/d36/f67 [0,4194304] 0 2024-03-18T14:51:08.867 INFO:tasks.workunit.client.0.smithi012.stdout:4/839: dwrite d6/d1a/d30/d46/f6b [0,4194304] 0 2024-03-18T14:51:08.867 INFO:tasks.workunit.client.0.smithi012.stdout:4/840: dread - d6/da/d89/d97/fa9 zero size 2024-03-18T14:51:08.867 INFO:tasks.workunit.client.1.smithi100.stdout:9/582: unlink da/d2c/d2f/f5b 0 2024-03-18T14:51:08.870 INFO:tasks.workunit.client.0.smithi012.stdout:3/870: creat dc/d3c/d72/df3/f114 x:0 0 0 2024-03-18T14:51:08.873 INFO:tasks.workunit.client.1.smithi100.stdout:8/633: link d4/dc/dd/de/f57 d4/dc/dd/de/d19/fe0 0 2024-03-18T14:51:08.876 INFO:tasks.workunit.client.1.smithi100.stdout:9/583: rename da/dd/d3b/l6a to da/d2c/d70/lb9 0 2024-03-18T14:51:08.882 INFO:tasks.workunit.client.0.smithi012.stdout:4/841: getdents d6/da/d58/d113/d120 0 2024-03-18T14:51:08.883 INFO:tasks.workunit.client.0.smithi012.stdout:4/842: creat d6/da/f123 x:0 0 0 2024-03-18T14:51:08.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/634: dread d4/f43 [0,4194304] 0 2024-03-18T14:51:08.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/635: chown d4/dc/dd/de/d21/ca7 69 1 2024-03-18T14:51:08.890 INFO:tasks.workunit.client.0.smithi012.stdout:3/871: creat dc/d30/d3a/da0/dc5/f115 x:0 0 0 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:6/545: sync 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:0/647: sync 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:5/556: sync 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:2/585: sync 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:3/620: sync 2024-03-18T14:51:08.892 INFO:tasks.workunit.client.1.smithi100.stdout:2/586: truncate d3/f10 1306346 0 2024-03-18T14:51:08.893 INFO:tasks.workunit.client.1.smithi100.stdout:3/621: truncate d3/d16/d13/d1f/f90 112811 0 2024-03-18T14:51:08.893 INFO:tasks.workunit.client.1.smithi100.stdout:1/558: sync 2024-03-18T14:51:08.896 INFO:tasks.workunit.client.0.smithi012.stdout:4/843: unlink d6/d1a/d30/d46/lbf 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:8/636: dread d4/f15 [0,4194304] 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:8/637: fdatasync d4/ff 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:8/638: truncate d4/dc/dd/de/d21/d23/d47/d54/fb9 571861 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:5/557: getdents d29/d2d/d32/d38/d9e 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:5/558: stat d29/d2d/d39/f4c 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:5/559: chown d29/d2d/d8f/caf 0 1 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:6/546: dread d6/d7/df/f7c [0,4194304] 0 2024-03-18T14:51:08.899 INFO:tasks.workunit.client.1.smithi100.stdout:6/547: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa6 x:0 0 0 2024-03-18T14:51:08.901 INFO:tasks.workunit.client.1.smithi100.stdout:2/587: link d3/da/de/d20/d82/d95/la0 d3/d1a/d76/lce 0 2024-03-18T14:51:08.901 INFO:tasks.workunit.client.1.smithi100.stdout:2/588: readlink d3/da/de/d48/lb6 0 2024-03-18T14:51:08.903 INFO:tasks.workunit.client.0.smithi012.stdout:4/844: rename d6/d1a/d30/d46/d6f/d8b/fee to d6/db6/f124 0 2024-03-18T14:51:08.905 INFO:tasks.workunit.client.1.smithi100.stdout:7/594: dwrite d5/d14/d16/d44/fa7 [0,4194304] 0 2024-03-18T14:51:08.905 INFO:tasks.workunit.client.1.smithi100.stdout:7/595: chown d5/d14/f1e 615173 1 2024-03-18T14:51:08.905 INFO:tasks.workunit.client.1.smithi100.stdout:9/584: dwrite da/d51/d55/d7e/f9d [0,4194304] 0 2024-03-18T14:51:08.905 INFO:tasks.workunit.client.1.smithi100.stdout:9/585: read - da/dd/d29/d71/f98 zero size 2024-03-18T14:51:08.906 INFO:tasks.workunit.client.1.smithi100.stdout:9/586: write da/dd/d29/f2e [822421,107768] 0 2024-03-18T14:51:08.907 INFO:tasks.workunit.client.0.smithi012.stdout:3/872: rmdir dc/d30/d3a/db7/dd3 39 2024-03-18T14:51:08.909 INFO:tasks.workunit.client.1.smithi100.stdout:8/639: mknod d4/dc/dd/d48/dd8/ce1 0 2024-03-18T14:51:08.911 INFO:tasks.workunit.client.0.smithi012.stdout:4/845: rename d6/d19/lf6 to d6/da/d89/d94/dbe/l125 0 2024-03-18T14:51:08.912 INFO:tasks.workunit.client.1.smithi100.stdout:5/560: getdents d29/d2d/d8f 0 2024-03-18T14:51:08.912 INFO:tasks.workunit.client.1.smithi100.stdout:5/561: fsync d29/d2d/d32/d38/f5f 0 2024-03-18T14:51:08.914 INFO:tasks.workunit.client.0.smithi012.stdout:4/846: mkdir d6/d1a/d30/d46/d6f/d8b/d11f/d126 0 2024-03-18T14:51:08.915 INFO:tasks.workunit.client.1.smithi100.stdout:6/548: write d6/d7/df/f2a [1703329,115403] 0 2024-03-18T14:51:08.916 INFO:tasks.workunit.client.0.smithi012.stdout:3/873: mkdir dc/d116 0 2024-03-18T14:51:08.919 INFO:tasks.workunit.client.0.smithi012.stdout:4/847: mkdir d6/d19/da5/d127 0 2024-03-18T14:51:08.919 INFO:tasks.workunit.client.1.smithi100.stdout:1/559: dwrite d8/d1e/d11/d27/d73/d89/fa4 [0,4194304] 0 2024-03-18T14:51:08.922 INFO:tasks.workunit.client.1.smithi100.stdout:7/596: unlink d5/d14/d16/d2e/d7b/f95 0 2024-03-18T14:51:08.922 INFO:tasks.workunit.client.1.smithi100.stdout:7/597: readlink d5/d14/d16/d2e/d62/d66/ld4 0 2024-03-18T14:51:08.922 INFO:tasks.workunit.client.1.smithi100.stdout:3/622: dwrite d3/dd/d49/f87 [0,4194304] 0 2024-03-18T14:51:08.922 INFO:tasks.workunit.client.1.smithi100.stdout:3/623: dread - d3/dd/d21/fd7 zero size 2024-03-18T14:51:08.925 INFO:tasks.workunit.client.0.smithi012.stdout:3/874: rename dc/d3c/d4c/d57/d103/d9a/d64/fd4 to dc/d30/d3a/f117 0 2024-03-18T14:51:08.926 INFO:tasks.workunit.client.1.smithi100.stdout:2/589: dwrite d3/da/f43 [4194304,4194304] 0 2024-03-18T14:51:08.930 INFO:tasks.workunit.client.1.smithi100.stdout:2/590: write d3/da/f25 [5052384,110521] 0 2024-03-18T14:51:08.930 INFO:tasks.workunit.client.0.smithi012.stdout:4/848: dread d6/d27/f56 [0,4194304] 0 2024-03-18T14:51:08.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/591: dread d3/d13/d40/fa7 [0,4194304] 0 2024-03-18T14:51:08.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/592: chown d3/d6/d29/f35 808881 1 2024-03-18T14:51:08.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/593: dread - d3/da/de/d20/d82/d95/faa zero size 2024-03-18T14:51:08.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/594: write d3/da/de/d48/f8e [940735,84695] 0 2024-03-18T14:51:08.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/595: creat d3/d6/d5f/d6a/fcf x:0 0 0 2024-03-18T14:51:08.934 INFO:tasks.workunit.client.0.smithi012.stdout:3/875: mknod dc/d30/d3a/da0/dc5/dc7/dd8/c118 0 2024-03-18T14:51:08.936 INFO:tasks.workunit.client.1.smithi100.stdout:9/587: link da/l57 da/d2c/lba 0 2024-03-18T14:51:08.937 INFO:tasks.workunit.client.0.smithi012.stdout:4/849: mknod d6/d19/da5/d127/c128 0 2024-03-18T14:51:08.938 INFO:tasks.workunit.client.0.smithi012.stdout:3/876: symlink dc/d30/d3a/da0/de4/l119 0 2024-03-18T14:51:08.939 INFO:tasks.workunit.client.1.smithi100.stdout:8/640: symlink d4/dc/d97/le2 0 2024-03-18T14:51:08.939 INFO:tasks.workunit.client.1.smithi100.stdout:8/641: stat d4/dc/dd/de/d21/d23/d25/c70 0 2024-03-18T14:51:08.940 INFO:tasks.workunit.client.0.smithi012.stdout:4/850: creat d6/d19/d6c/d96/dda/f129 x:0 0 0 2024-03-18T14:51:08.941 INFO:tasks.workunit.client.0.smithi012.stdout:4/851: dread - d6/da/d89/d97/dc5/fff zero size 2024-03-18T14:51:08.941 INFO:tasks.workunit.client.0.smithi012.stdout:4/852: stat d6/d1a/d30/l75 0 2024-03-18T14:51:08.941 INFO:tasks.workunit.client.0.smithi012.stdout:4/853: chown d6/da/d58 4 1 2024-03-18T14:51:08.941 INFO:tasks.workunit.client.0.smithi012.stdout:4/854: chown d6/d19/d6c/f7a 454806012 1 2024-03-18T14:51:08.941 INFO:tasks.workunit.client.0.smithi012.stdout:4/855: fdatasync d6/d19/d6c/f7a 0 2024-03-18T14:51:08.942 INFO:tasks.workunit.client.0.smithi012.stdout:3/877: truncate dc/d91/fea 1198453 0 2024-03-18T14:51:08.942 INFO:tasks.workunit.client.1.smithi100.stdout:6/549: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/f13 [0,4194304] 0 2024-03-18T14:51:08.942 INFO:tasks.workunit.client.1.smithi100.stdout:3/624: dwrite d3/fa1 [0,4194304] 0 2024-03-18T14:51:08.942 INFO:tasks.workunit.client.1.smithi100.stdout:3/625: chown d3/da5 44 1 2024-03-18T14:51:08.943 INFO:tasks.workunit.client.1.smithi100.stdout:5/562: mknod d29/d2d/d39/d61/cb0 0 2024-03-18T14:51:08.946 INFO:tasks.workunit.client.0.smithi012.stdout:3/878: mkdir dc/d3c/d72/df3/d109/d11a 0 2024-03-18T14:51:08.950 INFO:tasks.workunit.client.0.smithi012.stdout:3/879: rename dc/d3c/d4c/d5f/lcf to dc/d3c/d4c/d57/d103/d9a/dd7/l11b 0 2024-03-18T14:51:08.950 INFO:tasks.workunit.client.0.smithi012.stdout:3/880: readlink dc/d30/d3a/db7/dd3/l105 0 2024-03-18T14:51:08.950 INFO:tasks.workunit.client.0.smithi012.stdout:3/881: dread - dc/d30/dab/f111 zero size 2024-03-18T14:51:08.951 INFO:tasks.workunit.client.1.smithi100.stdout:1/560: truncate d8/d2a/d37/f5d 5030137 0 2024-03-18T14:51:08.951 INFO:tasks.workunit.client.1.smithi100.stdout:1/561: dread - d8/d1e/d11/d6d/d59/d93/f9e zero size 2024-03-18T14:51:08.951 INFO:tasks.workunit.client.1.smithi100.stdout:1/562: read d8/d1e/d11/d27/d3e/f4b [1411,129023] 0 2024-03-18T14:51:08.951 INFO:tasks.workunit.client.1.smithi100.stdout:1/563: chown d8/d1e/d11/d2d/l79 4 1 2024-03-18T14:51:08.952 INFO:tasks.workunit.client.1.smithi100.stdout:1/564: chown d8/d1e/d11/d6d 5239 1 2024-03-18T14:51:08.952 INFO:tasks.workunit.client.1.smithi100.stdout:1/565: readlink d8/d1e/l8b 0 2024-03-18T14:51:08.952 INFO:tasks.workunit.client.1.smithi100.stdout:7/598: creat d5/d22/d3b/d9f/da0/db1/dd3/fe0 x:0 0 0 2024-03-18T14:51:08.954 INFO:tasks.workunit.client.0.smithi012.stdout:3/882: mknod dc/d3c/d4c/d57/d103/d9a/dd7/de3/c11c 0 2024-03-18T14:51:08.955 INFO:tasks.workunit.client.0.smithi012.stdout:4/856: dwrite d6/d19/f1d [0,4194304] 0 2024-03-18T14:51:08.962 INFO:tasks.workunit.client.0.smithi012.stdout:3/883: dread dc/f14 [0,4194304] 0 2024-03-18T14:51:08.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/857: symlink d6/d19/d6c/d96/dda/l12a 0 2024-03-18T14:51:08.963 INFO:tasks.workunit.client.0.smithi012.stdout:4/858: chown d6/da/d89/da1/dbb/ddc/lf2 4425554 1 2024-03-18T14:51:08.971 INFO:tasks.workunit.client.1.smithi100.stdout:2/596: symlink d3/d13/d40/d53/ld0 0 2024-03-18T14:51:08.971 INFO:tasks.workunit.client.0.smithi012.stdout:3/884: dread dc/f22 [0,4194304] 0 2024-03-18T14:51:08.972 INFO:tasks.workunit.client.0.smithi012.stdout:3/885: write dc/d3c/d4c/d57/fa1 [3000390,90647] 0 2024-03-18T14:51:08.972 INFO:tasks.workunit.client.0.smithi012.stdout:4/859: write d6/d19/d6c/f88 [2121155,2918] 0 2024-03-18T14:51:08.972 INFO:tasks.workunit.client.0.smithi012.stdout:4/860: write d6/da/d89/d97/f9a [215786,395] 0 2024-03-18T14:51:08.974 INFO:tasks.workunit.client.1.smithi100.stdout:8/642: link d4/dc/dd/c83 d4/dc/d1f/d59/d6c/d75/ce3 0 2024-03-18T14:51:08.980 INFO:tasks.workunit.client.0.smithi012.stdout:4/861: link d6/d1a/d30/l75 d6/d19/d39/l12b 0 2024-03-18T14:51:08.984 INFO:tasks.workunit.client.0.smithi012.stdout:4/862: link d6/d19/d6c/le3 d6/da/d89/d94/d10d/l12c 0 2024-03-18T14:51:08.985 INFO:tasks.workunit.client.1.smithi100.stdout:5/563: read d29/d2d/f6f [333255,104781] 0 2024-03-18T14:51:08.985 INFO:tasks.workunit.client.1.smithi100.stdout:5/564: dread - d29/d2d/d32/d38/f6e zero size 2024-03-18T14:51:08.987 INFO:tasks.workunit.client.0.smithi012.stdout:4/863: mkdir d6/d1a/d64/d12d 0 2024-03-18T14:51:08.989 INFO:tasks.workunit.client.1.smithi100.stdout:1/566: symlink d8/d2a/d37/la6 0 2024-03-18T14:51:08.992 INFO:tasks.workunit.client.1.smithi100.stdout:7/599: rename d5/f35 to d5/d22/d3b/d9f/da0/fe1 0 2024-03-18T14:51:08.993 INFO:tasks.workunit.client.1.smithi100.stdout:5/565: dread d29/f8c [0,4194304] 0 2024-03-18T14:51:08.993 INFO:tasks.workunit.client.1.smithi100.stdout:5/566: creat d29/d2d/d32/d38/fb1 x:0 0 0 2024-03-18T14:51:08.995 INFO:tasks.workunit.client.1.smithi100.stdout:2/597: creat d3/dc3/fd1 x:0 0 0 2024-03-18T14:51:08.995 INFO:tasks.workunit.client.1.smithi100.stdout:2/598: chown d3/d1a/l41 29114 1 2024-03-18T14:51:08.997 INFO:tasks.workunit.client.1.smithi100.stdout:9/588: rmdir da/dd/d3c/d6b 39 2024-03-18T14:51:08.998 INFO:tasks.workunit.client.1.smithi100.stdout:8/643: creat d4/dc/dd/de/d19/db0/fe4 x:0 0 0 2024-03-18T14:51:08.999 INFO:tasks.workunit.client.0.smithi012.stdout:4/864: dread d6/d19/f10c [0,4194304] 0 2024-03-18T14:51:09.003 INFO:tasks.workunit.client.1.smithi100.stdout:6/550: rmdir d6/d12/d2c 39 2024-03-18T14:51:09.003 INFO:tasks.workunit.client.1.smithi100.stdout:6/551: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f79 [3865977,41687] 0 2024-03-18T14:51:09.003 INFO:tasks.workunit.client.1.smithi100.stdout:6/552: write d6/d12/f1f [2571320,63] 0 2024-03-18T14:51:09.003 INFO:tasks.workunit.client.1.smithi100.stdout:6/553: chown d6/d7/df/d26/d4e/ca4 798227494 1 2024-03-18T14:51:09.004 INFO:tasks.workunit.client.1.smithi100.stdout:7/600: link d5/d14/c27 d5/d14/d16/d63/ce2 0 2024-03-18T14:51:09.004 INFO:tasks.workunit.client.1.smithi100.stdout:9/589: dread da/fa4 [0,4194304] 0 2024-03-18T14:51:09.005 INFO:tasks.workunit.client.1.smithi100.stdout:9/590: read da/d46/d4d/d67/f75 [108653,8305] 0 2024-03-18T14:51:09.007 INFO:tasks.workunit.client.1.smithi100.stdout:8/644: symlink d4/dc/le5 0 2024-03-18T14:51:09.008 INFO:tasks.workunit.client.1.smithi100.stdout:1/567: dwrite d8/d1e/d11/d8a/f97 [0,4194304] 0 2024-03-18T14:51:09.008 INFO:tasks.workunit.client.1.smithi100.stdout:6/554: link d6/d7/df/d72/d65/d7a/c86 d6/d7/df/d26/d4e/d66/d36/ca7 0 2024-03-18T14:51:09.009 INFO:tasks.workunit.client.1.smithi100.stdout:9/591: mknod da/dd/d29/d71/d72/db2/cbb 0 2024-03-18T14:51:09.009 INFO:tasks.workunit.client.1.smithi100.stdout:9/592: truncate da/d2c/d2f/f7d 653216 0 2024-03-18T14:51:09.050 INFO:tasks.workunit.client.1.smithi100.stdout:5/567: dwrite d29/d2d/d32/f68 [0,4194304] 0 2024-03-18T14:51:09.051 INFO:tasks.workunit.client.1.smithi100.stdout:8/645: symlink d4/dc/dd/de/d21/d23/d47/le6 0 2024-03-18T14:51:09.051 INFO:tasks.workunit.client.1.smithi100.stdout:5/568: write d29/d2d/d39/d61/f6b [123508,21794] 0 2024-03-18T14:51:09.053 INFO:tasks.workunit.client.1.smithi100.stdout:1/568: creat d8/d1e/d11/d27/fa7 x:0 0 0 2024-03-18T14:51:09.056 INFO:tasks.workunit.client.1.smithi100.stdout:9/593: rename da/d46/d4d/d67/f68 to da/dd/d3b/fbc 0 2024-03-18T14:51:09.060 INFO:tasks.workunit.client.1.smithi100.stdout:8/646: readlink d4/dc/dd/de/d19/l22 0 2024-03-18T14:51:09.061 INFO:tasks.workunit.client.1.smithi100.stdout:5/569: mkdir d29/d2d/d32/db2 0 2024-03-18T14:51:09.062 INFO:tasks.workunit.client.1.smithi100.stdout:1/569: creat d8/d1e/d11/d8a/fa8 x:0 0 0 2024-03-18T14:51:09.067 INFO:tasks.workunit.client.1.smithi100.stdout:1/570: truncate f7 865478 0 2024-03-18T14:51:09.067 INFO:tasks.workunit.client.1.smithi100.stdout:1/571: truncate d8/d30/d61/f63 901007 0 2024-03-18T14:51:09.068 INFO:tasks.workunit.client.1.smithi100.stdout:1/572: chown d8/d2a/d37/d50/f90 0 1 2024-03-18T14:51:09.068 INFO:tasks.workunit.client.0.smithi012.stdout:4/865: dwrite d6/da/d89/da1/fe8 [0,4194304] 0 2024-03-18T14:51:09.069 INFO:tasks.workunit.client.1.smithi100.stdout:1/573: rmdir d8/d2a/d37/d50/d6f/d81/d91 39 2024-03-18T14:51:09.069 INFO:tasks.workunit.client.1.smithi100.stdout:1/574: fsync d8/f69 0 2024-03-18T14:51:09.074 INFO:tasks.workunit.client.1.smithi100.stdout:6/555: dwrite d6/d12/d2c/f7e [0,4194304] 0 2024-03-18T14:51:09.076 INFO:tasks.workunit.client.1.smithi100.stdout:1/575: getdents d8/d1e/d11/d27/d73 0 2024-03-18T14:51:09.076 INFO:tasks.workunit.client.1.smithi100.stdout:1/576: stat d8/d1e/d11/d27/d3e/l6a 0 2024-03-18T14:51:09.076 INFO:tasks.workunit.client.1.smithi100.stdout:7/601: dwrite d5/d22/d3b/d9f/da0/fe1 [0,4194304] 0 2024-03-18T14:51:09.076 INFO:tasks.workunit.client.1.smithi100.stdout:8/647: write d4/dc/d31/d66/f9b [4202299,42194] 0 2024-03-18T14:51:09.078 INFO:tasks.workunit.client.0.smithi012.stdout:4/866: write d6/da/d89/fce [3248697,74424] 0 2024-03-18T14:51:09.083 INFO:tasks.workunit.client.1.smithi100.stdout:5/570: dwrite d29/d2d/d32/d92/daa/fa1 [0,4194304] 0 2024-03-18T14:51:09.083 INFO:tasks.workunit.client.1.smithi100.stdout:5/571: creat d29/d2d/d39/d71/fb3 x:0 0 0 2024-03-18T14:51:09.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/867: dread d6/d19/fad [0,4194304] 0 2024-03-18T14:51:09.090 INFO:tasks.workunit.client.1.smithi100.stdout:1/577: dwrite d8/f7e [0,4194304] 0 2024-03-18T14:51:09.091 INFO:tasks.workunit.client.1.smithi100.stdout:6/556: symlink d6/d7/df/d26/d4e/d53/la8 0 2024-03-18T14:51:09.092 INFO:tasks.workunit.client.0.smithi012.stdout:4/868: mkdir d6/da/d89/d97/d12e 0 2024-03-18T14:51:09.095 INFO:tasks.workunit.client.1.smithi100.stdout:7/602: creat d5/d22/d24/d6e/d7a/fe3 x:0 0 0 2024-03-18T14:51:09.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/648: symlink d4/dc/dd/de/d21/d23/d47/le7 0 2024-03-18T14:51:09.108 INFO:tasks.workunit.client.1.smithi100.stdout:5/572: symlink d29/d2d/d39/d65/lb4 0 2024-03-18T14:51:09.109 INFO:tasks.workunit.client.1.smithi100.stdout:5/573: write d29/d2d/d39/f84 [1087124,47867] 0 2024-03-18T14:51:09.111 INFO:tasks.workunit.client.1.smithi100.stdout:1/578: dwrite d8/d2a/f41 [0,4194304] 0 2024-03-18T14:51:09.115 INFO:tasks.workunit.client.1.smithi100.stdout:6/557: rmdir d6/d12/d78 39 2024-03-18T14:51:09.115 INFO:tasks.workunit.client.1.smithi100.stdout:7/603: rename d5/d22/d24/d6e/caa to d5/d22/d3b/d9f/da0/ce4 0 2024-03-18T14:51:09.115 INFO:tasks.workunit.client.1.smithi100.stdout:7/604: stat d5/d22/d3b/c6d 0 2024-03-18T14:51:09.118 INFO:tasks.workunit.client.1.smithi100.stdout:1/579: creat d8/d2a/fa9 x:0 0 0 2024-03-18T14:51:09.120 INFO:tasks.workunit.client.1.smithi100.stdout:6/558: truncate d6/d7/f8 1168647 0 2024-03-18T14:51:09.120 INFO:tasks.workunit.client.1.smithi100.stdout:6/559: dread - d6/f91 zero size 2024-03-18T14:51:09.120 INFO:tasks.workunit.client.1.smithi100.stdout:6/560: chown d6/d7/df/d26/d4e/d66/d36/d4f/c90 199 1 2024-03-18T14:51:09.120 INFO:tasks.workunit.client.1.smithi100.stdout:6/561: chown d6/d7/df/d26/d4e/d66/d36/d4f/d71 32 1 2024-03-18T14:51:09.122 INFO:tasks.workunit.client.1.smithi100.stdout:6/562: rmdir d6/d7/df/d26 39 2024-03-18T14:51:09.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/605: rmdir d5/d22/d3b/d9f/da0/db1 39 2024-03-18T14:51:09.125 INFO:tasks.workunit.client.0.smithi012.stdout:5/865: sync 2024-03-18T14:51:09.125 INFO:tasks.workunit.client.0.smithi012.stdout:5/866: readlink d3/d15/d42/dc6/d86/l118 0 2024-03-18T14:51:09.126 INFO:tasks.workunit.client.0.smithi012.stdout:5/867: readlink d3/d15/d42/l10e 0 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.0.smithi012.stdout:1/818: sync 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.0.smithi012.stdout:0/905: sync 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.0.smithi012.stdout:7/915: sync 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.0.smithi012.stdout:0/906: fdatasync d1/d16/d19/f47 0 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.0.smithi012.stdout:0/907: readlink d1/lb7 0 2024-03-18T14:51:09.127 INFO:tasks.workunit.client.1.smithi100.stdout:6/563: creat d6/d7/df/d72/d65/fa9 x:0 0 0 2024-03-18T14:51:09.129 INFO:tasks.workunit.client.1.smithi100.stdout:7/606: mkdir d5/d14/d16/d2e/d62/dd1/d86/d85/d87/de5 0 2024-03-18T14:51:09.132 INFO:tasks.workunit.client.0.smithi012.stdout:1/819: truncate d8/db/f41 1089039 0 2024-03-18T14:51:09.132 INFO:tasks.workunit.client.1.smithi100.stdout:7/607: read d5/d14/f1e [1270539,27425] 0 2024-03-18T14:51:09.132 INFO:tasks.workunit.client.1.smithi100.stdout:7/608: chown d5/d14/d16/c19 347202504 1 2024-03-18T14:51:09.133 INFO:tasks.workunit.client.0.smithi012.stdout:0/908: dread d1/f9c [4194304,4194304] 0 2024-03-18T14:51:09.133 INFO:tasks.workunit.client.1.smithi100.stdout:5/574: dread d29/d2d/f41 [0,4194304] 0 2024-03-18T14:51:09.134 INFO:tasks.workunit.client.1.smithi100.stdout:6/564: truncate d6/d7/df/d26/f45 4055665 0 2024-03-18T14:51:09.134 INFO:tasks.workunit.client.1.smithi100.stdout:0/648: sync 2024-03-18T14:51:09.134 INFO:tasks.workunit.client.1.smithi100.stdout:4/575: sync 2024-03-18T14:51:09.134 INFO:tasks.workunit.client.1.smithi100.stdout:3/626: sync 2024-03-18T14:51:09.135 INFO:tasks.workunit.client.1.smithi100.stdout:1/580: dread d8/d30/f40 [0,4194304] 0 2024-03-18T14:51:09.135 INFO:tasks.workunit.client.1.smithi100.stdout:9/594: dwrite da/d46/d8b/f3a [0,4194304] 0 2024-03-18T14:51:09.135 INFO:tasks.workunit.client.1.smithi100.stdout:9/595: truncate da/f4f 2136866 0 2024-03-18T14:51:09.135 INFO:tasks.workunit.client.1.smithi100.stdout:9/596: fsync da/db/f13 0 2024-03-18T14:51:09.136 INFO:tasks.workunit.client.1.smithi100.stdout:0/649: read dd/d2b/d95/d79/d7d/f7e [128400,130032] 0 2024-03-18T14:51:09.136 INFO:tasks.workunit.client.1.smithi100.stdout:0/650: dread - dd/d31/d91/fe1 zero size 2024-03-18T14:51:09.137 INFO:tasks.workunit.client.0.smithi012.stdout:4/869: dwrite d6/d1a/d92/fbc [0,4194304] 0 2024-03-18T14:51:09.137 INFO:tasks.workunit.client.1.smithi100.stdout:8/649: dwrite d4/dc/dd/de/d21/d23/f27 [0,4194304] 0 2024-03-18T14:51:09.138 INFO:tasks.workunit.client.0.smithi012.stdout:7/916: dread dd/d13/d23/fce [0,4194304] 0 2024-03-18T14:51:09.139 INFO:tasks.workunit.client.1.smithi100.stdout:4/576: dread d9/d27/f36 [0,4194304] 0 2024-03-18T14:51:09.139 INFO:tasks.workunit.client.1.smithi100.stdout:4/577: truncate d9/d27/d42/f4a 1493859 0 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/578: chown d9/l31 993438 1 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/579: fsync d9/d3a/f3c 0 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/580: chown d9/d27/d62/f73 9540 1 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/581: readlink d9/d3a/l55 0 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:7/609: mkdir d5/d14/d16/d2e/d62/dd1/d86/de6 0 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:2/599: sync 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:5/575: rename d29/d2d/d39/d60/f6d to d29/d2d/d39/d60/fb5 0 2024-03-18T14:51:09.140 INFO:tasks.workunit.client.1.smithi100.stdout:7/610: dread - d5/d14/d16/d2e/d62/d66/fbf zero size 2024-03-18T14:51:09.143 INFO:tasks.workunit.client.1.smithi100.stdout:6/565: mknod d6/d7/df/d26/d4e/d66/caa 0 2024-03-18T14:51:09.143 INFO:tasks.workunit.client.0.smithi012.stdout:1/820: write d8/db/d10/d31/fba [8182872,98993] 0 2024-03-18T14:51:09.148 INFO:tasks.workunit.client.1.smithi100.stdout:9/597: write da/dd/d3c/d6b/f84 [5035140,117425] 0 2024-03-18T14:51:09.151 INFO:tasks.workunit.client.1.smithi100.stdout:1/581: dread d8/d1e/f45 [0,4194304] 0 2024-03-18T14:51:09.152 INFO:tasks.workunit.client.1.smithi100.stdout:5/576: dread d29/d2d/d32/f35 [0,4194304] 0 2024-03-18T14:51:09.155 INFO:tasks.workunit.client.0.smithi012.stdout:4/870: unlink d6/da/d89/d95/cb8 0 2024-03-18T14:51:09.156 INFO:tasks.workunit.client.1.smithi100.stdout:5/577: write d29/d4a/f54 [1462806,102666] 0 2024-03-18T14:51:09.156 INFO:tasks.workunit.client.0.smithi012.stdout:5/868: dwrite d3/d29/d35/fd4 [0,4194304] 0 2024-03-18T14:51:09.157 INFO:tasks.workunit.client.1.smithi100.stdout:0/651: creat dd/d20/d44/feb x:0 0 0 2024-03-18T14:51:09.157 INFO:tasks.workunit.client.1.smithi100.stdout:0/652: chown dd/d31/l47 185675 1 2024-03-18T14:51:09.159 INFO:tasks.workunit.client.1.smithi100.stdout:8/650: creat d4/d3c/fe8 x:0 0 0 2024-03-18T14:51:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:0/909: write d1/d16/d31/f1d [1811209,67207] 0 2024-03-18T14:51:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:0/910: write d1/d20/d23/d3b/d33/d51/fa8 [2909606,99302] 0 2024-03-18T14:51:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:0/911: chown d1/d20/d23/d3b 663 1 2024-03-18T14:51:09.160 INFO:tasks.workunit.client.0.smithi012.stdout:0/912: chown d1/d16/d6e/dcc/f10f 33791 1 2024-03-18T14:51:09.161 INFO:tasks.workunit.client.0.smithi012.stdout:1/821: mkdir d8/d1e/d106 0 2024-03-18T14:51:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:7/917: dwrite dd/d13/d23/d76/d6f/db3/ff3 [0,4194304] 0 2024-03-18T14:51:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:7/918: dread - dd/d13/d23/d76/d6f/dd9/ddd/f12a zero size 2024-03-18T14:51:09.162 INFO:tasks.workunit.client.0.smithi012.stdout:7/919: chown dd/d13/d23/d76/d109/d129/dd5/f12d 38305832 1 2024-03-18T14:51:09.163 INFO:tasks.workunit.client.1.smithi100.stdout:4/582: creat d9/d27/da2/faf x:0 0 0 2024-03-18T14:51:09.166 INFO:tasks.workunit.client.1.smithi100.stdout:2/600: mknod d3/d1a/cd2 0 2024-03-18T14:51:09.166 INFO:tasks.workunit.client.1.smithi100.stdout:2/601: write d3/d6/d29/d36/f9e [5070193,106917] 0 2024-03-18T14:51:09.166 INFO:tasks.workunit.client.1.smithi100.stdout:2/602: chown d3/d13/d40/d7d/fb7 129365 1 2024-03-18T14:51:09.167 INFO:tasks.workunit.client.1.smithi100.stdout:0/653: dread dd/d20/f2c [0,4194304] 0 2024-03-18T14:51:09.167 INFO:tasks.workunit.client.1.smithi100.stdout:0/654: chown dd/d31/d91/d56/d62/fd8 3983089 1 2024-03-18T14:51:09.168 INFO:tasks.workunit.client.1.smithi100.stdout:7/611: mknod d5/d14/d16/d2e/d7b/ce7 0 2024-03-18T14:51:09.168 INFO:tasks.workunit.client.1.smithi100.stdout:7/612: chown d5/d14/d16/d2e/d62 3255107 1 2024-03-18T14:51:09.177 INFO:tasks.workunit.client.0.smithi012.stdout:5/869: link d3/d29/d9d/dba/fd7 d3/d29/d35/f121 0 2024-03-18T14:51:09.179 INFO:tasks.workunit.client.1.smithi100.stdout:9/598: link da/dd/f42 da/dd/d29/fbd 0 2024-03-18T14:51:09.184 INFO:tasks.workunit.client.0.smithi012.stdout:7/920: link dd/d13/d23/d76/d109/d129/d48/ce5 dd/d13/d23/d76/d6f/db3/dc7/d136/c13b 0 2024-03-18T14:51:09.185 INFO:tasks.workunit.client.1.smithi100.stdout:5/578: mknod d29/d2d/d39/d83/cb6 0 2024-03-18T14:51:09.187 INFO:tasks.workunit.client.1.smithi100.stdout:8/651: creat d4/dc/d31/d85/d8e/fe9 x:0 0 0 2024-03-18T14:51:09.192 INFO:tasks.workunit.client.1.smithi100.stdout:4/583: mknod d9/d13/d5a/d5b/da0/cb0 0 2024-03-18T14:51:09.192 INFO:tasks.workunit.client.1.smithi100.stdout:4/584: fsync d9/d17/f34 0 2024-03-18T14:51:09.192 INFO:tasks.workunit.client.0.smithi012.stdout:5/870: mkdir d3/d29/d35/d122 0 2024-03-18T14:51:09.198 INFO:tasks.workunit.client.0.smithi012.stdout:0/913: dwrite d1/d16/d19/dd0/d129/d67/f9a [0,4194304] 0 2024-03-18T14:51:09.198 INFO:tasks.workunit.client.0.smithi012.stdout:0/914: chown d1/d20/d23/d88/f125 14 1 2024-03-18T14:51:09.199 INFO:tasks.workunit.client.1.smithi100.stdout:2/603: mkdir d3/da/de/d20/d82/dd3 0 2024-03-18T14:51:09.199 INFO:tasks.workunit.client.1.smithi100.stdout:2/604: truncate d3/fba 34470 0 2024-03-18T14:51:09.200 INFO:tasks.workunit.client.0.smithi012.stdout:4/871: rename d6/d19/d6c/d96/df0 to d6/da/d89/d94/dbe/dd6/d12f 0 2024-03-18T14:51:09.200 INFO:tasks.workunit.client.0.smithi012.stdout:4/872: creat d6/da/d89/da1/f130 x:0 0 0 2024-03-18T14:51:09.200 INFO:tasks.workunit.client.1.smithi100.stdout:7/613: mknod d5/d14/ce8 0 2024-03-18T14:51:09.201 INFO:tasks.workunit.client.0.smithi012.stdout:5/871: link d3/d29/d36/cbf d3/d15/d5f/c123 0 2024-03-18T14:51:09.202 INFO:tasks.workunit.client.1.smithi100.stdout:6/566: truncate d6/d7/f8 2050103 0 2024-03-18T14:51:09.206 INFO:tasks.workunit.client.0.smithi012.stdout:0/915: symlink d1/d20/d23/d3b/d33/d64/l12c 0 2024-03-18T14:51:09.208 INFO:tasks.workunit.client.1.smithi100.stdout:9/599: rename da/d51/d55/c92 to da/dd/da0/cbe 0 2024-03-18T14:51:09.208 INFO:tasks.workunit.client.1.smithi100.stdout:9/600: write da/dd/d29/d9e/fa7 [187111,111637] 0 2024-03-18T14:51:09.208 INFO:tasks.workunit.client.1.smithi100.stdout:9/601: write da/d46/fa3 [1932635,60482] 0 2024-03-18T14:51:09.209 INFO:tasks.workunit.client.1.smithi100.stdout:1/582: rmdir d8/d1e/d11/d27/d83 39 2024-03-18T14:51:09.209 INFO:tasks.workunit.client.1.smithi100.stdout:1/583: chown d8/d2a/d37/d50/d6f/d71 0 1 2024-03-18T14:51:09.209 INFO:tasks.workunit.client.1.smithi100.stdout:9/602: write da/d51/f69 [206732,109991] 0 2024-03-18T14:51:09.209 INFO:tasks.workunit.client.1.smithi100.stdout:9/603: write da/dd/d3c/f59 [2896857,60677] 0 2024-03-18T14:51:09.209 INFO:tasks.workunit.client.1.smithi100.stdout:9/604: write da/d51/d55/f5c [504060,53124] 0 2024-03-18T14:51:09.210 INFO:tasks.workunit.client.0.smithi012.stdout:4/873: link d6/d1a/f2a d6/d1a/d30/d46/d6f/d7c/f131 0 2024-03-18T14:51:09.211 INFO:tasks.workunit.client.1.smithi100.stdout:5/579: mknod d29/d2d/d39/d71/cb7 0 2024-03-18T14:51:09.211 INFO:tasks.workunit.client.1.smithi100.stdout:5/580: fsync d29/d2d/d32/d92/daa/fa1 0 2024-03-18T14:51:09.212 INFO:tasks.workunit.client.0.smithi012.stdout:0/916: creat d1/d16/d19/dd0/d129/d67/f12d x:0 0 0 2024-03-18T14:51:09.212 INFO:tasks.workunit.client.0.smithi012.stdout:0/917: chown d1/d2d/f117 105746332 1 2024-03-18T14:51:09.212 INFO:tasks.workunit.client.0.smithi012.stdout:0/918: chown d1/d20/d23/d7f/f97 0 1 2024-03-18T14:51:09.213 INFO:tasks.workunit.client.1.smithi100.stdout:8/652: mkdir d4/dc/d1f/d59/d6c/d75/dea 0 2024-03-18T14:51:09.214 INFO:tasks.workunit.client.1.smithi100.stdout:6/567: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [0,4194304] 0 2024-03-18T14:51:09.215 INFO:tasks.workunit.client.1.smithi100.stdout:1/584: read d8/d1e/f28 [298695,74629] 0 2024-03-18T14:51:09.215 INFO:tasks.workunit.client.0.smithi012.stdout:4/874: unlink d6/da/d89/d94/dbe/dd6/d12f/d11c/c121 0 2024-03-18T14:51:09.216 INFO:tasks.workunit.client.0.smithi012.stdout:4/875: write d6/da/d89/da1/dbb/fc3 [5014176,89203] 0 2024-03-18T14:51:09.216 INFO:tasks.workunit.client.1.smithi100.stdout:2/605: mkdir d3/dd4 0 2024-03-18T14:51:09.217 INFO:tasks.workunit.client.0.smithi012.stdout:0/919: mknod d1/d20/c12e 0 2024-03-18T14:51:09.220 INFO:tasks.workunit.client.1.smithi100.stdout:7/614: link d5/lc d5/d30/d4d/db4/le9 0 2024-03-18T14:51:09.221 INFO:tasks.workunit.client.1.smithi100.stdout:9/605: write da/d51/d55/f7b [5328931,67759] 0 2024-03-18T14:51:09.221 INFO:tasks.workunit.client.1.smithi100.stdout:9/606: dread - da/d46/d8b/f66 zero size 2024-03-18T14:51:09.222 INFO:tasks.workunit.client.1.smithi100.stdout:9/607: write da/d46/d4d/f9f [759569,16151] 0 2024-03-18T14:51:09.222 INFO:tasks.workunit.client.1.smithi100.stdout:3/627: dwrite d3/d16/d13/f3b [0,4194304] 0 2024-03-18T14:51:09.224 INFO:tasks.workunit.client.1.smithi100.stdout:5/581: rename d29/d2d/d39/f4c to d29/d2d/d8f/fb8 0 2024-03-18T14:51:09.224 INFO:tasks.workunit.client.1.smithi100.stdout:3/628: dread d3/d16/f45 [0,4194304] 0 2024-03-18T14:51:09.224 INFO:tasks.workunit.client.1.smithi100.stdout:3/629: fdatasync d3/d16/d13/f5f 0 2024-03-18T14:51:09.225 INFO:tasks.workunit.client.0.smithi012.stdout:1/822: dwrite d8/dbf/ff0 [0,4194304] 0 2024-03-18T14:51:09.225 INFO:tasks.workunit.client.0.smithi012.stdout:1/823: truncate d8/db/d10/d16/d54/f93 1437402 0 2024-03-18T14:51:09.225 INFO:tasks.workunit.client.0.smithi012.stdout:4/876: mknod d6/d1a/c132 0 2024-03-18T14:51:09.226 INFO:tasks.workunit.client.1.smithi100.stdout:6/568: dread d6/d7/df/d26/d4e/d66/d36/f6e [0,4194304] 0 2024-03-18T14:51:09.231 INFO:tasks.workunit.client.1.smithi100.stdout:9/608: truncate da/d2c/d2f/f4a 773555 0 2024-03-18T14:51:09.233 INFO:tasks.workunit.client.0.smithi012.stdout:1/824: rename d8/db/d10/d16/d6b/dc3 to d8/db/d10/d16/d6b/d107 0 2024-03-18T14:51:09.235 INFO:tasks.workunit.client.1.smithi100.stdout:0/655: dwrite dd/f1f [0,4194304] 0 2024-03-18T14:51:09.237 INFO:tasks.workunit.client.0.smithi012.stdout:4/877: creat d6/d19/f133 x:0 0 0 2024-03-18T14:51:09.237 INFO:tasks.workunit.client.0.smithi012.stdout:4/878: chown d6/da/d89/d97/lac 1996418 1 2024-03-18T14:51:09.238 INFO:tasks.workunit.client.1.smithi100.stdout:6/569: write d6/d12/f5c [780635,46486] 0 2024-03-18T14:51:09.240 INFO:tasks.workunit.client.1.smithi100.stdout:5/582: rename d29/d2d/d32/f35 to d29/d2d/d32/d92/fb9 0 2024-03-18T14:51:09.241 INFO:tasks.workunit.client.0.smithi012.stdout:5/872: dwrite d3/d15/d5f/fc0 [0,4194304] 0 2024-03-18T14:51:09.242 INFO:tasks.workunit.client.0.smithi012.stdout:1/825: mkdir d8/db/d10/d31/d45/d108 0 2024-03-18T14:51:09.242 INFO:tasks.workunit.client.0.smithi012.stdout:1/826: dread - d8/d1e/ffb zero size 2024-03-18T14:51:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:7/921: dwrite dd/d13/d23/d76/d6f/db3/dc7/fd8 [0,4194304] 0 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/630: mkdir d3/d16/d13/d1f/d3f/d4f/de0 0 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/631: fdatasync d3/d16/d13/d17/f61 0 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/632: chown d3/dd/d49/f9f 7 1 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/633: stat d3/dd/dd0 0 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:7/615: dwrite d5/d22/d24/f8b [0,4194304] 0 2024-03-18T14:51:09.244 INFO:tasks.workunit.client.1.smithi100.stdout:7/616: write d5/d14/d16/d2e/d62/fdf [985311,1525] 0 2024-03-18T14:51:09.248 INFO:tasks.workunit.client.1.smithi100.stdout:8/653: rmdir d4/dc/d31/d85/d8e 39 2024-03-18T14:51:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:4/879: getdents d6/da/d89/d94/d10d/d11d 0 2024-03-18T14:51:09.251 INFO:tasks.workunit.client.1.smithi100.stdout:1/585: mkdir d8/d1e/d11/d27/d3f/daa 0 2024-03-18T14:51:09.251 INFO:tasks.workunit.client.1.smithi100.stdout:1/586: creat d8/fab x:0 0 0 2024-03-18T14:51:09.251 INFO:tasks.workunit.client.1.smithi100.stdout:1/587: write d8/d1e/d11/d27/f64 [481524,11964] 0 2024-03-18T14:51:09.252 INFO:tasks.workunit.client.1.smithi100.stdout:6/570: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f88 [0,4194304] 0 2024-03-18T14:51:09.253 INFO:tasks.workunit.client.1.smithi100.stdout:2/606: getdents d3/d6/d5f 0 2024-03-18T14:51:09.255 INFO:tasks.workunit.client.0.smithi012.stdout:5/873: creat d3/d15/d42/dc6/d73/f124 x:0 0 0 2024-03-18T14:51:09.256 INFO:tasks.workunit.client.1.smithi100.stdout:9/609: mknod da/db/cbf 0 2024-03-18T14:51:09.259 INFO:tasks.workunit.client.0.smithi012.stdout:4/880: dread d6/d1a/f31 [0,4194304] 0 2024-03-18T14:51:09.259 INFO:tasks.workunit.client.0.smithi012.stdout:1/827: mkdir d8/d1e/d27/d44/d109 0 2024-03-18T14:51:09.266 INFO:tasks.workunit.client.0.smithi012.stdout:5/874: write d3/d15/f106 [5532776,87274] 0 2024-03-18T14:51:09.269 INFO:tasks.workunit.client.1.smithi100.stdout:6/571: dwrite d6/d7/df/d72/f85 [0,4194304] 0 2024-03-18T14:51:09.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/922: symlink dd/d13/d23/d2f/d7a/l13c 0 2024-03-18T14:51:09.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/923: chown dd/d13/d23/d76/d6f/le2 276598 1 2024-03-18T14:51:09.269 INFO:tasks.workunit.client.0.smithi012.stdout:7/924: dread - dd/d4f/fa9 zero size 2024-03-18T14:51:09.273 INFO:tasks.workunit.client.0.smithi012.stdout:1/828: link d8/d1e/f2f d8/db/d10/d31/d43/d5e/da5/f10a 0 2024-03-18T14:51:09.273 INFO:tasks.workunit.client.0.smithi012.stdout:4/881: write d6/da/d58/d113/d120/fcd [5638967,106437] 0 2024-03-18T14:51:09.278 INFO:tasks.workunit.client.1.smithi100.stdout:0/656: symlink dd/d2b/d95/dc1/lec 0 2024-03-18T14:51:09.278 INFO:tasks.workunit.client.1.smithi100.stdout:0/657: read - dd/d2b/d95/d79/d7d/f9d zero size 2024-03-18T14:51:09.278 INFO:tasks.workunit.client.1.smithi100.stdout:0/658: creat dd/d31/d91/d56/d62/fed x:0 0 0 2024-03-18T14:51:09.279 INFO:tasks.workunit.client.0.smithi012.stdout:7/925: creat dd/d13/d23/d2f/d7a/dab/f13d x:0 0 0 2024-03-18T14:51:09.279 INFO:tasks.workunit.client.0.smithi012.stdout:7/926: dread - dd/d13/d23/d76/d6b/f11b zero size 2024-03-18T14:51:09.279 INFO:tasks.workunit.client.0.smithi012.stdout:5/875: write d3/d15/d42/dc6/d65/fa1 [3835803,75653] 0 2024-03-18T14:51:09.279 INFO:tasks.workunit.client.0.smithi012.stdout:4/882: symlink d6/d19/l134 0 2024-03-18T14:51:09.280 INFO:tasks.workunit.client.1.smithi100.stdout:5/583: mknod d29/d2d/d39/d61/cba 0 2024-03-18T14:51:09.280 INFO:tasks.workunit.client.1.smithi100.stdout:5/584: stat d29/d2d/d39/d65/f98 0 2024-03-18T14:51:09.284 INFO:tasks.workunit.client.0.smithi012.stdout:7/927: write dd/d13/d23/d2f/d53/fd2 [645546,122816] 0 2024-03-18T14:51:09.285 INFO:tasks.workunit.client.0.smithi012.stdout:2/943: sync 2024-03-18T14:51:09.285 INFO:tasks.workunit.client.0.smithi012.stdout:2/944: write d0/d10/fe8 [129017,89360] 0 2024-03-18T14:51:09.285 INFO:tasks.workunit.client.0.smithi012.stdout:2/945: creat d0/d6c/db7/f12d x:0 0 0 2024-03-18T14:51:09.292 INFO:tasks.workunit.client.0.smithi012.stdout:7/928: link dd/d13/d23/d76/d109/d129/dd5/f107 dd/d13/d23/d76/d109/dfa/f13e 0 2024-03-18T14:51:09.292 INFO:tasks.workunit.client.0.smithi012.stdout:5/876: dread d3/d15/d42/dc6/d65/fa1 [0,4194304] 0 2024-03-18T14:51:09.299 INFO:tasks.workunit.client.0.smithi012.stdout:5/877: rename d3/d15/d5f/c54 to d3/d15/d42/dc6/d73/d6b/df3/c125 0 2024-03-18T14:51:09.306 INFO:tasks.workunit.client.1.smithi100.stdout:7/617: mknod d5/d22/db3/cea 0 2024-03-18T14:51:09.306 INFO:tasks.workunit.client.0.smithi012.stdout:7/929: dread dd/d13/d23/d2f/f9f [4194304,4194304] 0 2024-03-18T14:51:09.306 INFO:tasks.workunit.client.0.smithi012.stdout:7/930: fsync dd/d13/d23/d2f/d53/fa5 0 2024-03-18T14:51:09.308 INFO:tasks.workunit.client.1.smithi100.stdout:8/654: link f1 d4/dc/d1f/d59/d6c/d75/dea/feb 0 2024-03-18T14:51:09.308 INFO:tasks.workunit.client.1.smithi100.stdout:8/655: chown d4/dc/dd/de/d21/d23/d25/c70 3193 1 2024-03-18T14:51:09.310 INFO:tasks.workunit.client.1.smithi100.stdout:1/588: chown d8/d1e/d11/d27/d3f/f51 62487755 1 2024-03-18T14:51:09.313 INFO:tasks.workunit.client.1.smithi100.stdout:2/607: truncate d3/d6/d5f/d63/f72 3088944 0 2024-03-18T14:51:09.314 INFO:tasks.workunit.client.1.smithi100.stdout:9/610: symlink da/d46/d4d/d62/lc0 0 2024-03-18T14:51:09.316 INFO:tasks.workunit.client.1.smithi100.stdout:5/585: dwrite d29/d2d/f41 [4194304,4194304] 0 2024-03-18T14:51:09.317 INFO:tasks.workunit.client.0.smithi012.stdout:4/883: dwrite d6/da/d89/d97/dc5/fff [0,4194304] 0 2024-03-18T14:51:09.317 INFO:tasks.workunit.client.0.smithi012.stdout:4/884: write d6/d1a/d30/d46/d6f/d8b/f112 [25783,11019] 0 2024-03-18T14:51:09.317 INFO:tasks.workunit.client.0.smithi012.stdout:4/885: fdatasync d6/d1a/d30/f47 0 2024-03-18T14:51:09.318 INFO:tasks.workunit.client.1.smithi100.stdout:6/572: rename d6/d12/f1f to d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d63/fab 0 2024-03-18T14:51:09.318 INFO:tasks.workunit.client.0.smithi012.stdout:2/946: dwrite d0/d3c/d40/d47/d85/fdb [0,4194304] 0 2024-03-18T14:51:09.319 INFO:tasks.workunit.client.0.smithi012.stdout:1/829: dwrite d8/db/d10/d31/d45/f6f [0,4194304] 0 2024-03-18T14:51:09.319 INFO:tasks.workunit.client.0.smithi012.stdout:1/830: fsync d8/db/d10/d31/d81/fb3 0 2024-03-18T14:51:09.321 INFO:tasks.workunit.client.1.smithi100.stdout:0/659: unlink dd/d61/c7f 0 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- 172.21.15.12:0/1942620699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 msgr2=0x7f87ac102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 --2- 172.21.15.12:0/1942620699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac102320 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f879c004940 tx=0x7f879c009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- 172.21.15.12:0/1942620699 shutdown_connections 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 --2- 172.21.15.12:0/1942620699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac102320 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 --2- 172.21.15.12:0/1942620699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- 172.21.15.12:0/1942620699 >> 172.21.15.12:0/1942620699 conn(0x7f87ac0f7430 msgr2=0x7f87ac0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- 172.21.15.12:0/1942620699 shutdown_connections 2024-03-18T14:51:09.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- 172.21.15.12:0/1942620699 wait complete. 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 Processor -- start 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.321+0000 7f87b3f57700 1 -- start start 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b3f57700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b3f57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b3f57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0fbe70 con 0x7f87ac0fff00 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b3f57700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0fbfe0 con 0x7f87ac0662e0 2024-03-18T14:51:09.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50274/0 (socket says 172.21.15.12:50274) 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 -- 172.21.15.12:0/4088421679 learned_addr learned my addr 172.21.15.12:0/4088421679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b1cf3700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 -- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 msgr2=0x7f87ac0fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b14f2700 1 -- 172.21.15.12:0/4088421679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c005040 con 0x7f87ac0fff00 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.322+0000 7f87b1cf3700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87b14f2700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f879c013870 tx=0x7f879c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c00e070 con 0x7f87ac0fff00 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0fc260 con 0x7f87ac0fff00 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f879c026460 con 0x7f87ac0fff00 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c01c600 con 0x7f87ac0fff00 2024-03-18T14:51:09.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.323+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac0fc680 con 0x7f87ac0fff00 2024-03-18T14:51:09.326 INFO:tasks.workunit.client.1.smithi100.stdout:3/634: truncate d3/d16/d13/d1f/d3f/d4f/d60/f6b 2175910 0 2024-03-18T14:51:09.326 INFO:tasks.workunit.client.1.smithi100.stdout:0/660: dread dd/d20/f4e [0,4194304] 0 2024-03-18T14:51:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.325+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac05f5f0 con 0x7f87ac0fff00 2024-03-18T14:51:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.325+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f879c01c760 con 0x7f87ac0fff00 2024-03-18T14:51:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.326+0000 7f87a27fc700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 0x7f879806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.326+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f879c08ba10 con 0x7f87ac0fff00 2024-03-18T14:51:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.326+0000 7f87b1cf3700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 0x7f879806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.328+0000 7f87b1cf3700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 0x7f879806db60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0fd150 tx=0x7f87a800b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:09.330 INFO:tasks.workunit.client.0.smithi012.stdout:4/886: rename d6/d1a/d30/d46/d6f/d8b/c9b to d6/da/d89/da1/dbb/ddc/de5/c135 0 2024-03-18T14:51:09.331 INFO:tasks.workunit.client.1.smithi100.stdout:7/618: dwrite d5/d14/d16/d2e/d62/d66/d4f/fd8 [0,4194304] 0 2024-03-18T14:51:09.331 INFO:tasks.workunit.client.1.smithi100.stdout:7/619: dread - d5/d30/f5d zero size 2024-03-18T14:51:09.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.335+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f879c055640 con 0x7f87ac0fff00 2024-03-18T14:51:09.338 INFO:tasks.workunit.client.1.smithi100.stdout:3/635: dread d3/fb6 [0,4194304] 0 2024-03-18T14:51:09.340 INFO:tasks.workunit.client.0.smithi012.stdout:2/947: link d0/d6b/dbf/d33/d80/c83 d0/d6b/dbf/d33/d71/de5/ded/c12e 0 2024-03-18T14:51:09.343 INFO:tasks.workunit.client.1.smithi100.stdout:8/656: dread d4/dc/dd/de/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:09.343 INFO:tasks.workunit.client.1.smithi100.stdout:8/657: write d4/f15 [4601748,78421] 0 2024-03-18T14:51:09.344 INFO:tasks.workunit.client.1.smithi100.stdout:8/658: chown d4/fa8 15 1 2024-03-18T14:51:09.344 INFO:tasks.workunit.client.1.smithi100.stdout:8/659: chown d4/fa 7045940 1 2024-03-18T14:51:09.346 INFO:tasks.workunit.client.1.smithi100.stdout:9/611: getdents da 0 2024-03-18T14:51:09.348 INFO:tasks.workunit.client.0.smithi012.stdout:4/887: dread d6/f33 [0,4194304] 0 2024-03-18T14:51:09.354 INFO:tasks.workunit.client.1.smithi100.stdout:7/620: dwrite d5/d14/f40 [0,4194304] 0 2024-03-18T14:51:09.359 INFO:tasks.workunit.client.1.smithi100.stdout:1/589: rename d8/d2a/d37/d50/f90 to d8/d2a/d37/d50/d6f/fac 0 2024-03-18T14:51:09.361 INFO:tasks.workunit.client.1.smithi100.stdout:6/573: mknod d6/d7/df/d72/d65/d7a/cac 0 2024-03-18T14:51:09.365 INFO:tasks.workunit.client.1.smithi100.stdout:2/608: rmdir d3/da/de/d6e 39 2024-03-18T14:51:09.366 INFO:tasks.workunit.client.0.smithi012.stdout:1/831: truncate d8/db/d10/d31/d81/fb3 3264440 0 2024-03-18T14:51:09.367 INFO:tasks.workunit.client.0.smithi012.stdout:1/832: creat d8/d1e/d27/d44/dc2/f10b x:0 0 0 2024-03-18T14:51:09.367 INFO:tasks.workunit.client.0.smithi012.stdout:2/948: dwrite d0/d3c/d40/d47/d85/f94 [0,4194304] 0 2024-03-18T14:51:09.368 INFO:tasks.workunit.client.1.smithi100.stdout:3/636: dwrite d3/dd/f38 [0,4194304] 0 2024-03-18T14:51:09.368 INFO:tasks.workunit.client.1.smithi100.stdout:3/637: write d3/d16/d13/d17/f4c [2382402,61750] 0 2024-03-18T14:51:09.368 INFO:tasks.workunit.client.1.smithi100.stdout:3/638: write d3/d16/d13/d17/f96 [759849,71944] 0 2024-03-18T14:51:09.372 INFO:tasks.workunit.client.1.smithi100.stdout:8/660: truncate d4/dc/dd/de/d21/d23/f27 8036575 0 2024-03-18T14:51:09.374 INFO:tasks.workunit.client.1.smithi100.stdout:9/612: mkdir da/d51/d55/d7e/dc1 0 2024-03-18T14:51:09.374 INFO:tasks.workunit.client.1.smithi100.stdout:9/613: truncate da/dd/fa1 584704 0 2024-03-18T14:51:09.376 INFO:tasks.workunit.client.0.smithi012.stdout:1/833: rename d8/db/d10/d16/fa8 to d8/d1e/d106/f10c 0 2024-03-18T14:51:09.378 INFO:tasks.workunit.client.1.smithi100.stdout:2/609: dwrite d3/d13/d4f/fb5 [0,4194304] 0 2024-03-18T14:51:09.382 INFO:tasks.workunit.client.1.smithi100.stdout:1/590: creat d8/d2a/d37/d50/d6f/d71/fad x:0 0 0 2024-03-18T14:51:09.382 INFO:tasks.workunit.client.1.smithi100.stdout:1/591: fdatasync f7 0 2024-03-18T14:51:09.382 INFO:tasks.workunit.client.1.smithi100.stdout:1/592: dread - d8/fab zero size 2024-03-18T14:51:09.385 INFO:tasks.workunit.client.0.smithi012.stdout:2/949: unlink d0/d6b/dbf/d33/d77/df5/d103/d61/l63 0 2024-03-18T14:51:09.385 INFO:tasks.workunit.client.0.smithi012.stdout:2/950: write d0/d6b/dbf/d33/d77/d10a/f128 [1606481,28379] 0 2024-03-18T14:51:09.391 INFO:tasks.workunit.client.1.smithi100.stdout:3/639: mknod d3/d16/d13/d17/dab/ce1 0 2024-03-18T14:51:09.391 INFO:tasks.workunit.client.1.smithi100.stdout:3/640: write d3/d16/d13/d17/d7d/fa3 [992882,124297] 0 2024-03-18T14:51:09.392 INFO:tasks.workunit.client.0.smithi012.stdout:2/951: mkdir d0/d6c/db7/d12f 0 2024-03-18T14:51:09.395 INFO:tasks.workunit.client.0.smithi012.stdout:1/834: getdents d8/db/d10/d31/d81 0 2024-03-18T14:51:09.396 INFO:tasks.workunit.client.1.smithi100.stdout:4/585: sync 2024-03-18T14:51:09.397 INFO:tasks.workunit.client.0.smithi012.stdout:2/952: creat d0/db4/f130 x:0 0 0 2024-03-18T14:51:09.400 INFO:tasks.workunit.client.0.smithi012.stdout:1/835: symlink d8/d1e/d27/d44/dc2/de8/l10d 0 2024-03-18T14:51:09.400 INFO:tasks.workunit.client.0.smithi012.stdout:1/836: write d8/db/d10/d31/d43/d5e/d65/d70/dc7/fce [319157,41336] 0 2024-03-18T14:51:09.402 INFO:tasks.workunit.client.0.smithi012.stdout:2/953: rmdir d0/d3c/d40/d47 39 2024-03-18T14:51:09.402 INFO:tasks.workunit.client.0.smithi012.stdout:2/954: fsync d0/d6c/fa9 0 2024-03-18T14:51:09.402 INFO:tasks.workunit.client.0.smithi012.stdout:2/955: dread - d0/d6b/dbf/d33/dac/dd0/f124 zero size 2024-03-18T14:51:09.402 INFO:tasks.workunit.client.0.smithi012.stdout:2/956: fdatasync d0/da2/f67 0 2024-03-18T14:51:09.406 INFO:tasks.workunit.client.0.smithi012.stdout:1/837: creat d8/db/d10/d31/f10e x:0 0 0 2024-03-18T14:51:09.407 INFO:tasks.workunit.client.0.smithi012.stdout:2/957: truncate d0/db4/fb9 1058768 0 2024-03-18T14:51:09.411 INFO:tasks.workunit.client.0.smithi012.stdout:2/958: creat d0/d6b/dbf/f131 x:0 0 0 2024-03-18T14:51:09.413 INFO:tasks.workunit.client.1.smithi100.stdout:6/574: getdents d6/d7/df/d26/d4e/d66/d36 0 2024-03-18T14:51:09.414 INFO:tasks.workunit.client.0.smithi012.stdout:2/959: creat d0/d6b/dbf/d33/d99/f132 x:0 0 0 2024-03-18T14:51:09.414 INFO:tasks.workunit.client.0.smithi012.stdout:2/960: readlink d0/d6b/dbf/d33/d77/l9e 0 2024-03-18T14:51:09.417 INFO:tasks.workunit.client.1.smithi100.stdout:5/586: rename d29/d2d/d32/d38 to d29/d2d/d32/dad/dbb 0 2024-03-18T14:51:09.419 INFO:tasks.workunit.client.1.smithi100.stdout:4/586: dwrite d9/d27/f98 [0,4194304] 0 2024-03-18T14:51:09.419 INFO:tasks.workunit.client.1.smithi100.stdout:4/587: stat d9/d3a 0 2024-03-18T14:51:09.419 INFO:tasks.workunit.client.1.smithi100.stdout:4/588: dread - d9/d13/d5a/da4/f96 zero size 2024-03-18T14:51:09.419 INFO:tasks.workunit.client.1.smithi100.stdout:4/589: truncate d9/d13/d25/d51/f5d 4548317 0 2024-03-18T14:51:09.419 INFO:tasks.workunit.client.1.smithi100.stdout:7/621: rename d5/d22/d3b/d9f/dce to d5/d14/d16/d2e/d62/dd1/d86/d85/d87/de5/deb 0 2024-03-18T14:51:09.420 INFO:tasks.workunit.client.0.smithi012.stdout:1/838: dread d8/d1e/d27/d44/f4f [0,4194304] 0 2024-03-18T14:51:09.420 INFO:tasks.workunit.client.0.smithi012.stdout:1/839: readlink d8/dbf/le1 0 2024-03-18T14:51:09.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/587: mknod d29/d2d/d32/d92/daa/cbc 0 2024-03-18T14:51:09.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/575: write d6/d12/d2c/f7e [507165,116840] 0 2024-03-18T14:51:09.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/576: fdatasync d6/d12/f5c 0 2024-03-18T14:51:09.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/577: chown d6/d7 18170 1 2024-03-18T14:51:09.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/578: stat d6/d7/df/d26/f84 0 2024-03-18T14:51:09.423 INFO:tasks.workunit.client.0.smithi012.stdout:1/840: symlink d8/db/d10/d31/d43/d5e/df6/l10f 0 2024-03-18T14:51:09.430 INFO:tasks.workunit.client.1.smithi100.stdout:9/614: rename da/dd/d3c/d4b/c6f to da/d51/d55/cc2 0 2024-03-18T14:51:09.430 INFO:tasks.workunit.client.1.smithi100.stdout:9/615: read da/f16 [1343421,79409] 0 2024-03-18T14:51:09.430 INFO:tasks.workunit.client.1.smithi100.stdout:9/616: read - da/dd/f42 zero size 2024-03-18T14:51:09.430 INFO:tasks.workunit.client.1.smithi100.stdout:7/622: link d5/d14/d16/d2e/d62/d66/d4f/fd8 d5/d14/d16/d2e/d62/d66/d4f/dcb/fec 0 2024-03-18T14:51:09.431 INFO:tasks.workunit.client.1.smithi100.stdout:5/588: creat d29/d2d/d39/d65/d7d/fbd x:0 0 0 2024-03-18T14:51:09.433 INFO:tasks.workunit.client.1.smithi100.stdout:1/593: rename d8/d1e/d11/l72 to d8/d1e/d11/d27/d83/lae 0 2024-03-18T14:51:09.439 INFO:tasks.workunit.client.1.smithi100.stdout:7/623: link d5/d30/d4d/l58 d5/d22/d3b/d9f/da0/db1/dd3/led 0 2024-03-18T14:51:09.440 INFO:tasks.workunit.client.1.smithi100.stdout:5/589: mkdir d29/d2d/dbe 0 2024-03-18T14:51:09.443 INFO:tasks.workunit.client.1.smithi100.stdout:1/594: mknod d8/d1e/d11/d6d/d59/caf 0 2024-03-18T14:51:09.444 INFO:tasks.workunit.client.1.smithi100.stdout:9/617: rmdir da/dd/d3b 39 2024-03-18T14:51:09.444 INFO:tasks.workunit.client.1.smithi100.stdout:9/618: fdatasync da/dd/f42 0 2024-03-18T14:51:09.445 INFO:tasks.workunit.client.1.smithi100.stdout:5/590: rmdir d29/d2d/d32/d92/daa 39 2024-03-18T14:51:09.446 INFO:tasks.workunit.client.1.smithi100.stdout:3/641: rename d3/d16/d13/d1f/d3f/d4f/d60/l9e to d3/dd/d21/dae/dc9/le2 0 2024-03-18T14:51:09.447 INFO:tasks.workunit.client.1.smithi100.stdout:1/595: creat d8/d1e/d11/d27/d3f/d76/fb0 x:0 0 0 2024-03-18T14:51:09.448 INFO:tasks.workunit.client.1.smithi100.stdout:9/619: mkdir da/dc3 0 2024-03-18T14:51:09.448 INFO:tasks.workunit.client.1.smithi100.stdout:9/620: write da/d46/d4d/d62/fb7 [439263,37979] 0 2024-03-18T14:51:09.449 INFO:tasks.workunit.client.1.smithi100.stdout:5/591: unlink d29/d2d/d32/l64 0 2024-03-18T14:51:09.451 INFO:tasks.workunit.client.1.smithi100.stdout:8/661: rename d4/dc/d31/f3d to d4/dc/dd/de/d19/d2c/fec 0 2024-03-18T14:51:09.453 INFO:tasks.workunit.client.1.smithi100.stdout:9/621: creat da/d2c/d70/fc4 x:0 0 0 2024-03-18T14:51:09.453 INFO:tasks.workunit.client.1.smithi100.stdout:9/622: chown da/d51/d55/d7e/dc1 26 1 2024-03-18T14:51:09.454 INFO:tasks.workunit.client.1.smithi100.stdout:5/592: mkdir d29/d2d/d39/d60/dbf 0 2024-03-18T14:51:09.455 INFO:tasks.workunit.client.0.smithi012.stdout:2/961: dwrite d0/d6b/dbf/d33/d77/df5/d103/fc4 [0,4194304] 0 2024-03-18T14:51:09.456 INFO:tasks.workunit.client.1.smithi100.stdout:4/590: dwrite d9/d27/d62/f7d [4194304,4194304] 0 2024-03-18T14:51:09.457 INFO:tasks.workunit.client.1.smithi100.stdout:7/624: rename d5/d14/d16/d2e/d62/d66/d4f/dcb/fec to d5/d22/d24/d6e/d7a/dc2/fee 0 2024-03-18T14:51:09.459 INFO:tasks.workunit.client.1.smithi100.stdout:8/662: link d4/dc/dd/de/d19/d37/f65 d4/d96/fed 0 2024-03-18T14:51:09.459 INFO:tasks.workunit.client.1.smithi100.stdout:8/663: stat d4/dc/dd/de/d21/d23/f28 0 2024-03-18T14:51:09.460 INFO:tasks.workunit.client.1.smithi100.stdout:3/642: dread d3/d16/f14 [0,4194304] 0 2024-03-18T14:51:09.460 INFO:tasks.workunit.client.1.smithi100.stdout:6/579: dwrite d6/d12/f75 [0,4194304] 0 2024-03-18T14:51:09.460 INFO:tasks.workunit.client.1.smithi100.stdout:9/623: mknod da/db/cc5 0 2024-03-18T14:51:09.460 INFO:tasks.workunit.client.1.smithi100.stdout:9/624: write da/d46/d4d/d62/fb7 [1050403,55073] 0 2024-03-18T14:51:09.461 INFO:tasks.workunit.client.0.smithi012.stdout:6/847: sync 2024-03-18T14:51:09.461 INFO:tasks.workunit.client.0.smithi012.stdout:6/848: truncate d17/d19/d27/d58/d91/ff6 974655 0 2024-03-18T14:51:09.461 INFO:tasks.workunit.client.0.smithi012.stdout:1/841: dwrite d8/db/d10/d31/d43/d5e/d65/d84/dc1/fdc [0,4194304] 0 2024-03-18T14:51:09.465 INFO:tasks.workunit.client.1.smithi100.stdout:6/580: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [0,4194304] 0 2024-03-18T14:51:09.473 INFO:tasks.workunit.client.1.smithi100.stdout:4/591: link d9/d17/f23 d9/d13/d39/d76/da3/fb1 0 2024-03-18T14:51:09.473 INFO:tasks.workunit.client.1.smithi100.stdout:4/592: creat d9/d27/fb2 x:0 0 0 2024-03-18T14:51:09.474 INFO:tasks.workunit.client.1.smithi100.stdout:4/593: truncate d9/d13/d39/d4c/f5e 813160 0 2024-03-18T14:51:09.474 INFO:tasks.workunit.client.1.smithi100.stdout:4/594: write d9/d13/d5a/d5b/f9e [653983,31314] 0 2024-03-18T14:51:09.474 INFO:tasks.workunit.client.1.smithi100.stdout:4/595: truncate d9/d13/d25/d51/f60 321870 0 2024-03-18T14:51:09.474 INFO:tasks.workunit.client.1.smithi100.stdout:4/596: creat d9/d3a/fb3 x:0 0 0 2024-03-18T14:51:09.474 INFO:tasks.workunit.client.1.smithi100.stdout:2/610: dwrite d3/d6/d5f/d6a/fb2 [0,4194304] 0 2024-03-18T14:51:09.475 INFO:tasks.workunit.client.1.smithi100.stdout:1/596: dwrite d8/d1e/d11/d27/d73/d89/fa4 [0,4194304] 0 2024-03-18T14:51:09.475 INFO:tasks.workunit.client.1.smithi100.stdout:1/597: truncate d8/d30/d61/d8d/f95 23786 0 2024-03-18T14:51:09.475 INFO:tasks.workunit.client.1.smithi100.stdout:8/664: rmdir d4/d5a/dbb 39 2024-03-18T14:51:09.477 INFO:tasks.workunit.client.1.smithi100.stdout:3/643: mknod d3/dd/d21/d3e/ce3 0 2024-03-18T14:51:09.478 INFO:tasks.workunit.client.1.smithi100.stdout:9/625: link da/fc da/dd/d3b/fc6 0 2024-03-18T14:51:09.479 INFO:tasks.workunit.client.1.smithi100.stdout:6/581: rename d6/d7/df/d26/d4e/d66/d36/f6e to d6/d7/df/d72/d48/d76/fad 0 2024-03-18T14:51:09.480 INFO:tasks.workunit.client.1.smithi100.stdout:6/582: readlink d6/d7/df/d26/d4e/l50 0 2024-03-18T14:51:09.480 INFO:tasks.workunit.client.1.smithi100.stdout:6/583: read d6/d7/df/d26/d4e/f82 [346057,37459] 0 2024-03-18T14:51:09.498 INFO:tasks.workunit.client.1.smithi100.stdout:5/593: dwrite d29/d2d/d32/f5c [4194304,4194304] 0 2024-03-18T14:51:09.501 INFO:tasks.workunit.client.0.smithi012.stdout:1/842: dwrite d8/db/d10/d31/d43/d5e/d65/d70/fcb [0,4194304] 0 2024-03-18T14:51:09.505 INFO:tasks.workunit.client.0.smithi012.stdout:2/962: dwrite d0/f95 [0,4194304] 0 2024-03-18T14:51:09.505 INFO:tasks.workunit.client.0.smithi012.stdout:2/963: creat d0/d6b/dbf/d33/dc9/f133 x:0 0 0 2024-03-18T14:51:09.505 INFO:tasks.workunit.client.1.smithi100.stdout:7/625: dwrite d5/d30/d4d/d5a/f6a [0,4194304] 0 2024-03-18T14:51:09.505 INFO:tasks.workunit.client.1.smithi100.stdout:8/665: dwrite d4/dc/dd/de/d19/d37/f7f [0,4194304] 0 2024-03-18T14:51:09.505 INFO:tasks.workunit.client.1.smithi100.stdout:6/584: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa6 [0,4194304] 0 2024-03-18T14:51:09.514 INFO:tasks.workunit.client.1.smithi100.stdout:5/594: dwrite f12 [0,4194304] 0 2024-03-18T14:51:09.516 INFO:tasks.workunit.client.1.smithi100.stdout:5/595: dread f7 [0,4194304] 0 2024-03-18T14:51:09.516 INFO:tasks.workunit.client.1.smithi100.stdout:5/596: write d29/d4a/f8e [4609187,20128] 0 2024-03-18T14:51:09.516 INFO:tasks.workunit.client.1.smithi100.stdout:5/597: creat d29/d2d/d39/d61/fc0 x:0 0 0 2024-03-18T14:51:09.522 INFO:tasks.workunit.client.1.smithi100.stdout:5/598: dread d29/d2d/d39/f4e [0,4194304] 0 2024-03-18T14:51:09.527 INFO:tasks.workunit.client.0.smithi012.stdout:2/964: dwrite d0/d6b/dbf/d33/f62 [0,4194304] 0 2024-03-18T14:51:09.527 INFO:tasks.workunit.client.1.smithi100.stdout:6/585: dwrite d6/d7/df/d72/d8b/f8c [0,4194304] 0 2024-03-18T14:51:09.527 INFO:tasks.workunit.client.1.smithi100.stdout:6/586: write d6/d7/df/d26/d4e/d66/d36/f54 [273083,85024] 0 2024-03-18T14:51:09.570 INFO:tasks.workunit.client.1.smithi100.stdout:4/597: link d9/d3a/f99 d9/d13/d25/d51/d8d/fb4 0 2024-03-18T14:51:09.574 INFO:tasks.workunit.client.0.smithi012.stdout:1/843: mkdir d8/db/d10/d31/d81/d110 0 2024-03-18T14:51:09.577 INFO:tasks.workunit.client.1.smithi100.stdout:2/611: symlink d3/ld5 0 2024-03-18T14:51:09.583 INFO:tasks.workunit.client.1.smithi100.stdout:1/598: truncate d8/d1e/d11/d6d/fa0 478373 0 2024-03-18T14:51:09.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/644: unlink d3/d16/c2f 0 2024-03-18T14:51:09.586 INFO:tasks.workunit.client.1.smithi100.stdout:3/645: stat d3/dd/d21/dcd 0 2024-03-18T14:51:09.592 INFO:tasks.workunit.client.1.smithi100.stdout:9/626: rename da/dd/f28 to da/d88/da9/fc7 0 2024-03-18T14:51:09.600 INFO:tasks.workunit.client.0.smithi012.stdout:1/844: unlink d8/db/d10/d16/da1/f23 0 2024-03-18T14:51:09.616 INFO:tasks.workunit.client.0.smithi012.stdout:1/845: dwrite d8/db/d10/d31/f10e [0,4194304] 0 2024-03-18T14:51:09.627 INFO:tasks.workunit.client.0.smithi012.stdout:1/846: dread d8/db/d10/d31/d43/d5e/da5/fb8 [0,4194304] 0 2024-03-18T14:51:09.627 INFO:tasks.workunit.client.0.smithi012.stdout:1/847: chown d8/db/d10/d16/d6b/ce3 249118 1 2024-03-18T14:51:09.627 INFO:tasks.workunit.client.1.smithi100.stdout:8/666: read d4/dc/dd/de/d19/f2f [559241,59262] 0 2024-03-18T14:51:09.633 INFO:tasks.workunit.client.0.smithi012.stdout:2/965: rmdir d0 39 2024-03-18T14:51:09.634 INFO:tasks.workunit.client.1.smithi100.stdout:6/587: creat d6/d7/df/d26/d4e/d66/d36/d4f/fae x:0 0 0 2024-03-18T14:51:09.635 INFO:tasks.workunit.client.1.smithi100.stdout:0/661: sync 2024-03-18T14:51:09.635 INFO:tasks.workunit.client.1.smithi100.stdout:0/662: chown dd/d20/f97 1162172 1 2024-03-18T14:51:09.639 INFO:tasks.workunit.client.1.smithi100.stdout:4/598: link d9/d13/d5a/d5b/c47 d9/d27/d42/cb5 0 2024-03-18T14:51:09.640 INFO:tasks.workunit.client.0.smithi012.stdout:2/966: symlink d0/d6b/dbf/d33/d77/da0/d11c/l134 0 2024-03-18T14:51:09.640 INFO:tasks.workunit.client.0.smithi012.stdout:2/967: dread - d0/d6b/dbf/d33/d80/fe9 zero size 2024-03-18T14:51:09.642 INFO:tasks.workunit.client.1.smithi100.stdout:2/612: unlink d3/da/de/d48/f77 0 2024-03-18T14:51:09.643 INFO:tasks.workunit.client.0.smithi012.stdout:1/848: dwrite d8/d1e/d27/d44/f97 [0,4194304] 0 2024-03-18T14:51:09.643 INFO:tasks.workunit.client.0.smithi012.stdout:1/849: stat d8/db/d10/d16/f98 0 2024-03-18T14:51:09.644 INFO:tasks.workunit.client.1.smithi100.stdout:1/599: truncate d8/d1e/d11/d2d/f25 490989 0 2024-03-18T14:51:09.647 INFO:tasks.workunit.client.1.smithi100.stdout:3/646: unlink d3/d16/d13/d1f/d3f/d4f/d65/f91 0 2024-03-18T14:51:09.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/626: rename d5/fa to d5/d22/db3/fef 0 2024-03-18T14:51:09.653 INFO:tasks.workunit.client.0.smithi012.stdout:2/968: dread d0/da2/d59/f69 [0,4194304] 0 2024-03-18T14:51:09.653 INFO:tasks.workunit.client.1.smithi100.stdout:9/627: symlink da/d2c/d70/lc8 0 2024-03-18T14:51:09.659 INFO:tasks.workunit.client.1.smithi100.stdout:8/667: symlink d4/dc/dd/de/d21/d23/d25/lee 0 2024-03-18T14:51:09.662 INFO:tasks.workunit.client.1.smithi100.stdout:6/588: truncate d6/d7/df/d72/f85 902083 0 2024-03-18T14:51:09.663 INFO:tasks.workunit.client.1.smithi100.stdout:6/589: chown d6/cd 1 1 2024-03-18T14:51:09.665 INFO:tasks.workunit.client.1.smithi100.stdout:0/663: truncate dd/d20/f32 2245311 0 2024-03-18T14:51:09.665 INFO:tasks.workunit.client.1.smithi100.stdout:2/613: dwrite d3/d6/d29/d36/d49/f5b [0,4194304] 0 2024-03-18T14:51:09.665 INFO:tasks.workunit.client.1.smithi100.stdout:2/614: chown d3/d6/d29/f92 4057730 1 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:0/920: sync 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:0/921: readlink d1/d20/d23/d3b/d33/d51/la3 0 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:0/922: fdatasync d1/faa 0 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:0/923: creat d1/d16/d19/dd0/d129/d67/f12f x:0 0 0 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:0/924: stat d1/d20/d23/ca9 0 2024-03-18T14:51:09.667 INFO:tasks.workunit.client.0.smithi012.stdout:3/886: sync 2024-03-18T14:51:09.668 INFO:tasks.workunit.client.1.smithi100.stdout:4/599: link d9/d27/fb2 d9/da6/fb6 0 2024-03-18T14:51:09.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.669+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ac0f8f70 con 0x7f879806b6b0 2024-03-18T14:51:09.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.671+0000 7f87a27fc700 1 -- 172.21.15.12:0/4088421679 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f87ac0f8f70 con 0x7f879806b6b0 2024-03-18T14:51:09.672 INFO:tasks.workunit.client.1.smithi100.stdout:1/600: link d8/d1e/d11/d8a/fa8 d8/d2a/fb1 0 2024-03-18T14:51:09.672 INFO:tasks.workunit.client.1.smithi100.stdout:1/601: truncate d8/d2a/fa9 928695 0 2024-03-18T14:51:09.674 INFO:tasks.workunit.client.0.smithi012.stdout:1/850: dwrite d8/d1e/f9f [0,4194304] 0 2024-03-18T14:51:09.674 INFO:tasks.workunit.client.1.smithi100.stdout:3/647: unlink d3/d16/f84 0 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 msgr2=0x7f879806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 0x7f879806db60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0fd150 tx=0x7f87a800b040 comp rx=0 tx=0).stop 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 msgr2=0x7f87ac0fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f879c013870 tx=0x7f879c009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 shutdown_connections 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f879806b6b0 0x7f879806db60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87ac0662e0 0x7f87ac0fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 --2- 172.21.15.12:0/4088421679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87ac0fff00 0x7f87ac0fb8e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.674+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 >> 172.21.15.12:0/4088421679 conn(0x7f87ac0f7430 msgr2=0x7f87ac0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:09.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.675+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 shutdown_connections 2024-03-18T14:51:09.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.675+0000 7f87b3f57700 1 -- 172.21.15.12:0/4088421679 wait complete. 2024-03-18T14:51:09.678 INFO:tasks.workunit.client.0.smithi012.stdout:3/887: link dc/d3c/c6f dc/d30/d3a/c11d 0 2024-03-18T14:51:09.678 INFO:tasks.workunit.client.0.smithi012.stdout:3/888: stat dc/f6a 0 2024-03-18T14:51:09.682 INFO:tasks.workunit.client.1.smithi100.stdout:5/599: rename d29/d2d/d32/dad to d29/d2d/d39/d83/dc1 0 2024-03-18T14:51:09.682 INFO:tasks.workunit.client.1.smithi100.stdout:5/600: fsync d29/d2d/d39/d83/dc1/dbb/f5d 0 2024-03-18T14:51:09.685 INFO:tasks.workunit.client.0.smithi012.stdout:2/969: dwrite d0/d6b/dbf/d33/d80/fc8 [0,4194304] 0 2024-03-18T14:51:09.686 INFO:tasks.workunit.client.1.smithi100.stdout:7/627: symlink d5/d14/d16/d2e/d62/dd1/d86/d85/d87/de5/deb/lf0 0 2024-03-18T14:51:09.686 INFO:tasks.workunit.client.1.smithi100.stdout:1/602: write f5 [3308898,103553] 0 2024-03-18T14:51:09.686 INFO:tasks.workunit.client.1.smithi100.stdout:3/648: write d3/d16/d13/f47 [2622487,110580] 0 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/628: unlink da/d46/d4d/d67/la8 0 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/629: chown da/dc3 0 1 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/630: write da/d51/f69 [1821314,10805] 0 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:9/631: chown da/d46/d4d/d62/fb7 407891918 1 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:8/668: symlink d4/dc/dd/lef 0 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.1.smithi100.stdout:8/669: truncate d4/dc/dd/de/d21/f72 313922 0 2024-03-18T14:51:09.691 INFO:tasks.workunit.client.0.smithi012.stdout:3/889: chown dc/c23 50799776 1 2024-03-18T14:51:09.695 INFO:tasks.workunit.client.1.smithi100.stdout:6/590: link d6/d7/df/d26/f84 d6/d12/da2/faf 0 2024-03-18T14:51:09.698 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:51:09.699 INFO:tasks.workunit.client.0.smithi012.stdout:3/890: creat dc/d3c/d4c/d57/d103/d9a/dd7/de3/f11e x:0 0 0 2024-03-18T14:51:09.699 INFO:tasks.workunit.client.0.smithi012.stdout:3/891: stat dc/d30/d3a/da0/dc5 0 2024-03-18T14:51:09.701 INFO:tasks.workunit.client.1.smithi100.stdout:2/615: truncate d3/da/de/fa5 699270 0 2024-03-18T14:51:09.703 INFO:tasks.workunit.client.1.smithi100.stdout:4/600: symlink d9/d27/da2/lb7 0 2024-03-18T14:51:09.703 INFO:tasks.workunit.client.1.smithi100.stdout:4/601: write d9/d17/f34 [3992371,16637] 0 2024-03-18T14:51:09.705 INFO:tasks.workunit.client.0.smithi012.stdout:3/892: rename dc/d3c/d4c/d57/d103/d9a/d54/f97 to dc/d30/d3a/db7/f11f 0 2024-03-18T14:51:09.705 INFO:tasks.workunit.client.1.smithi100.stdout:5/601: mkdir d29/d2d/dbe/dc2 0 2024-03-18T14:51:09.705 INFO:tasks.workunit.client.1.smithi100.stdout:5/602: fsync d29/d2d/d39/d61/f6b 0 2024-03-18T14:51:09.706 INFO:tasks.workunit.client.0.smithi012.stdout:5/878: sync 2024-03-18T14:51:09.707 INFO:tasks.workunit.client.1.smithi100.stdout:7/628: creat d5/d22/d3b/d9f/da0/db1/dd3/ff1 x:0 0 0 2024-03-18T14:51:09.709 INFO:tasks.workunit.client.0.smithi012.stdout:3/893: rename dc/f48 to dc/d3c/d72/df3/d109/d11a/f120 0 2024-03-18T14:51:09.709 INFO:tasks.workunit.client.0.smithi012.stdout:3/894: read dc/d3c/d4c/d5f/f67 [2667143,92247] 0 2024-03-18T14:51:09.709 INFO:tasks.workunit.client.0.smithi012.stdout:3/895: write dc/d3c/d72/f7b [1108762,114674] 0 2024-03-18T14:51:09.711 INFO:tasks.workunit.client.1.smithi100.stdout:1/603: mknod d8/d1e/d11/d27/d73/cb2 0 2024-03-18T14:51:09.711 INFO:tasks.workunit.client.1.smithi100.stdout:1/604: stat d8/d30/d61/da2 0 2024-03-18T14:51:09.711 INFO:tasks.workunit.client.1.smithi100.stdout:5/603: dread d29/d2d/d39/f84 [0,4194304] 0 2024-03-18T14:51:09.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/896: mknod dc/d30/d3a/da0/dc5/dc7/dd8/c121 0 2024-03-18T14:51:09.714 INFO:tasks.workunit.client.1.smithi100.stdout:9/632: creat da/dd/d3c/fc9 x:0 0 0 2024-03-18T14:51:09.714 INFO:tasks.workunit.client.1.smithi100.stdout:8/670: rmdir d4/dc/d1f/d59/d6c/d75/dea 39 2024-03-18T14:51:09.714 INFO:tasks.workunit.client.1.smithi100.stdout:8/671: chown d4/dc/d1f/d33/c52 6099727 1 2024-03-18T14:51:09.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/591: unlink d6/f5e 0 2024-03-18T14:51:09.718 INFO:tasks.workunit.client.1.smithi100.stdout:2/616: mknod d3/d6/d29/d36/d4a/cd6 0 2024-03-18T14:51:09.718 INFO:tasks.workunit.client.1.smithi100.stdout:2/617: readlink d3/d6/d29/d36/d49/l8d 0 2024-03-18T14:51:09.718 INFO:tasks.workunit.client.1.smithi100.stdout:2/618: write d3/d6/d29/d36/d49/f7a [4294139,31959] 0 2024-03-18T14:51:09.718 INFO:tasks.workunit.client.1.smithi100.stdout:0/664: dwrite dd/d31/d91/d56/d59/f7c [0,4194304] 0 2024-03-18T14:51:09.719 INFO:tasks.workunit.client.1.smithi100.stdout:4/602: symlink d9/d13/d39/d76/lb8 0 2024-03-18T14:51:09.719 INFO:tasks.workunit.client.1.smithi100.stdout:4/603: fdatasync d9/d13/d5a/f78 0 2024-03-18T14:51:09.719 INFO:tasks.workunit.client.1.smithi100.stdout:9/633: dread da/d2c/d2f/f4a [0,4194304] 0 2024-03-18T14:51:09.720 INFO:tasks.workunit.client.1.smithi100.stdout:7/629: unlink d5/d14/d16/c3a 0 2024-03-18T14:51:09.720 INFO:tasks.workunit.client.0.smithi012.stdout:0/925: dwrite d1/d20/d23/d6f/d93/dba/fe1 [0,4194304] 0 2024-03-18T14:51:09.720 INFO:tasks.workunit.client.0.smithi012.stdout:2/970: dwrite d0/da2/d59/fa4 [0,4194304] 0 2024-03-18T14:51:09.721 INFO:tasks.workunit.client.0.smithi012.stdout:1/851: dwrite d8/d1e/d106/f10c [0,4194304] 0 2024-03-18T14:51:09.721 INFO:tasks.workunit.client.1.smithi100.stdout:1/605: mknod d8/d1e/d11/cb3 0 2024-03-18T14:51:09.721 INFO:tasks.workunit.client.1.smithi100.stdout:1/606: write d8/d30/d61/d8d/f95 [635911,94053] 0 2024-03-18T14:51:09.723 INFO:tasks.workunit.client.0.smithi012.stdout:5/879: write d3/d1e/d33/f4b [3207394,116631] 0 2024-03-18T14:51:09.723 INFO:tasks.workunit.client.1.smithi100.stdout:5/604: mknod d29/d2d/d39/d83/dc1/cc3 0 2024-03-18T14:51:09.723 INFO:tasks.workunit.client.1.smithi100.stdout:5/605: fdatasync d29/d2d/d39/d83/f85 0 2024-03-18T14:51:09.730 INFO:tasks.workunit.client.1.smithi100.stdout:8/672: mkdir d4/dc/dd/d48/dd8/df0 0 2024-03-18T14:51:09.730 INFO:tasks.workunit.client.1.smithi100.stdout:6/592: link d6/d7/df/d72/d65/d7a/cac d6/d7/df/d72/d8b/cb0 0 2024-03-18T14:51:09.730 INFO:tasks.workunit.client.1.smithi100.stdout:6/593: chown d6/d12/d78/c9b 14217 1 2024-03-18T14:51:09.730 INFO:tasks.workunit.client.0.smithi012.stdout:3/897: dwrite dc/f12 [0,4194304] 0 2024-03-18T14:51:09.732 INFO:tasks.workunit.client.0.smithi012.stdout:2/971: rmdir d0/d3c/d40/d47/d85 39 2024-03-18T14:51:09.732 INFO:tasks.workunit.client.0.smithi012.stdout:2/972: readlink d0/d6b/dbf/d33/l56 0 2024-03-18T14:51:09.732 INFO:tasks.workunit.client.0.smithi012.stdout:2/973: truncate d0/d6b/dbf/d4f/f6f 2123945 0 2024-03-18T14:51:09.733 INFO:tasks.workunit.client.1.smithi100.stdout:2/619: getdents d3/d6/d5f/d63 0 2024-03-18T14:51:09.735 INFO:tasks.workunit.client.0.smithi012.stdout:0/926: mknod d1/d16/d31/c130 0 2024-03-18T14:51:09.735 INFO:tasks.workunit.client.0.smithi012.stdout:0/927: chown d1/d16/d19/dd0/d129/d67/d10c/c115 209 1 2024-03-18T14:51:09.741 INFO:tasks.workunit.client.1.smithi100.stdout:0/665: symlink dd/d61/lee 0 2024-03-18T14:51:09.742 INFO:tasks.workunit.client.1.smithi100.stdout:0/666: dread - dd/d31/d91/fe1 zero size 2024-03-18T14:51:09.742 INFO:tasks.workunit.client.1.smithi100.stdout:0/667: stat dd/d61/lee 0 2024-03-18T14:51:09.743 INFO:tasks.workunit.client.0.smithi012.stdout:1/852: creat d8/db/d10/d31/d43/d5e/da5/f111 x:0 0 0 2024-03-18T14:51:09.744 INFO:tasks.workunit.client.1.smithi100.stdout:4/604: symlink d9/d13/d5a/lb9 0 2024-03-18T14:51:09.744 INFO:tasks.workunit.client.1.smithi100.stdout:0/668: dread dd/d20/fa0 [0,4194304] 0 2024-03-18T14:51:09.744 INFO:tasks.workunit.client.1.smithi100.stdout:2/620: dread d3/d6/d5f/f6d [0,4194304] 0 2024-03-18T14:51:09.746 INFO:tasks.workunit.client.0.smithi012.stdout:1/853: dread d8/db/d10/d31/d43/d5e/d65/fda [0,4194304] 0 2024-03-18T14:51:09.746 INFO:tasks.workunit.client.0.smithi012.stdout:1/854: creat d8/db/dbe/f112 x:0 0 0 2024-03-18T14:51:09.747 INFO:tasks.workunit.client.1.smithi100.stdout:7/630: dwrite d5/d14/d16/d2e/d62/dd1/f43 [0,4194304] 0 2024-03-18T14:51:09.747 INFO:tasks.workunit.client.1.smithi100.stdout:7/631: creat d5/d14/d16/d2e/d62/ff2 x:0 0 0 2024-03-18T14:51:09.747 INFO:tasks.workunit.client.1.smithi100.stdout:9/634: link da/dd/fa1 da/d46/d4d/d67/fca 0 2024-03-18T14:51:09.749 INFO:tasks.workunit.client.0.smithi012.stdout:5/880: rename d3/d84/f85 to d3/d15/d42/f126 0 2024-03-18T14:51:09.749 INFO:tasks.workunit.client.0.smithi012.stdout:5/881: fdatasync d3/dad/df2/ffc 0 2024-03-18T14:51:09.749 INFO:tasks.workunit.client.1.smithi100.stdout:2/621: dread d3/d13/d40/fa7 [0,4194304] 0 2024-03-18T14:51:09.749 INFO:tasks.workunit.client.1.smithi100.stdout:2/622: creat d3/fd7 x:0 0 0 2024-03-18T14:51:09.750 INFO:tasks.workunit.client.1.smithi100.stdout:1/607: mkdir d8/d30/d61/db4 0 2024-03-18T14:51:09.751 INFO:tasks.workunit.client.1.smithi100.stdout:5/606: dwrite d29/d2d/d39/d65/f6a [0,4194304] 0 2024-03-18T14:51:09.758 INFO:tasks.workunit.client.0.smithi012.stdout:1/855: write d8/db/d10/d31/fba [1191998,118797] 0 2024-03-18T14:51:09.759 INFO:tasks.workunit.client.0.smithi012.stdout:1/856: dread d8/d1e/d27/f28 [0,4194304] 0 2024-03-18T14:51:09.765 INFO:tasks.workunit.client.0.smithi012.stdout:3/898: unlink dc/d3c/fde 0 2024-03-18T14:51:09.767 INFO:tasks.workunit.client.0.smithi012.stdout:2/974: symlink d0/d6b/dbf/d33/d77/df5/l135 0 2024-03-18T14:51:09.769 INFO:tasks.workunit.client.1.smithi100.stdout:8/673: truncate d4/dc/f8a 3364565 0 2024-03-18T14:51:09.769 INFO:tasks.workunit.client.1.smithi100.stdout:8/674: getdents d4/dc2 0 2024-03-18T14:51:09.769 INFO:tasks.workunit.client.1.smithi100.stdout:8/675: chown d4/dc/dd/f44 11 1 2024-03-18T14:51:09.770 INFO:tasks.workunit.client.0.smithi012.stdout:0/928: truncate d1/f10 2180187 0 2024-03-18T14:51:09.771 INFO:tasks.workunit.client.0.smithi012.stdout:0/929: write d1/d20/d4e/f118 [356601,41522] 0 2024-03-18T14:51:09.773 INFO:tasks.workunit.client.1.smithi100.stdout:6/594: rename d6/d7/c44 to d6/d12/d2c/cb1 0 2024-03-18T14:51:09.777 INFO:tasks.workunit.client.1.smithi100.stdout:4/605: mknod d9/d27/d62/d7b/cba 0 2024-03-18T14:51:09.777 INFO:tasks.workunit.client.1.smithi100.stdout:4/606: fdatasync d9/d13/d39/f8f 0 2024-03-18T14:51:09.779 INFO:tasks.workunit.client.1.smithi100.stdout:0/669: getdents dd/d2b/d95/d79/d7d/da9 0 2024-03-18T14:51:09.787 INFO:tasks.workunit.client.1.smithi100.stdout:7/632: creat d5/d14/ff3 x:0 0 0 2024-03-18T14:51:09.787 INFO:tasks.workunit.client.1.smithi100.stdout:7/633: dread - d5/d30/d4d/d5a/f82 zero size 2024-03-18T14:51:09.787 INFO:tasks.workunit.client.1.smithi100.stdout:9/635: truncate da/dd/d29/f34 373286 0 2024-03-18T14:51:09.788 INFO:tasks.workunit.client.0.smithi012.stdout:4/888: sync 2024-03-18T14:51:09.788 INFO:tasks.workunit.client.0.smithi012.stdout:7/931: sync 2024-03-18T14:51:09.788 INFO:tasks.workunit.client.0.smithi012.stdout:6/849: sync 2024-03-18T14:51:09.788 INFO:tasks.workunit.client.0.smithi012.stdout:7/932: creat dd/d13/d103/d12b/f13f x:0 0 0 2024-03-18T14:51:09.791 INFO:tasks.workunit.client.1.smithi100.stdout:2/623: rmdir d3/da/de/dc0 0 2024-03-18T14:51:09.793 INFO:tasks.workunit.client.1.smithi100.stdout:1/608: creat d8/d1e/d11/d27/d3f/daa/fb5 x:0 0 0 2024-03-18T14:51:09.793 INFO:tasks.workunit.client.1.smithi100.stdout:1/609: creat d8/fb6 x:0 0 0 2024-03-18T14:51:09.795 INFO:tasks.workunit.client.0.smithi012.stdout:5/882: rename d3/d1e/c2e to d3/d1e/d33/c127 0 2024-03-18T14:51:09.802 INFO:tasks.workunit.client.0.smithi012.stdout:6/850: dread d17/f75 [0,4194304] 0 2024-03-18T14:51:09.804 INFO:tasks.workunit.client.0.smithi012.stdout:7/933: dread dd/d13/d23/d105/f127 [0,4194304] 0 2024-03-18T14:51:09.804 INFO:tasks.workunit.client.0.smithi012.stdout:7/934: stat dd/d13/d23/d2f/d53/d8d/l116 0 2024-03-18T14:51:09.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/676: dwrite d4/dc/d1f/f9c [0,4194304] 0 2024-03-18T14:51:09.805 INFO:tasks.workunit.client.0.smithi012.stdout:0/930: link d1/d2d/l11e d1/d20/d7b/l131 0 2024-03-18T14:51:09.806 INFO:tasks.workunit.client.0.smithi012.stdout:2/975: dwrite d0/d6b/dbf/d33/d99/daa/f129 [0,4194304] 0 2024-03-18T14:51:09.807 INFO:tasks.workunit.client.1.smithi100.stdout:5/607: rmdir d29/d2d/d32/d92/daa 39 2024-03-18T14:51:09.807 INFO:tasks.workunit.client.1.smithi100.stdout:5/608: truncate d29/d2d/d39/d83/dc1/dbb/fb1 755663 0 2024-03-18T14:51:09.810 INFO:tasks.workunit.client.0.smithi012.stdout:3/899: rename dc/d3c/d72/fee to dc/d30/d3a/d112/f122 0 2024-03-18T14:51:09.813 INFO:tasks.workunit.client.1.smithi100.stdout:9/636: dwrite da/dd/d29/d71/d72/faa [0,4194304] 0 2024-03-18T14:51:09.813 INFO:tasks.workunit.client.1.smithi100.stdout:9/637: truncate da/dd/f10 3204417 0 2024-03-18T14:51:09.820 INFO:tasks.workunit.client.0.smithi012.stdout:6/851: creat d17/d19/d27/d6e/d84/ffa x:0 0 0 2024-03-18T14:51:09.821 INFO:tasks.workunit.client.1.smithi100.stdout:8/677: dread d4/dc/dd/de/f57 [0,4194304] 0 2024-03-18T14:51:09.821 INFO:tasks.workunit.client.0.smithi012.stdout:0/931: truncate d1/d16/d19/f56 424391 0 2024-03-18T14:51:09.823 INFO:tasks.workunit.client.1.smithi100.stdout:6/595: creat d6/d12/d78/fb2 x:0 0 0 2024-03-18T14:51:09.823 INFO:tasks.workunit.client.1.smithi100.stdout:4/607: mknod d9/d13/d5a/d5b/cbb 0 2024-03-18T14:51:09.825 INFO:tasks.workunit.client.1.smithi100.stdout:5/609: dwrite d29/d2d/d39/d60/f70 [0,4194304] 0 2024-03-18T14:51:09.825 INFO:tasks.workunit.client.0.smithi012.stdout:4/889: dwrite d6/d19/d39/fc4 [0,4194304] 0 2024-03-18T14:51:09.826 INFO:tasks.workunit.client.1.smithi100.stdout:0/670: rename dd/d20/d44/l73 to dd/lef 0 2024-03-18T14:51:09.829 INFO:tasks.workunit.client.1.smithi100.stdout:7/634: creat d5/d14/d16/ff4 x:0 0 0 2024-03-18T14:51:09.829 INFO:tasks.workunit.client.1.smithi100.stdout:7/635: write d5/d14/d16/d2e/d62/ff2 [90103,97696] 0 2024-03-18T14:51:09.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/671: dread dd/d20/f2d [0,4194304] 0 2024-03-18T14:51:09.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/672: read dd/d20/f32 [121823,89987] 0 2024-03-18T14:51:09.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/673: truncate dd/d31/d91/d56/d62/fe2 948086 0 2024-03-18T14:51:09.832 INFO:tasks.workunit.client.1.smithi100.stdout:4/608: write d9/f32 [6651744,72260] 0 2024-03-18T14:51:09.832 INFO:tasks.workunit.client.1.smithi100.stdout:4/609: write d9/d13/d39/d4c/f7a [5087624,124788] 0 2024-03-18T14:51:09.832 INFO:tasks.workunit.client.1.smithi100.stdout:4/610: chown d9/d27/fa9 11 1 2024-03-18T14:51:09.834 INFO:tasks.workunit.client.0.smithi012.stdout:6/852: link d17/d19/d27/f68 d17/d26/d36/d4d/d82/d97/ffb 0 2024-03-18T14:51:09.835 INFO:tasks.workunit.client.0.smithi012.stdout:1/857: dwrite d8/db/f7d [0,4194304] 0 2024-03-18T14:51:09.835 INFO:tasks.workunit.client.0.smithi012.stdout:1/858: getdents d8/db/d10/d31/d45/d108 0 2024-03-18T14:51:09.836 INFO:tasks.workunit.client.1.smithi100.stdout:1/610: mkdir d8/d2a/d37/d50/d6f/d71/db7 0 2024-03-18T14:51:09.842 INFO:tasks.workunit.client.0.smithi012.stdout:0/932: getdents d1/d20/d23/d82 0 2024-03-18T14:51:09.843 INFO:tasks.workunit.client.1.smithi100.stdout:3/649: sync 2024-03-18T14:51:09.846 INFO:tasks.workunit.client.0.smithi012.stdout:7/935: rename dd/d13/d23/d76/d6f/db3 to dd/d13/d23/d60/d140 0 2024-03-18T14:51:09.848 INFO:tasks.workunit.client.0.smithi012.stdout:6/853: creat d17/d18/dc1/dde/ffc x:0 0 0 2024-03-18T14:51:09.850 INFO:tasks.workunit.client.1.smithi100.stdout:9/638: mkdir da/dcb 0 2024-03-18T14:51:09.851 INFO:tasks.workunit.client.1.smithi100.stdout:9/639: dread - da/dd/f8f zero size 2024-03-18T14:51:09.851 INFO:tasks.workunit.client.1.smithi100.stdout:9/640: dread da/dd/fa1 [0,4194304] 0 2024-03-18T14:51:09.851 INFO:tasks.workunit.client.1.smithi100.stdout:9/641: readlink da/l22 0 2024-03-18T14:51:09.851 INFO:tasks.workunit.client.1.smithi100.stdout:9/642: dread - da/d46/d8b/f37 zero size 2024-03-18T14:51:09.853 INFO:tasks.workunit.client.1.smithi100.stdout:8/678: dread d4/dc/dd/de/d21/f72 [0,4194304] 0 2024-03-18T14:51:09.856 INFO:tasks.workunit.client.1.smithi100.stdout:6/596: link d6/d7/df/d72/d65/fa9 d6/fb3 0 2024-03-18T14:51:09.856 INFO:tasks.workunit.client.1.smithi100.stdout:6/597: creat d6/d12/d78/fb4 x:0 0 0 2024-03-18T14:51:09.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 -- 172.21.15.12:0/2585123001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 msgr2=0x7f82d00fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/2585123001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d00fe4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82c0004880 tx=0x7f82c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:09.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 -- 172.21.15.12:0/2585123001 shutdown_connections 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/2585123001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d00fe4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/2585123001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d00fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 -- 172.21.15.12:0/2585123001 >> 172.21.15.12:0/2585123001 conn(0x7f82d00f85a0 msgr2=0x7f82d00fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 -- 172.21.15.12:0/2585123001 shutdown_connections 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.856+0000 7f82d7a2d700 1 -- 172.21.15.12:0/2585123001 wait complete. 2024-03-18T14:51:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.857+0000 7f82d7a2d700 1 Processor -- start 2024-03-18T14:51:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.857+0000 7f82d7a2d700 1 -- start start 2024-03-18T14:51:09.859 INFO:tasks.workunit.client.0.smithi012.stdout:1/859: unlink d8/db/d10/d31/d43/d5e/d65/d84/fcf 0 2024-03-18T14:51:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d7a2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d7a2d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d7a2d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d01116e0 con 0x7f82d00fcf00 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d7a2d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0111850 con 0x7f82d00fe070 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50292/0 (socket says 172.21.15.12:50292) 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 -- 172.21.15.12:0/1611292139 learned_addr learned my addr 172.21.15.12:0/1611292139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d4fc8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 -- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 msgr2=0x7f82d0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.858+0000 7f82d57c9700 1 -- 172.21.15.12:0/1611292139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0005040 con 0x7f82d00fcf00 2024-03-18T14:51:09.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.859+0000 7f82d57c9700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0041d0 tx=0x7f82cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.859+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc007cc0 con 0x7f82d00fcf00 2024-03-18T14:51:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.859+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d0070910 con 0x7f82d00fcf00 2024-03-18T14:51:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.859+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82cc00f040 con 0x7f82d00fcf00 2024-03-18T14:51:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.860+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d0070e80 con 0x7f82d00fcf00 2024-03-18T14:51:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.860+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82cc0092b0 con 0x7f82d00fcf00 2024-03-18T14:51:09.862 INFO:tasks.workunit.client.0.smithi012.stdout:0/933: creat d1/d16/d6e/d109/f132 x:0 0 0 2024-03-18T14:51:09.862 INFO:tasks.workunit.client.0.smithi012.stdout:0/934: dread - d1/d16/d31/fec zero size 2024-03-18T14:51:09.862 INFO:tasks.workunit.client.0.smithi012.stdout:0/935: fsync d1/d16/d19/dd0/d129/d67/ff4 0 2024-03-18T14:51:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.861+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f82cc00c810 con 0x7f82d00fcf00 2024-03-18T14:51:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.861+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d005f5f0 con 0x7f82d00fcf00 2024-03-18T14:51:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.862+0000 7f82c5ffb700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 0x7f82bc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.862+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f82cc089c60 con 0x7f82d00fcf00 2024-03-18T14:51:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.862+0000 7f82d4fc8700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 0x7f82bc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:09.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.863+0000 7f82d4fc8700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 0x7f82bc06d960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82c0013830 tx=0x7f82c0009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:09.864 INFO:tasks.workunit.client.1.smithi100.stdout:8/679: dread d4/dc/dd/de/d21/d23/f27 [0,4194304] 0 2024-03-18T14:51:09.864 INFO:tasks.workunit.client.1.smithi100.stdout:8/680: readlink d4/d3c/lcd 0 2024-03-18T14:51:09.867 INFO:tasks.workunit.client.0.smithi012.stdout:4/890: rename d6/da/d89/da1/fe8 to d6/d1a/d98/daf/f136 0 2024-03-18T14:51:09.867 INFO:tasks.workunit.client.0.smithi012.stdout:4/891: dread - d6/d19/d6c/d96/f116 zero size 2024-03-18T14:51:09.871 INFO:tasks.workunit.client.1.smithi100.stdout:7/636: dread - d5/d22/d3b/d9f/da0/db1/dd3/fe0 zero size 2024-03-18T14:51:09.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:09.870+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f82cc053810 con 0x7f82d00fcf00 2024-03-18T14:51:09.873 INFO:tasks.workunit.client.0.smithi012.stdout:1/860: write d8/d1e/d106/f10c [586697,57058] 0 2024-03-18T14:51:09.873 INFO:tasks.workunit.client.0.smithi012.stdout:7/936: rename dd/d13/d23/d76/d109/d129/dd5/f12d to dd/d13/d103/f141 0 2024-03-18T14:51:09.873 INFO:tasks.workunit.client.0.smithi012.stdout:7/937: fdatasync dd/d13/d23/d2f/d7a/db6/f9b 0 2024-03-18T14:51:09.873 INFO:tasks.workunit.client.0.smithi012.stdout:7/938: chown dd/d13/d23/d76/d109/d129/fa8 1 1 2024-03-18T14:51:09.873 INFO:tasks.workunit.client.0.smithi012.stdout:7/939: write dd/d13/d23/d60/f89 [954368,92430] 0 2024-03-18T14:51:09.874 INFO:tasks.workunit.client.0.smithi012.stdout:7/940: dread - dd/d13/d23/d76/d109/dfa/f123 zero size 2024-03-18T14:51:09.874 INFO:tasks.workunit.client.1.smithi100.stdout:3/650: dwrite d3/d16/d13/d1f/d3f/d4f/f81 [0,4194304] 0 2024-03-18T14:51:09.876 INFO:tasks.workunit.client.1.smithi100.stdout:0/674: mknod dd/d31/d91/d7b/cf0 0 2024-03-18T14:51:09.876 INFO:tasks.workunit.client.1.smithi100.stdout:0/675: write dd/d2b/d95/d79/d7d/f7e [12622,4856] 0 2024-03-18T14:51:09.876 INFO:tasks.workunit.client.0.smithi012.stdout:1/861: dread d8/db/d10/f34 [0,4194304] 0 2024-03-18T14:51:09.878 INFO:tasks.workunit.client.0.smithi012.stdout:1/862: write d8/db/d10/d31/d43/d5e/fbc [35121,29557] 0 2024-03-18T14:51:09.878 INFO:tasks.workunit.client.0.smithi012.stdout:1/863: readlink d8/db/d10/d31/l78 0 2024-03-18T14:51:09.878 INFO:tasks.workunit.client.0.smithi012.stdout:7/941: getdents dd/d13 0 2024-03-18T14:51:09.881 INFO:tasks.workunit.client.0.smithi012.stdout:1/864: rename d8/db/d10/d31/d43/d5e/d65/d70/fcb to d8/d1e/d27/d44/dc2/f113 0 2024-03-18T14:51:09.882 INFO:tasks.workunit.client.0.smithi012.stdout:6/854: dwrite f16 [4194304,4194304] 0 2024-03-18T14:51:09.884 INFO:tasks.workunit.client.1.smithi100.stdout:1/611: mknod d8/d1e/d11/d27/d73/d89/cb8 0 2024-03-18T14:51:09.884 INFO:tasks.workunit.client.1.smithi100.stdout:1/612: dread - d8/d2a/d37/d50/f85 zero size 2024-03-18T14:51:09.886 INFO:tasks.workunit.client.1.smithi100.stdout:9/643: creat da/d51/d55/fcc x:0 0 0 2024-03-18T14:51:09.887 INFO:tasks.workunit.client.1.smithi100.stdout:4/611: dread d9/d13/d25/f48 [0,4194304] 0 2024-03-18T14:51:09.887 INFO:tasks.workunit.client.1.smithi100.stdout:4/612: readlink d9/d13/d39/d4c/l75 0 2024-03-18T14:51:09.887 INFO:tasks.workunit.client.1.smithi100.stdout:4/613: fsync d9/f11 0 2024-03-18T14:51:09.887 INFO:tasks.workunit.client.1.smithi100.stdout:4/614: chown d9/d27/d62/d7b/d82/c91 1 1 2024-03-18T14:51:09.887 INFO:tasks.workunit.client.1.smithi100.stdout:7/637: write d5/f72 [3458566,70630] 0 2024-03-18T14:51:09.889 INFO:tasks.workunit.client.0.smithi012.stdout:0/936: dwrite d1/d20/d23/d88/f125 [0,4194304] 0 2024-03-18T14:51:09.889 INFO:tasks.workunit.client.0.smithi012.stdout:0/937: chown d1/d20/d23/d3b/d33/d51/fae 920084 1 2024-03-18T14:51:09.889 INFO:tasks.workunit.client.0.smithi012.stdout:0/938: fdatasync d1/faa 0 2024-03-18T14:51:09.890 INFO:tasks.workunit.client.0.smithi012.stdout:0/939: write d1/d16/d19/f12a [197413,48953] 0 2024-03-18T14:51:09.893 INFO:tasks.workunit.client.0.smithi012.stdout:6/855: creat d17/d19/ffd x:0 0 0 2024-03-18T14:51:09.894 INFO:tasks.workunit.client.1.smithi100.stdout:6/598: mkdir d6/d7/df/d26/d4e/db5 0 2024-03-18T14:51:09.895 INFO:tasks.workunit.client.0.smithi012.stdout:6/856: dread d17/d38/d51/d66/fcf [0,4194304] 0 2024-03-18T14:51:09.896 INFO:tasks.workunit.client.0.smithi012.stdout:6/857: fdatasync f11 0 2024-03-18T14:51:09.896 INFO:tasks.workunit.client.0.smithi012.stdout:7/942: write dd/d13/d23/d76/f102 [1959212,9936] 0 2024-03-18T14:51:09.896 INFO:tasks.workunit.client.0.smithi012.stdout:4/892: dwrite d6/d19/d6c/f7a [0,4194304] 0 2024-03-18T14:51:09.902 INFO:tasks.workunit.client.0.smithi012.stdout:1/865: dwrite d8/dbf/ff0 [4194304,4194304] 0 2024-03-18T14:51:09.905 INFO:tasks.workunit.client.1.smithi100.stdout:6/599: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f79 [0,4194304] 0 2024-03-18T14:51:09.905 INFO:tasks.workunit.client.1.smithi100.stdout:8/681: creat d4/dc/d31/d85/dd6/ff1 x:0 0 0 2024-03-18T14:51:09.908 INFO:tasks.workunit.client.1.smithi100.stdout:5/610: getdents d29/d2d/d39 0 2024-03-18T14:51:09.911 INFO:tasks.workunit.client.0.smithi012.stdout:6/858: rename d17/d18/c1c to d17/d38/d5b/cfe 0 2024-03-18T14:51:09.912 INFO:tasks.workunit.client.0.smithi012.stdout:6/859: dread d17/d18/f23 [0,4194304] 0 2024-03-18T14:51:09.916 INFO:tasks.workunit.client.0.smithi012.stdout:0/940: dwrite d1/d20/d7b/ff [0,4194304] 0 2024-03-18T14:51:09.917 INFO:tasks.workunit.client.1.smithi100.stdout:0/676: symlink dd/d31/d91/d56/d62/lf1 0 2024-03-18T14:51:09.923 INFO:tasks.workunit.client.0.smithi012.stdout:6/860: creat d17/d38/d51/d66/d7d/fff x:0 0 0 2024-03-18T14:51:09.923 INFO:tasks.workunit.client.0.smithi012.stdout:6/861: write d17/d26/d36/d4d/d82/fb0 [34193,78561] 0 2024-03-18T14:51:09.923 INFO:tasks.workunit.client.0.smithi012.stdout:6/862: readlink d17/d38/d51/dcc/dd8/lb6 0 2024-03-18T14:51:09.929 INFO:tasks.workunit.client.0.smithi012.stdout:6/863: rename d17/d38/d51/dcc to d17/d38/d51/da8/d100 0 2024-03-18T14:51:09.930 INFO:tasks.workunit.client.1.smithi100.stdout:4/615: dwrite d9/d13/d25/f4d [0,4194304] 0 2024-03-18T14:51:09.930 INFO:tasks.workunit.client.1.smithi100.stdout:3/651: read d3/dd/d21/d3e/f7e [4035540,26753] 0 2024-03-18T14:51:09.930 INFO:tasks.workunit.client.1.smithi100.stdout:3/652: fsync d3/dd/d49/f70 0 2024-03-18T14:51:09.931 INFO:tasks.workunit.client.1.smithi100.stdout:0/677: dread dd/d2b/d95/d79/fc7 [0,4194304] 0 2024-03-18T14:51:09.931 INFO:tasks.workunit.client.1.smithi100.stdout:1/613: rmdir d8/d1e/d11/d6d/d59 39 2024-03-18T14:51:09.931 INFO:tasks.workunit.client.1.smithi100.stdout:1/614: fdatasync d8/d2a/fa9 0 2024-03-18T14:51:09.932 INFO:tasks.workunit.client.1.smithi100.stdout:1/615: fdatasync d8/d1e/d11/d27/fa7 0 2024-03-18T14:51:09.933 INFO:tasks.workunit.client.0.smithi012.stdout:6/864: truncate f16 3036433 0 2024-03-18T14:51:09.934 INFO:tasks.workunit.client.1.smithi100.stdout:9/644: mkdir da/dd/d29/d9e/dcd 0 2024-03-18T14:51:09.937 INFO:tasks.workunit.client.0.smithi012.stdout:6/865: creat d17/d38/d51/df7/f101 x:0 0 0 2024-03-18T14:51:09.937 INFO:tasks.workunit.client.0.smithi012.stdout:6/866: truncate d17/d38/d51/fbf 1530877 0 2024-03-18T14:51:09.940 INFO:tasks.workunit.client.1.smithi100.stdout:8/682: rename d4/dc/d1f/d59/fbf to d4/dc/d31/d85/d8e/ff2 0 2024-03-18T14:51:09.941 INFO:tasks.workunit.client.1.smithi100.stdout:8/683: fsync d4/dc/dd/f44 0 2024-03-18T14:51:09.944 INFO:tasks.workunit.client.1.smithi100.stdout:5/611: symlink d29/d2d/lc4 0 2024-03-18T14:51:09.946 INFO:tasks.workunit.client.1.smithi100.stdout:3/653: symlink d3/d16/d13/d17/dab/le4 0 2024-03-18T14:51:09.946 INFO:tasks.workunit.client.1.smithi100.stdout:3/654: creat d3/d16/d13/d17/fe5 x:0 0 0 2024-03-18T14:51:09.947 INFO:tasks.workunit.client.0.smithi012.stdout:7/943: dwrite dd/d13/d23/d60/d140/f114 [0,4194304] 0 2024-03-18T14:51:09.947 INFO:tasks.workunit.client.0.smithi012.stdout:1/866: dwrite d8/d1e/d27/d44/f97 [0,4194304] 0 2024-03-18T14:51:09.947 INFO:tasks.workunit.client.0.smithi012.stdout:0/941: dwrite d1/d16/d19/dd0/d129/d67/f12d [0,4194304] 0 2024-03-18T14:51:09.947 INFO:tasks.workunit.client.0.smithi012.stdout:0/942: chown d1/d16/d19/dd0/d129/d67/d10c/c115 91097 1 2024-03-18T14:51:09.947 INFO:tasks.workunit.client.0.smithi012.stdout:0/943: truncate d1/d20/d23/d3b/d33/f122 4900653 0 2024-03-18T14:51:09.948 INFO:tasks.workunit.client.1.smithi100.stdout:5/612: read d29/f50 [294606,56362] 0 2024-03-18T14:51:09.949 INFO:tasks.workunit.client.1.smithi100.stdout:0/678: creat dd/d31/d91/d56/d59/ff2 x:0 0 0 2024-03-18T14:51:09.952 INFO:tasks.workunit.client.1.smithi100.stdout:1/616: truncate d8/d1e/f45 1084729 0 2024-03-18T14:51:09.953 INFO:tasks.workunit.client.1.smithi100.stdout:6/600: read d6/d7/df/f55 [1651461,91973] 0 2024-03-18T14:51:09.955 INFO:tasks.workunit.client.0.smithi012.stdout:6/867: write d17/d26/fb3 [3527399,78927] 0 2024-03-18T14:51:09.956 INFO:tasks.workunit.client.1.smithi100.stdout:4/616: rename d9/f1f to d9/d13/d5a/fbc 0 2024-03-18T14:51:09.956 INFO:tasks.workunit.client.1.smithi100.stdout:4/617: chown d9/d27/f36 117394 1 2024-03-18T14:51:09.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/624: sync 2024-03-18T14:51:09.958 INFO:tasks.workunit.client.1.smithi100.stdout:2/625: fsync d3/da/de/f3c 0 2024-03-18T14:51:09.958 INFO:tasks.workunit.client.1.smithi100.stdout:4/618: dread d9/f11 [0,4194304] 0 2024-03-18T14:51:09.958 INFO:tasks.workunit.client.1.smithi100.stdout:4/619: truncate d9/d27/d62/f6c 4874982 0 2024-03-18T14:51:09.959 INFO:tasks.workunit.client.1.smithi100.stdout:0/679: write dd/d2b/f40 [427967,113422] 0 2024-03-18T14:51:09.960 INFO:tasks.workunit.client.1.smithi100.stdout:8/684: symlink d4/dc/dd/de/d21/d23/d25/lf3 0 2024-03-18T14:51:09.965 INFO:tasks.workunit.client.0.smithi012.stdout:7/944: mkdir dd/d13/d23/d60/d142 0 2024-03-18T14:51:09.965 INFO:tasks.workunit.client.0.smithi012.stdout:1/867: symlink d8/db/d10/d31/d43/d5e/da5/l114 0 2024-03-18T14:51:09.965 INFO:tasks.workunit.client.0.smithi012.stdout:1/868: chown d8/d1e/d27/ffa 1981009 1 2024-03-18T14:51:09.966 INFO:tasks.workunit.client.1.smithi100.stdout:3/655: mkdir d3/d16/d13/d1f/d3f/d4f/de0/de6 0 2024-03-18T14:51:09.968 INFO:tasks.workunit.client.1.smithi100.stdout:1/617: mknod d8/d2a/cb9 0 2024-03-18T14:51:09.969 INFO:tasks.workunit.client.1.smithi100.stdout:6/601: symlink d6/d7/df/d26/d4e/d53/lb6 0 2024-03-18T14:51:09.970 INFO:tasks.workunit.client.0.smithi012.stdout:0/944: getdents d1/d20/d23/d7f 0 2024-03-18T14:51:09.973 INFO:tasks.workunit.client.1.smithi100.stdout:9/645: rename da/dd/d29/d71/d72/c95 to da/d88/cce 0 2024-03-18T14:51:09.974 INFO:tasks.workunit.client.1.smithi100.stdout:9/646: dread da/dd/fa1 [0,4194304] 0 2024-03-18T14:51:09.976 INFO:tasks.workunit.client.1.smithi100.stdout:7/638: dwrite d5/d22/d24/d6e/d7a/f89 [0,4194304] 0 2024-03-18T14:51:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:7/945: read dd/d13/d23/d60/d140/dc7/fd8 [3997174,126986] 0 2024-03-18T14:51:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:7/946: read dd/d13/d23/d105/f127 [439221,23088] 0 2024-03-18T14:51:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:7/947: read - dd/d13/d23/d76/d6f/f133 zero size 2024-03-18T14:51:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:4/893: dwrite d6/da/d58/fbd [0,4194304] 0 2024-03-18T14:51:09.977 INFO:tasks.workunit.client.0.smithi012.stdout:6/868: mknod d17/d26/d36/d4d/d82/c102 0 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.1.smithi100.stdout:3/656: dread d3/d16/d13/d17/f61 [0,4194304] 0 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.1.smithi100.stdout:3/657: chown d3/d16/d13/d17/c4d 0 1 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/626: creat d3/da/de/d20/d82/fd8 x:0 0 0 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/627: readlink d3/d13/d40/d53/ld0 0 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/628: fsync d3/d13/d40/fa7 0 2024-03-18T14:51:09.979 INFO:tasks.workunit.client.0.smithi012.stdout:1/869: truncate d8/db/d10/d31/d43/d5e/d65/d84/fde 1213758 0 2024-03-18T14:51:09.981 INFO:tasks.workunit.client.1.smithi100.stdout:4/620: link d9/d13/d39/d4c/l85 d9/d27/d62/d7b/d82/lbd 0 2024-03-18T14:51:09.984 INFO:tasks.workunit.client.1.smithi100.stdout:7/639: write d5/d14/d16/d2e/d62/d66/f54 [2679065,50006] 0 2024-03-18T14:51:09.984 INFO:tasks.workunit.client.1.smithi100.stdout:7/640: stat d5/d22/d3b/c56 0 2024-03-18T14:51:09.984 INFO:tasks.workunit.client.1.smithi100.stdout:5/613: dwrite d29/d2d/d39/d65/fa8 [0,4194304] 0 2024-03-18T14:51:09.984 INFO:tasks.workunit.client.0.smithi012.stdout:0/945: symlink d1/d20/d23/d6f/d93/l133 0 2024-03-18T14:51:09.986 INFO:tasks.workunit.client.0.smithi012.stdout:7/948: dread dd/d13/d23/d2f/f9f [0,4194304] 0 2024-03-18T14:51:09.988 INFO:tasks.workunit.client.1.smithi100.stdout:0/680: symlink dd/d31/d91/d56/d62/d6e/ddd/lf3 0 2024-03-18T14:51:09.990 INFO:tasks.workunit.client.1.smithi100.stdout:1/618: getdents d8/d2a/d37/d50/d6f/d81/d91 0 2024-03-18T14:51:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:7/949: dread dd/d13/d23/d60/d140/dc7/fd8 [0,4194304] 0 2024-03-18T14:51:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:7/950: read dd/d13/d23/d76/d6b/f100 [170526,124152] 0 2024-03-18T14:51:09.993 INFO:tasks.workunit.client.0.smithi012.stdout:7/951: fdatasync dd/d13/d23/d2f/d53/dc6/ff9 0 2024-03-18T14:51:09.994 INFO:tasks.workunit.client.0.smithi012.stdout:4/894: mknod d6/da/d89/d94/d10d/d11d/c137 0 2024-03-18T14:51:09.998 INFO:tasks.workunit.client.1.smithi100.stdout:6/602: rename d6/d7/df/d26/d4e/d53/lb6 to d6/d7/df/d26/d4e/d66/lb7 0 2024-03-18T14:51:09.999 INFO:tasks.workunit.client.1.smithi100.stdout:7/641: dwrite d5/d14/d16/d2e/d62/dd1/d86/d85/d87/f88 [0,4194304] 0 2024-03-18T14:51:09.999 INFO:tasks.workunit.client.1.smithi100.stdout:7/642: chown d5/d14/c1b 1830687564 1 2024-03-18T14:51:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:6/869: mkdir d17/d38/d51/d66/dee/d103 0 2024-03-18T14:51:10.000 INFO:tasks.workunit.client.0.smithi012.stdout:6/870: fsync d17/d38/d51/da8/d100/dd8/f48 0 2024-03-18T14:51:10.002 INFO:tasks.workunit.client.1.smithi100.stdout:9/647: symlink da/dd/d29/d71/lcf 0 2024-03-18T14:51:10.002 INFO:tasks.workunit.client.1.smithi100.stdout:9/648: fsync da/d46/d8b/f66 0 2024-03-18T14:51:10.002 INFO:tasks.workunit.client.1.smithi100.stdout:9/649: truncate da/dd/d29/d71/d72/f9b 995787 0 2024-03-18T14:51:10.002 INFO:tasks.workunit.client.1.smithi100.stdout:9/650: truncate da/d51/d55/f5c 617025 0 2024-03-18T14:51:10.003 INFO:tasks.workunit.client.1.smithi100.stdout:9/651: chown da/d46/d8b/f35 14408955 1 2024-03-18T14:51:10.003 INFO:tasks.workunit.client.1.smithi100.stdout:9/652: creat da/d46/d4d/d62/fd0 x:0 0 0 2024-03-18T14:51:10.003 INFO:tasks.workunit.client.1.smithi100.stdout:9/653: fsync da/d51/d55/d7e/f9d 0 2024-03-18T14:51:10.004 INFO:tasks.workunit.client.0.smithi012.stdout:1/870: creat d8/db/d10/d16/d6b/f115 x:0 0 0 2024-03-18T14:51:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:0/946: link d1/d20/d23/d26/f36 d1/d16/d19/dd0/d129/d8a/f134 0 2024-03-18T14:51:10.007 INFO:tasks.workunit.client.0.smithi012.stdout:0/947: creat d1/d20/d23/d3b/d33/d51/f135 x:0 0 0 2024-03-18T14:51:10.008 INFO:tasks.workunit.client.0.smithi012.stdout:0/948: write d1/d16/d19/dd0/d129/d67/ff4 [1072768,71155] 0 2024-03-18T14:51:10.010 INFO:tasks.workunit.client.0.smithi012.stdout:7/952: getdents dd/d13/d23 0 2024-03-18T14:51:10.013 INFO:tasks.workunit.client.0.smithi012.stdout:1/871: link d8/db/f30 d8/f116 0 2024-03-18T14:51:10.016 INFO:tasks.workunit.client.1.smithi100.stdout:2/629: stat d3/da/de/c31 0 2024-03-18T14:51:10.016 INFO:tasks.workunit.client.1.smithi100.stdout:2/630: chown d3/d6/d5f/c6b 3964085 1 2024-03-18T14:51:10.017 INFO:tasks.workunit.client.1.smithi100.stdout:2/631: write d3/d6/d5f/f68 [445933,96721] 0 2024-03-18T14:51:10.017 INFO:tasks.workunit.client.0.smithi012.stdout:7/953: creat dd/d13/d23/d76/d109/d129/dd5/f143 x:0 0 0 2024-03-18T14:51:10.018 INFO:tasks.workunit.client.1.smithi100.stdout:4/621: mknod d9/d3a/cbe 0 2024-03-18T14:51:10.019 INFO:tasks.workunit.client.1.smithi100.stdout:4/622: write d9/f1b [921716,5163] 0 2024-03-18T14:51:10.023 INFO:tasks.workunit.client.0.smithi012.stdout:0/949: dread d1/d20/d23/d3b/d33/f122 [0,4194304] 0 2024-03-18T14:51:10.025 INFO:tasks.workunit.client.0.smithi012.stdout:1/872: creat d8/f117 x:0 0 0 2024-03-18T14:51:10.026 INFO:tasks.workunit.client.0.smithi012.stdout:1/873: stat d8/db/d10/d31/d81/fd3 0 2024-03-18T14:51:10.026 INFO:tasks.workunit.client.1.smithi100.stdout:6/603: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [0,4194304] 0 2024-03-18T14:51:10.026 INFO:tasks.workunit.client.0.smithi012.stdout:4/895: dwrite d6/d19/f10c [0,4194304] 0 2024-03-18T14:51:10.028 INFO:tasks.workunit.client.0.smithi012.stdout:0/950: dread d1/d16/f17 [0,4194304] 0 2024-03-18T14:51:10.028 INFO:tasks.workunit.client.0.smithi012.stdout:0/951: readlink d1/d16/d19/l34 0 2024-03-18T14:51:10.033 INFO:tasks.workunit.client.0.smithi012.stdout:7/954: dread dd/d13/d23/fce [0,4194304] 0 2024-03-18T14:51:10.033 INFO:tasks.workunit.client.0.smithi012.stdout:6/871: dwrite d17/d19/f2b [0,4194304] 0 2024-03-18T14:51:10.033 INFO:tasks.workunit.client.0.smithi012.stdout:7/955: write dd/d13/d23/d76/d6b/f11b [12298,11468] 0 2024-03-18T14:51:10.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/681: mknod dd/d31/d91/d56/d62/cf4 0 2024-03-18T14:51:10.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/682: truncate dd/d31/d91/d56/d62/f84 1313277 0 2024-03-18T14:51:10.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/683: write dd/d31/d91/d7b/fad [5075708,85695] 0 2024-03-18T14:51:10.060 INFO:tasks.workunit.client.0.smithi012.stdout:4/896: rename d6/d1a/d30/d46/d6f/la6 to d6/da/d89/d94/d10d/d11d/d122/l138 0 2024-03-18T14:51:10.063 INFO:tasks.workunit.client.1.smithi100.stdout:1/619: unlink d8/d1e/l13 0 2024-03-18T14:51:10.064 INFO:tasks.workunit.client.1.smithi100.stdout:1/620: fdatasync d8/d1e/f45 0 2024-03-18T14:51:10.064 INFO:tasks.workunit.client.1.smithi100.stdout:1/621: write d8/f29 [4377881,84799] 0 2024-03-18T14:51:10.065 INFO:tasks.workunit.client.1.smithi100.stdout:5/614: rename d29/d2d/d39/c44 to d29/d2d/d32/db2/cc5 0 2024-03-18T14:51:10.070 INFO:tasks.workunit.client.0.smithi012.stdout:0/952: dwrite d1/d20/d23/d88/f125 [0,4194304] 0 2024-03-18T14:51:10.071 INFO:tasks.workunit.client.0.smithi012.stdout:1/874: dwrite d8/d1e/d27/d44/f4f [0,4194304] 0 2024-03-18T14:51:10.071 INFO:tasks.workunit.client.0.smithi012.stdout:1/875: write d8/db/d10/d31/d43/d5e/da5/f111 [174552,17052] 0 2024-03-18T14:51:10.071 INFO:tasks.workunit.client.0.smithi012.stdout:1/876: readlink d8/dbf/le1 0 2024-03-18T14:51:10.072 INFO:tasks.workunit.client.1.smithi100.stdout:7/643: mkdir d5/d22/d3b/df5 0 2024-03-18T14:51:10.075 INFO:tasks.workunit.client.1.smithi100.stdout:9/654: mknod da/dd/d3b/cd1 0 2024-03-18T14:51:10.077 INFO:tasks.workunit.client.1.smithi100.stdout:3/658: getdents d3/da5 0 2024-03-18T14:51:10.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/632: creat d3/d6/d5f/d63/fd9 x:0 0 0 2024-03-18T14:51:10.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/633: stat d3/da/de/d20/f90 0 2024-03-18T14:51:10.079 INFO:tasks.workunit.client.1.smithi100.stdout:2/634: chown d3/d6/d29/d36/d49/c58 74563977 1 2024-03-18T14:51:10.080 INFO:tasks.workunit.client.1.smithi100.stdout:0/684: dwrite dd/f1f [0,4194304] 0 2024-03-18T14:51:10.081 INFO:tasks.workunit.client.1.smithi100.stdout:0/685: creat dd/d31/d91/d65/ff5 x:0 0 0 2024-03-18T14:51:10.087 INFO:tasks.workunit.client.0.smithi012.stdout:0/953: dread d1/faa [0,4194304] 0 2024-03-18T14:51:10.089 INFO:tasks.workunit.client.1.smithi100.stdout:6/604: mkdir d6/d7/df/d26/d4e/d66/d36/d4f/db8 0 2024-03-18T14:51:10.095 INFO:tasks.workunit.client.0.smithi012.stdout:7/956: mkdir dd/d13/d23/d76/d109/d129/d144 0 2024-03-18T14:51:10.095 INFO:tasks.workunit.client.0.smithi012.stdout:7/957: chown dd/d13/d23/d2f/d7a/db6/l7c 9 1 2024-03-18T14:51:10.097 INFO:tasks.workunit.client.0.smithi012.stdout:4/897: mknod d6/da/d58/dd2/c139 0 2024-03-18T14:51:10.098 INFO:tasks.workunit.client.0.smithi012.stdout:1/877: dwrite d8/db/d10/d31/d43/ff3 [0,4194304] 0 2024-03-18T14:51:10.099 INFO:tasks.workunit.client.0.smithi012.stdout:0/954: write d1/d20/d23/d3b/d33/d51/fa8 [3948431,31992] 0 2024-03-18T14:51:10.099 INFO:tasks.workunit.client.0.smithi012.stdout:0/955: chown d1/d20/d7b/l10d 0 1 2024-03-18T14:51:10.099 INFO:tasks.workunit.client.1.smithi100.stdout:1/622: creat d8/d1e/d11/d6d/d59/d93/fba x:0 0 0 2024-03-18T14:51:10.100 INFO:tasks.workunit.client.1.smithi100.stdout:9/655: dwrite da/d88/da9/fc7 [0,4194304] 0 2024-03-18T14:51:10.101 INFO:tasks.workunit.client.1.smithi100.stdout:6/605: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [0,4194304] 0 2024-03-18T14:51:10.101 INFO:tasks.workunit.client.1.smithi100.stdout:6/606: creat d6/d12/d78/fb9 x:0 0 0 2024-03-18T14:51:10.102 INFO:tasks.workunit.client.0.smithi012.stdout:6/872: mknod d17/d38/c104 0 2024-03-18T14:51:10.103 INFO:tasks.workunit.client.1.smithi100.stdout:5/615: symlink d29/d2d/d32/d92/lc6 0 2024-03-18T14:51:10.103 INFO:tasks.workunit.client.1.smithi100.stdout:5/616: write d29/d2d/d32/f68 [4844763,80152] 0 2024-03-18T14:51:10.104 INFO:tasks.workunit.client.0.smithi012.stdout:7/958: mknod dd/d13/d23/d60/d140/dc7/c145 0 2024-03-18T14:51:10.107 INFO:tasks.workunit.client.0.smithi012.stdout:4/898: symlink d6/da/d89/d94/l13a 0 2024-03-18T14:51:10.109 INFO:tasks.workunit.client.1.smithi100.stdout:3/659: symlink d3/d16/d13/d1f/d3f/d4f/d65/le7 0 2024-03-18T14:51:10.113 INFO:tasks.workunit.client.1.smithi100.stdout:2/635: creat d3/d6/d29/fda x:0 0 0 2024-03-18T14:51:10.113 INFO:tasks.workunit.client.1.smithi100.stdout:2/636: write d3/d6/d29/d36/d49/f51 [1035131,62221] 0 2024-03-18T14:51:10.113 INFO:tasks.workunit.client.1.smithi100.stdout:2/637: dread - d3/d6/d5f/d6a/fc8 zero size 2024-03-18T14:51:10.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/878: unlink d8/db/d10/d31/d43/ff3 0 2024-03-18T14:51:10.114 INFO:tasks.workunit.client.0.smithi012.stdout:1/879: write d8/db/d10/d16/da1/d2e/d9c/fea [1253265,89894] 0 2024-03-18T14:51:10.122 INFO:tasks.workunit.client.0.smithi012.stdout:4/899: mkdir d6/d1a/d30/d46/d6f/d8b/d13b 0 2024-03-18T14:51:10.128 INFO:tasks.workunit.client.0.smithi012.stdout:1/880: rename d8/db/d10/d31/d43/ld8 to d8/db/d10/d16/da1/d2e/d9c/d9d/l118 0 2024-03-18T14:51:10.131 INFO:tasks.workunit.client.0.smithi012.stdout:7/959: rmdir dd/d13/d23/d2f/de1 39 2024-03-18T14:51:10.131 INFO:tasks.workunit.client.0.smithi012.stdout:7/960: fdatasync dd/d13/d23/d2f/d7a/dab/fbe 0 2024-03-18T14:51:10.136 INFO:tasks.workunit.client.0.smithi012.stdout:1/881: link d8/db/d10/d31/d43/d5e/d65/d84/dc0/lec d8/db/d10/d31/l119 0 2024-03-18T14:51:10.137 INFO:tasks.workunit.client.0.smithi012.stdout:0/956: dwrite d1/d16/d19/dd0/d129/d67/f12d [0,4194304] 0 2024-03-18T14:51:10.137 INFO:tasks.workunit.client.0.smithi012.stdout:6/873: dwrite d17/d26/d36/d4d/fba [0,4194304] 0 2024-03-18T14:51:10.139 INFO:tasks.workunit.client.1.smithi100.stdout:3/660: dwrite d3/d16/d13/d1f/f90 [0,4194304] 0 2024-03-18T14:51:10.139 INFO:tasks.workunit.client.1.smithi100.stdout:2/638: dwrite d3/d13/d40/d53/f85 [0,4194304] 0 2024-03-18T14:51:10.139 INFO:tasks.workunit.client.1.smithi100.stdout:2/639: readlink d3/d1a/l4d 0 2024-03-18T14:51:10.139 INFO:tasks.workunit.client.1.smithi100.stdout:9/656: creat da/d51/d55/d7e/fd2 x:0 0 0 2024-03-18T14:51:10.141 INFO:tasks.workunit.client.0.smithi012.stdout:7/961: creat dd/d13/d103/f146 x:0 0 0 2024-03-18T14:51:10.143 INFO:tasks.workunit.client.0.smithi012.stdout:4/900: rmdir d6/d1a/d30/d46/d6f/d7c 39 2024-03-18T14:51:10.145 INFO:tasks.workunit.client.1.smithi100.stdout:6/607: mkdir d6/d7/df/d26/d4e/db5/dba 0 2024-03-18T14:51:10.146 INFO:tasks.workunit.client.1.smithi100.stdout:6/608: chown d6/d12/l4d 189214641 1 2024-03-18T14:51:10.147 INFO:tasks.workunit.client.0.smithi012.stdout:1/882: link d8/db/d10/d16/da1/d2e/d9c/fb1 d8/db/d10/d31/d81/f11a 0 2024-03-18T14:51:10.148 INFO:tasks.workunit.client.0.smithi012.stdout:0/957: truncate d1/d16/d31/f1d 994744 0 2024-03-18T14:51:10.156 INFO:tasks.workunit.client.0.smithi012.stdout:6/874: dwrite d17/d19/f2b [4194304,4194304] 0 2024-03-18T14:51:10.159 INFO:tasks.workunit.client.1.smithi100.stdout:1/623: symlink d8/d1e/d11/d27/d3e/lbb 0 2024-03-18T14:51:10.159 INFO:tasks.workunit.client.1.smithi100.stdout:1/624: dread - d8/d1e/d11/d8a/fa8 zero size 2024-03-18T14:51:10.159 INFO:tasks.workunit.client.0.smithi012.stdout:7/962: mkdir dd/d13/d23/d76/d6f/d147 0 2024-03-18T14:51:10.166 INFO:tasks.workunit.client.1.smithi100.stdout:3/661: creat d3/da5/fe8 x:0 0 0 2024-03-18T14:51:10.167 INFO:tasks.workunit.client.0.smithi012.stdout:4/901: getdents d6/da/d89/da1/dbb/ddc/de5 0 2024-03-18T14:51:10.167 INFO:tasks.workunit.client.1.smithi100.stdout:6/609: dwrite d6/d7/f8d [0,4194304] 0 2024-03-18T14:51:10.173 INFO:tasks.workunit.client.1.smithi100.stdout:9/657: creat da/dcb/fd3 x:0 0 0 2024-03-18T14:51:10.174 INFO:tasks.workunit.client.0.smithi012.stdout:1/883: chown d8/db/d10/d56/l87 1305 1 2024-03-18T14:51:10.176 INFO:tasks.workunit.client.0.smithi012.stdout:4/902: dread d6/f25 [0,4194304] 0 2024-03-18T14:51:10.176 INFO:tasks.workunit.client.0.smithi012.stdout:4/903: write d6/d1a/d98/daf/f136 [4878748,4660] 0 2024-03-18T14:51:10.177 INFO:tasks.workunit.client.0.smithi012.stdout:7/963: read dd/d13/d23/d60/d140/ff3 [723147,61539] 0 2024-03-18T14:51:10.179 INFO:tasks.workunit.client.1.smithi100.stdout:7/644: rename d5/d14/d16/d2e/d7b to d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6 0 2024-03-18T14:51:10.180 INFO:tasks.workunit.client.1.smithi100.stdout:7/645: read d5/d14/d16/d2e/d62/f8c [216061,114579] 0 2024-03-18T14:51:10.180 INFO:tasks.workunit.client.1.smithi100.stdout:9/658: write da/dd/d29/d71/f82 [3214375,85031] 0 2024-03-18T14:51:10.180 INFO:tasks.workunit.client.1.smithi100.stdout:9/659: readlink da/dd/d3c/d4b/l6c 0 2024-03-18T14:51:10.184 INFO:tasks.workunit.client.1.smithi100.stdout:1/625: dread d8/d30/d61/f63 [0,4194304] 0 2024-03-18T14:51:10.184 INFO:tasks.workunit.client.0.smithi012.stdout:0/958: mknod d1/d20/d23/d6f/d93/c136 0 2024-03-18T14:51:10.185 INFO:tasks.workunit.client.1.smithi100.stdout:3/662: truncate d3/d16/d13/d17/f61 3690379 0 2024-03-18T14:51:10.188 INFO:tasks.workunit.client.1.smithi100.stdout:2/640: rmdir d3/d6/d5f/d6a 39 2024-03-18T14:51:10.188 INFO:tasks.workunit.client.1.smithi100.stdout:2/641: stat d3/da/f3a 0 2024-03-18T14:51:10.188 INFO:tasks.workunit.client.1.smithi100.stdout:2/642: fsync d3/d6/d29/f92 0 2024-03-18T14:51:10.193 INFO:tasks.workunit.client.1.smithi100.stdout:0/686: rename dd/fc9 to dd/d20/db5/ff6 0 2024-03-18T14:51:10.201 INFO:tasks.workunit.client.1.smithi100.stdout:8/685: sync 2024-03-18T14:51:10.201 INFO:tasks.workunit.client.1.smithi100.stdout:4/623: sync 2024-03-18T14:51:10.201 INFO:tasks.workunit.client.1.smithi100.stdout:9/660: mkdir da/dd/d3c/d4b/dd4 0 2024-03-18T14:51:10.201 INFO:tasks.workunit.client.1.smithi100.stdout:9/661: truncate da/dd/f73 716877 0 2024-03-18T14:51:10.202 INFO:tasks.workunit.client.0.smithi012.stdout:6/875: mkdir d17/d26/d36/d105 0 2024-03-18T14:51:10.204 INFO:tasks.workunit.client.0.smithi012.stdout:4/904: mkdir d6/da/d89/d95/d13c 0 2024-03-18T14:51:10.206 INFO:tasks.workunit.client.1.smithi100.stdout:3/663: mkdir d3/d16/d13/d1f/d3f/d4f/d65/de9 0 2024-03-18T14:51:10.206 INFO:tasks.workunit.client.1.smithi100.stdout:3/664: fsync d3/dd/d21/d3e/f32 0 2024-03-18T14:51:10.208 INFO:tasks.workunit.client.0.smithi012.stdout:7/964: unlink dd/d13/d23/d2f/c11f 0 2024-03-18T14:51:10.208 INFO:tasks.workunit.client.0.smithi012.stdout:7/965: truncate dd/d13/d103/f141 670598 0 2024-03-18T14:51:10.210 INFO:tasks.workunit.client.0.smithi012.stdout:1/884: dwrite d8/db/d10/d16/da1/d2e/f100 [0,4194304] 0 2024-03-18T14:51:10.211 INFO:tasks.workunit.client.1.smithi100.stdout:2/643: dwrite d3/d6/d29/f92 [0,4194304] 0 2024-03-18T14:51:10.211 INFO:tasks.workunit.client.1.smithi100.stdout:6/610: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d63/fab [0,4194304] 0 2024-03-18T14:51:10.212 INFO:tasks.workunit.client.1.smithi100.stdout:5/617: rename d29/d2d/d39/d65/l7b to d29/d2d/d39/d74/lc7 0 2024-03-18T14:51:10.212 INFO:tasks.workunit.client.1.smithi100.stdout:5/618: chown f15 1328 1 2024-03-18T14:51:10.212 INFO:tasks.workunit.client.1.smithi100.stdout:5/619: fdatasync d29/d2d/d39/d71/fb3 0 2024-03-18T14:51:10.213 INFO:tasks.workunit.client.0.smithi012.stdout:4/905: write d6/d19/f10c [2997224,103370] 0 2024-03-18T14:51:10.214 INFO:tasks.workunit.client.0.smithi012.stdout:0/959: symlink d1/d2d/l137 0 2024-03-18T14:51:10.215 INFO:tasks.workunit.client.1.smithi100.stdout:0/687: mknod dd/d2b/d95/dc1/cf7 0 2024-03-18T14:51:10.216 INFO:tasks.workunit.client.0.smithi012.stdout:0/960: dread d1/d16/d6e/dcc/f10f [0,4194304] 0 2024-03-18T14:51:10.217 INFO:tasks.workunit.client.0.smithi012.stdout:6/876: symlink d17/d19/d27/dea/l106 0 2024-03-18T14:51:10.217 INFO:tasks.workunit.client.0.smithi012.stdout:6/877: creat d17/d19/d27/d6e/f107 x:0 0 0 2024-03-18T14:51:10.219 INFO:tasks.workunit.client.1.smithi100.stdout:7/646: getdents d5/d22/d3b/d47 0 2024-03-18T14:51:10.219 INFO:tasks.workunit.client.1.smithi100.stdout:7/647: chown d5/d14/d16/d2e/d62/fdf 53 1 2024-03-18T14:51:10.221 INFO:tasks.workunit.client.0.smithi012.stdout:7/966: truncate dd/d13/d23/d76/f10a 919434 0 2024-03-18T14:51:10.222 INFO:tasks.workunit.client.1.smithi100.stdout:8/686: rmdir d4/dc/dd/de/d19 39 2024-03-18T14:51:10.224 INFO:tasks.workunit.client.1.smithi100.stdout:4/624: mknod d9/d13/d25/cbf 0 2024-03-18T14:51:10.224 INFO:tasks.workunit.client.1.smithi100.stdout:1/626: dwrite d8/d1e/d11/d27/d3f/f74 [0,4194304] 0 2024-03-18T14:51:10.225 INFO:tasks.workunit.client.1.smithi100.stdout:4/625: write d9/d13/d39/d76/f94 [179672,96805] 0 2024-03-18T14:51:10.225 INFO:tasks.workunit.client.1.smithi100.stdout:4/626: dread - d9/d13/d39/f74 zero size 2024-03-18T14:51:10.226 INFO:tasks.workunit.client.1.smithi100.stdout:9/662: unlink da/d2c/c49 0 2024-03-18T14:51:10.228 INFO:tasks.workunit.client.0.smithi012.stdout:4/906: dread d6/d19/f1d [0,4194304] 0 2024-03-18T14:51:10.229 INFO:tasks.workunit.client.1.smithi100.stdout:9/663: dread da/d46/d4d/d62/f93 [0,4194304] 0 2024-03-18T14:51:10.230 INFO:tasks.workunit.client.1.smithi100.stdout:3/665: creat d3/d16/d13/d71/d74/fea x:0 0 0 2024-03-18T14:51:10.231 INFO:tasks.workunit.client.1.smithi100.stdout:3/666: dread - d3/d16/d13/d17/d7d/fad zero size 2024-03-18T14:51:10.231 INFO:tasks.workunit.client.1.smithi100.stdout:3/667: creat d3/d16/d13/d1f/d3f/d4f/d65/feb x:0 0 0 2024-03-18T14:51:10.238 INFO:tasks.workunit.client.1.smithi100.stdout:1/627: write d8/d1e/d11/d8a/f97 [3473141,53272] 0 2024-03-18T14:51:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:09 smithi012 ceph-mon[29182]: mgrmap e25: smithi100.rwnrcn(active, since 11s), standbys: smithi012.tmscgt 2024-03-18T14:51:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:09 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:51:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:09 smithi012 ceph-mon[29182]: pgmap v9: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 232 MiB/s rd, 264 MiB/s wr, 411 op/s 2024-03-18T14:51:10.242 INFO:tasks.workunit.client.1.smithi100.stdout:2/644: link d3/d6/d29/d36/f9e d3/d6/d5f/d63/fdb 0 2024-03-18T14:51:10.244 INFO:tasks.workunit.client.1.smithi100.stdout:6/611: mkdir d6/d7/df/d26/d4e/d66/d36/dbb 0 2024-03-18T14:51:10.245 INFO:tasks.workunit.client.1.smithi100.stdout:6/612: chown d6/d7/df/d26/d4e/d66/lb7 9 1 2024-03-18T14:51:10.245 INFO:tasks.workunit.client.0.smithi012.stdout:4/907: dwrite d6/db6/fd9 [0,4194304] 0 2024-03-18T14:51:10.248 INFO:tasks.workunit.client.1.smithi100.stdout:0/688: rmdir dd/d31/d91/d56/d71/dcc 39 2024-03-18T14:51:10.249 INFO:tasks.workunit.client.0.smithi012.stdout:6/878: mknod d17/d38/d51/d72/c108 0 2024-03-18T14:51:10.249 INFO:tasks.workunit.client.0.smithi012.stdout:6/879: write d17/d19/d27/db1/dd7/fe1 [37452,111079] 0 2024-03-18T14:51:10.249 INFO:tasks.workunit.client.0.smithi012.stdout:6/880: chown d17/d19/d27/d6e/d84/ffa 1 1 2024-03-18T14:51:10.252 INFO:tasks.workunit.client.1.smithi100.stdout:8/687: link d4/dc/dd/de/d21/ca7 d4/dc2/cf4 0 2024-03-18T14:51:10.252 INFO:tasks.workunit.client.1.smithi100.stdout:6/613: dread d6/d7/df/d26/d4e/d66/d36/d4f/f73 [0,4194304] 0 2024-03-18T14:51:10.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/967: rename dd/l14 to dd/d13/d23/d2f/d53/dc6/l148 0 2024-03-18T14:51:10.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/968: chown f5 257 1 2024-03-18T14:51:10.254 INFO:tasks.workunit.client.0.smithi012.stdout:7/969: chown dd/d13/d23/d105 415 1 2024-03-18T14:51:10.257 INFO:tasks.workunit.client.0.smithi012.stdout:5/883: sync 2024-03-18T14:51:10.257 INFO:tasks.workunit.client.0.smithi012.stdout:7/970: dread dd/d13/d23/d2f/d7a/db6/f9b [0,4194304] 0 2024-03-18T14:51:10.257 INFO:tasks.workunit.client.0.smithi012.stdout:5/884: creat d3/d29/d36/d9b/f128 x:0 0 0 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/908: rmdir d6/d1a/d30/d46/d6f 39 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/909: chown d6/da/f91 1609 1 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/910: fdatasync d6/da/fd 0 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/911: fdatasync d6/d19/da5/fdd 0 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/912: write d6/d1a/d30/d46/d6f/d7c/f81 [1385106,122240] 0 2024-03-18T14:51:10.261 INFO:tasks.workunit.client.0.smithi012.stdout:4/913: chown d6/da/d89/d94/dbe/dd6/d12f/ff1 5303 1 2024-03-18T14:51:10.262 INFO:tasks.workunit.client.1.smithi100.stdout:9/664: truncate da/d51/f69 1408967 0 2024-03-18T14:51:10.264 INFO:tasks.workunit.client.0.smithi012.stdout:0/961: getdents d1/d20/d4e 0 2024-03-18T14:51:10.267 INFO:tasks.workunit.client.0.smithi012.stdout:6/881: getdents d17/d18/dc1 0 2024-03-18T14:51:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.271+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82d0105a00 con 0x7f82bc06b4b0 2024-03-18T14:51:10.272 INFO:tasks.workunit.client.0.smithi012.stdout:7/971: symlink dd/d13/d23/d76/d109/d129/d144/l149 0 2024-03-18T14:51:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.272+0000 7f82c5ffb700 1 -- 172.21.15.12:0/1611292139 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f82d0105a00 con 0x7f82bc06b4b0 2024-03-18T14:51:10.275 INFO:tasks.workunit.client.0.smithi012.stdout:5/885: mknod d3/d29/d36/d9b/ddf/def/c129 0 2024-03-18T14:51:10.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 msgr2=0x7f82bc06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 0x7f82bc06d960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82c0013830 tx=0x7f82c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 msgr2=0x7f82d0071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0041d0 tx=0x7f82cc00b040 comp rx=0 tx=0).stop 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 shutdown_connections 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f82bc06b4b0 0x7f82bc06d960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82d00fcf00 0x7f82d0071e40 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0041d0 tx=0x7f82cc00b040 comp rx=0 tx=0).stop 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 --2- 172.21.15.12:0/1611292139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82d00fe070 0x7f82d0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.277+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 >> 172.21.15.12:0/1611292139 conn(0x7f82d00f85a0 msgr2=0x7f82d00f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:10.279 INFO:tasks.workunit.client.1.smithi100.stdout:1/628: mknod d8/d30/d61/db4/cbc 0 2024-03-18T14:51:10.279 INFO:tasks.workunit.client.0.smithi012.stdout:4/914: truncate d6/da/f91 1058370 0 2024-03-18T14:51:10.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.278+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 shutdown_connections 2024-03-18T14:51:10.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.278+0000 7f82d7a2d700 1 -- 172.21.15.12:0/1611292139 wait complete. 2024-03-18T14:51:10.283 INFO:tasks.workunit.client.0.smithi012.stdout:6/882: symlink d17/d38/d51/d66/dee/l109 0 2024-03-18T14:51:10.283 INFO:tasks.workunit.client.0.smithi012.stdout:6/883: fdatasync d17/d19/d27/d58/d91/fca 0 2024-03-18T14:51:10.283 INFO:tasks.workunit.client.0.smithi012.stdout:6/884: write d17/d19/d27/d6e/d84/ffa [325234,98474] 0 2024-03-18T14:51:10.286 INFO:tasks.workunit.client.1.smithi100.stdout:1/629: dread d8/d1e/d11/f77 [4194304,4194304] 0 2024-03-18T14:51:10.287 INFO:tasks.workunit.client.0.smithi012.stdout:5/886: rename d3/d15/d42/dc6/d65/f6c to d3/de0/f12a 0 2024-03-18T14:51:10.288 INFO:tasks.workunit.client.0.smithi012.stdout:5/887: creat d3/d1e/f12b x:0 0 0 2024-03-18T14:51:10.288 INFO:tasks.workunit.client.0.smithi012.stdout:5/888: read d3/d29/d36/f45 [3019667,27275] 0 2024-03-18T14:51:10.288 INFO:tasks.workunit.client.0.smithi012.stdout:7/972: dread dd/d13/d23/d76/f3e [0,4194304] 0 2024-03-18T14:51:10.291 INFO:tasks.workunit.client.0.smithi012.stdout:0/962: getdents d1/d20/d23/d6f 0 2024-03-18T14:51:10.292 INFO:tasks.workunit.client.0.smithi012.stdout:0/963: write d1/d20/d23/d7f/f97 [1020110,99099] 0 2024-03-18T14:51:10.293 INFO:tasks.workunit.client.0.smithi012.stdout:5/889: dread d3/d1e/f1a [0,4194304] 0 2024-03-18T14:51:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:09 smithi100 ceph-mon[37807]: mgrmap e25: smithi100.rwnrcn(active, since 11s), standbys: smithi012.tmscgt 2024-03-18T14:51:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:09 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:51:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:09 smithi100 ceph-mon[37807]: pgmap v9: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 232 MiB/s rd, 264 MiB/s wr, 411 op/s 2024-03-18T14:51:10.296 INFO:tasks.workunit.client.0.smithi012.stdout:6/885: creat d17/d38/d51/da8/f10a x:0 0 0 2024-03-18T14:51:10.296 INFO:tasks.workunit.client.0.smithi012.stdout:6/886: fsync d17/d26/d36/d4d/fba 0 2024-03-18T14:51:10.302 INFO:tasks.workunit.client.1.smithi100.stdout:2/645: rename d3/da/f30 to d3/da/de/d20/d82/d95/fdc 0 2024-03-18T14:51:10.304 INFO:tasks.workunit.client.0.smithi012.stdout:5/890: mknod d3/d15/d42/c12c 0 2024-03-18T14:51:10.305 INFO:tasks.workunit.client.1.smithi100.stdout:7/648: mknod d5/d22/db3/cf7 0 2024-03-18T14:51:10.306 INFO:tasks.workunit.client.0.smithi012.stdout:6/887: truncate d17/d26/f4a 3612533 0 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/888: write d17/d26/d36/d4d/d82/fb0 [754966,60775] 0 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/889: creat d17/d19/d27/db1/dd7/f10b x:0 0 0 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/890: read - d17/d19/d27/de8/fb2 zero size 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/891: write d17/d19/d98/fd2 [1632001,16367] 0 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/892: dread - d17/d19/d27/d58/d91/fa0 zero size 2024-03-18T14:51:10.307 INFO:tasks.workunit.client.0.smithi012.stdout:6/893: write d17/d19/d27/d2f/f62 [3408577,124188] 0 2024-03-18T14:51:10.310 INFO:tasks.workunit.client.1.smithi100.stdout:6/614: rmdir d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27 39 2024-03-18T14:51:10.312 INFO:tasks.workunit.client.1.smithi100.stdout:9/665: creat da/db/fd5 x:0 0 0 2024-03-18T14:51:10.313 INFO:tasks.workunit.client.1.smithi100.stdout:4/627: dwrite d9/d13/d25/d51/f60 [0,4194304] 0 2024-03-18T14:51:10.313 INFO:tasks.workunit.client.1.smithi100.stdout:4/628: truncate d9/d13/d25/fa5 53645 0 2024-03-18T14:51:10.315 INFO:tasks.workunit.client.0.smithi012.stdout:6/894: dread d17/d38/d5b/f80 [0,4194304] 0 2024-03-18T14:51:10.316 INFO:tasks.workunit.client.1.smithi100.stdout:1/630: unlink d8/d1e/d11/d2d/f25 0 2024-03-18T14:51:10.318 INFO:tasks.workunit.client.1.smithi100.stdout:8/688: getdents d4/dc/dd/d48 0 2024-03-18T14:51:10.318 INFO:tasks.workunit.client.1.smithi100.stdout:7/649: write d5/d14/d16/d2e/d62/d66/d4f/fd8 [3752236,18820] 0 2024-03-18T14:51:10.320 INFO:tasks.workunit.client.1.smithi100.stdout:4/629: mkdir d9/d13/d39/dc0 0 2024-03-18T14:51:10.322 INFO:tasks.workunit.client.1.smithi100.stdout:8/689: rmdir d4/d96 39 2024-03-18T14:51:10.325 INFO:tasks.workunit.client.1.smithi100.stdout:7/650: link d5/d14/d16/f1d d5/d30/d4d/ff8 0 2024-03-18T14:51:10.326 INFO:tasks.workunit.client.1.smithi100.stdout:9/666: write da/f4f [1510223,65913] 0 2024-03-18T14:51:10.326 INFO:tasks.workunit.client.1.smithi100.stdout:9/667: stat da/dd/cae 0 2024-03-18T14:51:10.329 INFO:tasks.workunit.client.0.smithi012.stdout:6/895: dread d17/d19/d27/d58/d91/fc3 [0,4194304] 0 2024-03-18T14:51:10.329 INFO:tasks.workunit.client.0.smithi012.stdout:6/896: stat d17/d19/d27/d58/d91/fca 0 2024-03-18T14:51:10.329 INFO:tasks.workunit.client.1.smithi100.stdout:1/631: link d8/d1e/d11/d27/d83/c8e d8/d2a/d37/d50/cbd 0 2024-03-18T14:51:10.329 INFO:tasks.workunit.client.1.smithi100.stdout:1/632: fdatasync d8/d2a/d37/d50/f85 0 2024-03-18T14:51:10.331 INFO:tasks.workunit.client.1.smithi100.stdout:8/690: rename d4/dc/dd/de/d21/d23/f28 to d4/dc/d31/d85/ff5 0 2024-03-18T14:51:10.333 INFO:tasks.workunit.client.0.smithi012.stdout:6/897: mknod d17/d26/d36/d4d/d82/c10c 0 2024-03-18T14:51:10.334 INFO:tasks.workunit.client.1.smithi100.stdout:9/668: mkdir da/d46/dd6 0 2024-03-18T14:51:10.334 INFO:tasks.workunit.client.1.smithi100.stdout:9/669: readlink da/dd/d3c/l65 0 2024-03-18T14:51:10.334 INFO:tasks.workunit.client.1.smithi100.stdout:9/670: write da/f16 [1237180,37402] 0 2024-03-18T14:51:10.335 INFO:tasks.workunit.client.1.smithi100.stdout:1/633: symlink d8/d1e/d11/d8a/lbe 0 2024-03-18T14:51:10.335 INFO:tasks.workunit.client.1.smithi100.stdout:1/634: fsync d8/d2a/f6b 0 2024-03-18T14:51:10.337 INFO:tasks.workunit.client.1.smithi100.stdout:8/691: unlink d4/dc/dd/d48/caf 0 2024-03-18T14:51:10.339 INFO:tasks.workunit.client.1.smithi100.stdout:9/671: mknod da/dd/d3e/cd7 0 2024-03-18T14:51:10.341 INFO:tasks.workunit.client.1.smithi100.stdout:8/692: rmdir d4/dc/d1f/d59/d6c 39 2024-03-18T14:51:10.342 INFO:tasks.workunit.client.1.smithi100.stdout:9/672: dread da/d46/d4d/f9f [0,4194304] 0 2024-03-18T14:51:10.342 INFO:tasks.workunit.client.1.smithi100.stdout:9/673: read da/f4f [335869,8783] 0 2024-03-18T14:51:10.343 INFO:tasks.workunit.client.1.smithi100.stdout:8/693: unlink d4/dc/d31/d85/d8e/fe9 0 2024-03-18T14:51:10.345 INFO:tasks.workunit.client.1.smithi100.stdout:9/674: mknod da/dd/d29/d71/d72/cd8 0 2024-03-18T14:51:10.347 INFO:tasks.workunit.client.1.smithi100.stdout:8/694: creat d4/dc/d31/d85/d8e/ff6 x:0 0 0 2024-03-18T14:51:10.347 INFO:tasks.workunit.client.1.smithi100.stdout:8/695: write d4/d3c/fe8 [820251,55037] 0 2024-03-18T14:51:10.347 INFO:tasks.workunit.client.1.smithi100.stdout:8/696: creat d4/dc/dd/de/ff7 x:0 0 0 2024-03-18T14:51:10.347 INFO:tasks.workunit.client.1.smithi100.stdout:8/697: chown d4/dc/dd/de/d19/fe0 251 1 2024-03-18T14:51:10.348 INFO:tasks.workunit.client.1.smithi100.stdout:1/635: dread d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:10.348 INFO:tasks.workunit.client.1.smithi100.stdout:1/636: chown d8/d1e/d11/d27/la3 3738 1 2024-03-18T14:51:10.348 INFO:tasks.workunit.client.1.smithi100.stdout:9/675: link da/dd/d29/d71/d72/faa da/dc3/fd9 0 2024-03-18T14:51:10.352 INFO:tasks.workunit.client.1.smithi100.stdout:8/698: mkdir d4/d96/df8 0 2024-03-18T14:51:10.353 INFO:tasks.workunit.client.1.smithi100.stdout:1/637: mknod d8/d30/d61/d8d/cbf 0 2024-03-18T14:51:10.356 INFO:tasks.workunit.client.0.smithi012.stdout:5/891: dwrite d3/d29/d9d/dba/fd7 [0,4194304] 0 2024-03-18T14:51:10.356 INFO:tasks.workunit.client.0.smithi012.stdout:5/892: truncate d3/d84/f9c 1049616 0 2024-03-18T14:51:10.356 INFO:tasks.workunit.client.0.smithi012.stdout:4/915: dwrite d6/f21 [0,4194304] 0 2024-03-18T14:51:10.356 INFO:tasks.workunit.client.0.smithi012.stdout:4/916: readlink d6/d1a/d30/d46/l4d 0 2024-03-18T14:51:10.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/699: mkdir d4/dc/d31/d85/dd6/df9 0 2024-03-18T14:51:10.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/700: dread - d4/dc/d1f/d59/d6c/fcf zero size 2024-03-18T14:51:10.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/701: readlink d4/d3c/lbe 0 2024-03-18T14:51:10.357 INFO:tasks.workunit.client.1.smithi100.stdout:2/646: dwrite d3/d6/d5f/d63/fdb [0,4194304] 0 2024-03-18T14:51:10.358 INFO:tasks.workunit.client.1.smithi100.stdout:2/647: symlink d3/da/de/ldd 0 2024-03-18T14:51:10.358 INFO:tasks.workunit.client.1.smithi100.stdout:2/648: creat d3/d1a/fde x:0 0 0 2024-03-18T14:51:10.361 INFO:tasks.workunit.client.1.smithi100.stdout:7/651: dwrite d5/d30/d4d/d5a/f78 [0,4194304] 0 2024-03-18T14:51:10.361 INFO:tasks.workunit.client.0.smithi012.stdout:7/973: dwrite dd/d13/d23/d2f/d7a/dab/f13d [0,4194304] 0 2024-03-18T14:51:10.364 INFO:tasks.workunit.client.1.smithi100.stdout:6/615: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [0,4194304] 0 2024-03-18T14:51:10.364 INFO:tasks.workunit.client.1.smithi100.stdout:6/616: write d6/d7/df/d26/d4e/d66/d36/f54 [507339,95800] 0 2024-03-18T14:51:10.364 INFO:tasks.workunit.client.1.smithi100.stdout:4/630: dwrite d9/da6/fb6 [0,4194304] 0 2024-03-18T14:51:10.364 INFO:tasks.workunit.client.0.smithi012.stdout:6/898: dwrite d17/d19/d27/d6e/d84/ffa [0,4194304] 0 2024-03-18T14:51:10.365 INFO:tasks.workunit.client.1.smithi100.stdout:1/638: dread d8/d1e/d11/d27/d3f/f51 [0,4194304] 0 2024-03-18T14:51:10.366 INFO:tasks.workunit.client.0.smithi012.stdout:5/893: mknod d3/d15/d42/dc6/c12d 0 2024-03-18T14:51:10.368 INFO:tasks.workunit.client.1.smithi100.stdout:2/649: getdents d3/da/de/d6e 0 2024-03-18T14:51:10.371 INFO:tasks.workunit.client.0.smithi012.stdout:4/917: link d6/d1a/f62 d6/d1a/d30/d46/d6f/f13d 0 2024-03-18T14:51:10.373 INFO:tasks.workunit.client.0.smithi012.stdout:4/918: dread d6/f11e [0,4194304] 0 2024-03-18T14:51:10.373 INFO:tasks.workunit.client.0.smithi012.stdout:4/919: write d6/d19/d39/f50 [4318931,118638] 0 2024-03-18T14:51:10.373 INFO:tasks.workunit.client.0.smithi012.stdout:4/920: fdatasync d6/da/d89/d94/dbe/dd6/d12f/ffb 0 2024-03-18T14:51:10.373 INFO:tasks.workunit.client.1.smithi100.stdout:7/652: dread d5/d14/d16/d2e/d62/d66/d4f/fd8 [0,4194304] 0 2024-03-18T14:51:10.379 INFO:tasks.workunit.client.1.smithi100.stdout:2/650: dread d3/d1a/d6c/f8b [0,4194304] 0 2024-03-18T14:51:10.381 INFO:tasks.workunit.client.0.smithi012.stdout:7/974: rename dd/d13/d23/d2f/d53/dc6 to dd/d13/d23/d76/d6b/d14a 0 2024-03-18T14:51:10.382 INFO:tasks.workunit.client.1.smithi100.stdout:8/702: dwrite d4/dc/dd/de/d21/d23/d47/f50 [4194304,4194304] 0 2024-03-18T14:51:10.385 INFO:tasks.workunit.client.0.smithi012.stdout:6/899: link d17/d38/d51/da8/d100/dd8/le5 d17/d38/d51/da8/d100/l10d 0 2024-03-18T14:51:10.387 INFO:tasks.workunit.client.1.smithi100.stdout:9/676: dwrite da/d46/d8b/f66 [0,4194304] 0 2024-03-18T14:51:10.389 INFO:tasks.workunit.client.0.smithi012.stdout:5/894: unlink d3/d15/d5f/fe1 0 2024-03-18T14:51:10.390 INFO:tasks.workunit.client.1.smithi100.stdout:6/617: mknod d6/d7/df/d26/d4e/d66/d36/cbc 0 2024-03-18T14:51:10.392 INFO:tasks.workunit.client.0.smithi012.stdout:4/921: unlink d6/da/ffc 0 2024-03-18T14:51:10.393 INFO:tasks.workunit.client.0.smithi012.stdout:4/922: chown d6/d19/d39/f6d 18502409 1 2024-03-18T14:51:10.395 INFO:tasks.workunit.client.1.smithi100.stdout:1/639: link d8/d1e/d11/d6d/l9c d8/d30/lc0 0 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/975: dread dd/d13/d23/d76/d6b/f108 [0,4194304] 0 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/976: chown dd/d13/d23/d2f/d7a/db6/c97 13165 1 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/977: write dd/d13/d23/d60/f89 [1829936,33718] 0 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/978: dread - dd/d13/d23/d76/d109/dfa/f123 zero size 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/979: dread - dd/d13/d23/d76/d109/dfa/f123 zero size 2024-03-18T14:51:10.396 INFO:tasks.workunit.client.0.smithi012.stdout:7/980: readlink dd/lda 0 2024-03-18T14:51:10.398 INFO:tasks.workunit.client.1.smithi100.stdout:7/653: unlink d5/d14/ce8 0 2024-03-18T14:51:10.400 INFO:tasks.workunit.client.1.smithi100.stdout:3/668: sync 2024-03-18T14:51:10.400 INFO:tasks.workunit.client.1.smithi100.stdout:3/669: read - d3/dd/d21/d3e/fcc zero size 2024-03-18T14:51:10.400 INFO:tasks.workunit.client.1.smithi100.stdout:3/670: write d3/d16/d13/d17/fe5 [690063,61363] 0 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/900: dread d17/d26/d36/f67 [0,4194304] 0 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/901: creat d17/d19/d27/d2f/f10e x:0 0 0 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/902: fdatasync d17/d19/d27/d6e/ff9 0 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/903: chown d17/d19/d27/d2f/lab 854 1 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/904: write d17/d19/d27/d2f/f10e [735864,66781] 0 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/905: chown d17/d26/d36/d4d/f81 34230150 1 2024-03-18T14:51:10.401 INFO:tasks.workunit.client.0.smithi012.stdout:6/906: chown l3 1195162 1 2024-03-18T14:51:10.402 INFO:tasks.workunit.client.0.smithi012.stdout:6/907: write d17/d38/d51/d66/d7d/fd9 [833527,26056] 0 2024-03-18T14:51:10.402 INFO:tasks.workunit.client.0.smithi012.stdout:6/908: dread - d17/d19/d27/db1/dd7/f10b zero size 2024-03-18T14:51:10.402 INFO:tasks.workunit.client.1.smithi100.stdout:2/651: mkdir d3/d6/d29/d36/ddf 0 2024-03-18T14:51:10.402 INFO:tasks.workunit.client.1.smithi100.stdout:2/652: truncate d3/d13/d40/f83 939497 0 2024-03-18T14:51:10.410 INFO:tasks.workunit.client.1.smithi100.stdout:0/689: dwrite dd/d31/f52 [0,4194304] 0 2024-03-18T14:51:10.411 INFO:tasks.workunit.client.1.smithi100.stdout:0/690: chown dd/d2b/d95/d79/d7d/da9/db0/fbe 5459 1 2024-03-18T14:51:10.411 INFO:tasks.workunit.client.1.smithi100.stdout:8/703: rename d4/dc/d31/c6a to d4/dc/d1f/d59/d6c/ddb/cfa 0 2024-03-18T14:51:10.411 INFO:tasks.workunit.client.1.smithi100.stdout:8/704: dread - d4/dc/dd/d48/fda zero size 2024-03-18T14:51:10.411 INFO:tasks.workunit.client.1.smithi100.stdout:5/620: dwrite d29/d2d/d39/d83/f8b [0,4194304] 0 2024-03-18T14:51:10.416 INFO:tasks.workunit.client.0.smithi012.stdout:5/895: creat d3/d29/d35/d5b/f12e x:0 0 0 2024-03-18T14:51:10.419 INFO:tasks.workunit.client.0.smithi012.stdout:4/923: mkdir d6/da/d58/d113/d13e 0 2024-03-18T14:51:10.423 INFO:tasks.workunit.client.0.smithi012.stdout:7/981: dread dd/d13/d23/d76/d6b/f10e [0,4194304] 0 2024-03-18T14:51:10.426 INFO:tasks.workunit.client.1.smithi100.stdout:6/618: link d6/f2f d6/d7/df/d26/d4e/d66/d36/fbd 0 2024-03-18T14:51:10.427 INFO:tasks.workunit.client.0.smithi012.stdout:4/924: rename d6/da/d58/c103 to d6/da/d89/d97/d12e/c13f 0 2024-03-18T14:51:10.427 INFO:tasks.workunit.client.0.smithi012.stdout:4/925: chown d6/d19/d39/l102 165 1 2024-03-18T14:51:10.428 INFO:tasks.workunit.client.1.smithi100.stdout:1/640: dwrite d8/d1e/d11/d6d/d59/d93/f9e [0,4194304] 0 2024-03-18T14:51:10.428 INFO:tasks.workunit.client.1.smithi100.stdout:7/654: mknod d5/d90/da1/cf9 0 2024-03-18T14:51:10.430 INFO:tasks.workunit.client.0.smithi012.stdout:6/909: dwrite d17/d19/d27/d58/f6f [4194304,4194304] 0 2024-03-18T14:51:10.432 INFO:tasks.workunit.client.0.smithi012.stdout:4/926: symlink d6/d19/da5/l140 0 2024-03-18T14:51:10.434 INFO:tasks.workunit.client.1.smithi100.stdout:3/671: mknod d3/d16/d13/d17/dab/cec 0 2024-03-18T14:51:10.438 INFO:tasks.workunit.client.1.smithi100.stdout:2/653: mknod d3/da/de/d20/d82/ce0 0 2024-03-18T14:51:10.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/910: link d17/d19/c3d d17/d38/d51/da8/d100/c10f 0 2024-03-18T14:51:10.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/911: readlink d17/d26/d36/l5f 0 2024-03-18T14:51:10.446 INFO:tasks.workunit.client.0.smithi012.stdout:6/912: dread - d17/d19/d27/de8/fb2 zero size 2024-03-18T14:51:10.449 INFO:tasks.workunit.client.1.smithi100.stdout:0/691: mkdir dd/d61/df8 0 2024-03-18T14:51:10.451 INFO:tasks.workunit.client.0.smithi012.stdout:4/927: unlink d6/d1a/c55 0 2024-03-18T14:51:10.451 INFO:tasks.workunit.client.0.smithi012.stdout:4/928: write d6/da/d89/d94/dbe/dd6/d12f/ff1 [5231777,130262] 0 2024-03-18T14:51:10.451 INFO:tasks.workunit.client.0.smithi012.stdout:4/929: write d6/d1a/d30/d46/d6f/d7c/fa8 [656811,54979] 0 2024-03-18T14:51:10.452 INFO:tasks.workunit.client.1.smithi100.stdout:8/705: symlink d4/dc/d31/lfb 0 2024-03-18T14:51:10.453 INFO:tasks.workunit.client.0.smithi012.stdout:6/913: truncate d17/d19/d27/f54 1323248 0 2024-03-18T14:51:10.459 INFO:tasks.workunit.client.0.smithi012.stdout:4/930: mknod d6/da/d89/d94/da2/c141 0 2024-03-18T14:51:10.459 INFO:tasks.workunit.client.0.smithi012.stdout:4/931: stat d6/d1a/d30/d46/d6f/l99 0 2024-03-18T14:51:10.459 INFO:tasks.workunit.client.0.smithi012.stdout:4/932: chown d6/d1a/d30/d46/d6f/d8b/d11f 851626295 1 2024-03-18T14:51:10.459 INFO:tasks.workunit.client.1.smithi100.stdout:5/621: truncate f15 1794308 0 2024-03-18T14:51:10.460 INFO:tasks.workunit.client.0.smithi012.stdout:6/914: mknod d17/d26/d36/c110 0 2024-03-18T14:51:10.460 INFO:tasks.workunit.client.0.smithi012.stdout:6/915: fdatasync d17/d38/d51/d72/f74 0 2024-03-18T14:51:10.461 INFO:tasks.workunit.client.1.smithi100.stdout:9/677: getdents da/dd/d29/d71/d72 0 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 -- 172.21.15.12:0/1389292560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 msgr2=0x7f2e840ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 --2- 172.21.15.12:0/1389292560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e840ffa60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e74004940 tx=0x7f2e74009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 -- 172.21.15.12:0/1389292560 shutdown_connections 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 --2- 172.21.15.12:0/1389292560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e84102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 --2- 172.21.15.12:0/1389292560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e840ffa60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 -- 172.21.15.12:0/1389292560 >> 172.21.15.12:0/1389292560 conn(0x7f2e840f7430 msgr2=0x7f2e840f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:10.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 -- 172.21.15.12:0/1389292560 shutdown_connections 2024-03-18T14:51:10.464 INFO:tasks.workunit.client.1.smithi100.stdout:3/672: dwrite d3/d16/d13/d1f/f5c [0,4194304] 0 2024-03-18T14:51:10.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/678: dread da/d88/f8c [0,4194304] 0 2024-03-18T14:51:10.465 INFO:tasks.workunit.client.1.smithi100.stdout:9/679: truncate da/dd/d3c/fc9 71905 0 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.462+0000 7f2e88f55700 1 -- 172.21.15.12:0/1389292560 wait complete. 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 Processor -- start 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 -- start start 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e8410b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e8410bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8410c390 con 0x7f2e84066240 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e88f55700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8410cbe0 con 0x7f2e840fffa0 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e8259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e8410b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:10.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e8259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e8410b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50310/0 (socket says 172.21.15.12:50310) 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e8259c700 1 -- 172.21.15.12:0/3530325510 learned_addr learned my addr 172.21.15.12:0/3530325510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.463+0000 7f2e81d9b700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e8410bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.464+0000 7f2e81d9b700 1 -- 172.21.15.12:0/3530325510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 msgr2=0x7f2e8410b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.464+0000 7f2e81d9b700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e8410b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.464+0000 7f2e81d9b700 1 -- 172.21.15.12:0/3530325510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74005040 con 0x7f2e840fffa0 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.464+0000 7f2e81d9b700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e8410bd40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fd0 tx=0x7f2e7800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.465+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e78004e90 con 0x7f2e840fffa0 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.465+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e7800f040 con 0x7f2e840fffa0 2024-03-18T14:51:10.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.465+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7800ca50 con 0x7f2e840fffa0 2024-03-18T14:51:10.468 INFO:tasks.workunit.client.1.smithi100.stdout:6/619: mknod d6/d7/df/d26/cbe 0 2024-03-18T14:51:10.468 INFO:tasks.workunit.client.1.smithi100.stdout:6/620: truncate d6/d7/df/d26/d4e/d66/d36/d4f/d71/f13 4379700 0 2024-03-18T14:51:10.468 INFO:tasks.workunit.client.1.smithi100.stdout:8/706: dread d4/dc/d1f/d33/f8d [0,4194304] 0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.465+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8410ce60 con 0x7f2e840fffa0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.465+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8410d3a0 con 0x7f2e840fffa0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.466+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8405f5f0 con 0x7f2e840fffa0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.467+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f2e78007cc0 con 0x7f2e840fffa0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.467+0000 7f2e72ffd700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 0x7f2e6c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.468+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2e78089b60 con 0x7f2e840fffa0 2024-03-18T14:51:10.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.468+0000 7f2e8259c700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 0x7f2e6c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:10.469 INFO:tasks.workunit.client.0.smithi012.stdout:6/916: creat d17/d38/d51/d66/dee/f111 x:0 0 0 2024-03-18T14:51:10.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.468+0000 7f2e8259c700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 0x7f2e6c06db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b0d0 tx=0x7f2e74009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:10.470 INFO:tasks.workunit.client.0.smithi012.stdout:6/917: truncate d17/f2c 3735963 0 2024-03-18T14:51:10.471 INFO:tasks.workunit.client.1.smithi100.stdout:1/641: rename d8/d1e/d11/d6d/d59/d93/f9e to d8/d30/d61/d8d/fc1 0 2024-03-18T14:51:10.472 INFO:tasks.workunit.client.0.smithi012.stdout:6/918: mkdir d17/d38/d51/da8/d112 0 2024-03-18T14:51:10.473 INFO:tasks.workunit.client.1.smithi100.stdout:7/655: mknod d5/d14/d16/d2e/d62/d66/d4f/dcb/cfa 0 2024-03-18T14:51:10.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.473+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f2e78050040 con 0x7f2e840fffa0 2024-03-18T14:51:10.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/654: getdents d3/da 0 2024-03-18T14:51:10.475 INFO:tasks.workunit.client.1.smithi100.stdout:2/655: fsync d3/da/de/d20/d82/fd8 0 2024-03-18T14:51:10.476 INFO:tasks.workunit.client.0.smithi012.stdout:6/919: creat d17/d26/d36/d105/f113 x:0 0 0 2024-03-18T14:51:10.476 INFO:tasks.workunit.client.0.smithi012.stdout:6/920: read - d17/d38/fc7 zero size 2024-03-18T14:51:10.477 INFO:tasks.workunit.client.0.smithi012.stdout:6/921: creat d17/d26/d36/d4d/d82/f114 x:0 0 0 2024-03-18T14:51:10.477 INFO:tasks.workunit.client.0.smithi012.stdout:6/922: truncate d17/d19/d27/db1/dd7/f10b 103937 0 2024-03-18T14:51:10.478 INFO:tasks.workunit.client.1.smithi100.stdout:2/656: dread d3/d6/f9 [0,4194304] 0 2024-03-18T14:51:10.479 INFO:tasks.workunit.client.1.smithi100.stdout:5/622: unlink d29/d2d/d32/f5c 0 2024-03-18T14:51:10.480 INFO:tasks.workunit.client.1.smithi100.stdout:2/657: dread d3/da/de/fa5 [0,4194304] 0 2024-03-18T14:51:10.487 INFO:tasks.workunit.client.0.smithi012.stdout:6/923: symlink d17/d19/d27/d58/l115 0 2024-03-18T14:51:10.496 INFO:tasks.workunit.client.0.smithi012.stdout:6/924: write d17/f22 [630827,62832] 0 2024-03-18T14:51:10.496 INFO:tasks.workunit.client.1.smithi100.stdout:3/673: chown d3/dd/d21/dae/dc9 0 1 2024-03-18T14:51:10.498 INFO:tasks.workunit.client.1.smithi100.stdout:6/621: creat d6/d7/df/d26/d4e/d66/d36/dbb/fbf x:0 0 0 2024-03-18T14:51:10.498 INFO:tasks.workunit.client.1.smithi100.stdout:6/622: stat d6/d7/df/d26/d4e/l9e 0 2024-03-18T14:51:10.498 INFO:tasks.workunit.client.1.smithi100.stdout:2/658: dwrite d3/d6/d5f/fc6 [0,4194304] 0 2024-03-18T14:51:10.500 INFO:tasks.workunit.client.1.smithi100.stdout:8/707: symlink d4/dc/dd/de/d21/d23/d47/d54/lfc 0 2024-03-18T14:51:10.502 INFO:tasks.workunit.client.0.smithi012.stdout:6/925: creat d17/d38/d51/da8/d100/f116 x:0 0 0 2024-03-18T14:51:10.503 INFO:tasks.workunit.client.1.smithi100.stdout:6/623: dread d6/d7/df/f2a [0,4194304] 0 2024-03-18T14:51:10.506 INFO:tasks.workunit.client.0.smithi012.stdout:2/976: sync 2024-03-18T14:51:10.506 INFO:tasks.workunit.client.0.smithi012.stdout:2/977: chown d0/f6e 2806262 1 2024-03-18T14:51:10.510 INFO:tasks.workunit.client.0.smithi012.stdout:6/926: mknod d17/d18/c117 0 2024-03-18T14:51:10.510 INFO:tasks.workunit.client.0.smithi012.stdout:6/927: readlink d17/d19/lf3 0 2024-03-18T14:51:10.510 INFO:tasks.workunit.client.0.smithi012.stdout:6/928: fsync d17/d18/dc1/dde/ffc 0 2024-03-18T14:51:10.512 INFO:tasks.workunit.client.0.smithi012.stdout:2/978: symlink d0/d6b/dbf/d33/d77/df5/l136 0 2024-03-18T14:51:10.512 INFO:tasks.workunit.client.0.smithi012.stdout:2/979: chown d0/c2e 154544 1 2024-03-18T14:51:10.512 INFO:tasks.workunit.client.1.smithi100.stdout:6/624: write d6/f2f [4422406,762] 0 2024-03-18T14:51:10.513 INFO:tasks.workunit.client.1.smithi100.stdout:6/625: truncate d6/d7/f8 2097315 0 2024-03-18T14:51:10.513 INFO:tasks.workunit.client.1.smithi100.stdout:7/656: symlink d5/d22/d3b/d9f/da0/db1/dd3/lfb 0 2024-03-18T14:51:10.515 INFO:tasks.workunit.client.1.smithi100.stdout:5/623: mkdir d29/d2d/d39/d71/d96/dc8 0 2024-03-18T14:51:10.516 INFO:tasks.workunit.client.1.smithi100.stdout:8/708: dwrite d4/dc/dd/de/d21/d23/d25/f71 [0,4194304] 0 2024-03-18T14:51:10.521 INFO:tasks.workunit.client.1.smithi100.stdout:5/624: write f18 [363310,38312] 0 2024-03-18T14:51:10.521 INFO:tasks.workunit.client.1.smithi100.stdout:5/625: fdatasync d29/d2d/d39/d83/f85 0 2024-03-18T14:51:10.522 INFO:tasks.workunit.client.0.smithi012.stdout:6/929: unlink d17/d19/d27/d58/fdf 0 2024-03-18T14:51:10.523 INFO:tasks.workunit.client.0.smithi012.stdout:6/930: fdatasync d17/d19/d27/d58/d91/ff6 0 2024-03-18T14:51:10.525 INFO:tasks.workunit.client.0.smithi012.stdout:2/980: mknod d0/d6b/dbf/d33/d80/de1/c137 0 2024-03-18T14:51:10.525 INFO:tasks.workunit.client.0.smithi012.stdout:2/981: chown d0/d3c/d40/d47/f49 27625098 1 2024-03-18T14:51:10.529 INFO:tasks.workunit.client.1.smithi100.stdout:2/659: unlink d3/d6/d5f/d63/fdb 0 2024-03-18T14:51:10.529 INFO:tasks.workunit.client.1.smithi100.stdout:3/674: read d3/dd/d21/f22 [3904997,85609] 0 2024-03-18T14:51:10.531 INFO:tasks.workunit.client.0.smithi012.stdout:2/982: unlink d0/d6c/fa9 0 2024-03-18T14:51:10.533 INFO:tasks.workunit.client.1.smithi100.stdout:6/626: unlink d6/d7/df/d26/d4e/d66/d36/d4f/d71/c21 0 2024-03-18T14:51:10.534 INFO:tasks.workunit.client.0.smithi012.stdout:6/931: mkdir d17/d38/d118 0 2024-03-18T14:51:10.538 INFO:tasks.workunit.client.0.smithi012.stdout:6/932: truncate d17/d19/d27/fd6 1731943 0 2024-03-18T14:51:10.539 INFO:tasks.workunit.client.0.smithi012.stdout:6/933: dread d17/d18/f45 [0,4194304] 0 2024-03-18T14:51:10.540 INFO:tasks.workunit.client.1.smithi100.stdout:4/631: sync 2024-03-18T14:51:10.541 INFO:tasks.workunit.client.0.smithi012.stdout:4/933: dwrite d6/da/f91 [0,4194304] 0 2024-03-18T14:51:10.541 INFO:tasks.workunit.client.0.smithi012.stdout:4/934: write d6/d19/f133 [797632,13652] 0 2024-03-18T14:51:10.546 INFO:tasks.workunit.client.0.smithi012.stdout:2/983: write d0/f95 [530136,130308] 0 2024-03-18T14:51:10.547 INFO:tasks.workunit.client.1.smithi100.stdout:8/709: creat d4/dc/dd/de/d21/d23/d47/d56/ffd x:0 0 0 2024-03-18T14:51:10.547 INFO:tasks.workunit.client.1.smithi100.stdout:8/710: chown d4/dc/dd/de/d21/d23/fd3 222 1 2024-03-18T14:51:10.550 INFO:tasks.workunit.client.1.smithi100.stdout:5/626: rmdir d29/d2d/d39/d83/dc1 39 2024-03-18T14:51:10.550 INFO:tasks.workunit.client.1.smithi100.stdout:5/627: write d29/d2d/d39/fae [774902,50208] 0 2024-03-18T14:51:10.551 INFO:tasks.workunit.client.1.smithi100.stdout:4/632: dread d9/d27/f2e [0,4194304] 0 2024-03-18T14:51:10.552 INFO:tasks.workunit.client.1.smithi100.stdout:2/660: symlink d3/da/de/d20/d82/dad/dc5/le1 0 2024-03-18T14:51:10.553 INFO:tasks.workunit.client.0.smithi012.stdout:4/935: symlink d6/d1a/d30/d46/d6f/d8b/d13b/l142 0 2024-03-18T14:51:10.554 INFO:tasks.workunit.client.1.smithi100.stdout:3/675: truncate d3/d16/d13/d1f/f90 119244 0 2024-03-18T14:51:10.559 INFO:tasks.workunit.client.1.smithi100.stdout:0/692: rename dd/d31/d91/d56/d62/d6b to dd/d20/d44/df9 0 2024-03-18T14:51:10.560 INFO:tasks.workunit.client.1.smithi100.stdout:6/627: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [0,4194304] 0 2024-03-18T14:51:10.563 INFO:tasks.workunit.client.0.smithi012.stdout:6/934: dwrite f4 [0,4194304] 0 2024-03-18T14:51:10.563 INFO:tasks.workunit.client.0.smithi012.stdout:6/935: stat d17/d19/d27/ccb 0 2024-03-18T14:51:10.563 INFO:tasks.workunit.client.0.smithi012.stdout:6/936: write d17/d38/d51/fa9 [943195,85282] 0 2024-03-18T14:51:10.563 INFO:tasks.workunit.client.0.smithi012.stdout:2/984: creat d0/d6b/dbf/d33/d80/f138 x:0 0 0 2024-03-18T14:51:10.565 INFO:tasks.workunit.client.1.smithi100.stdout:7/657: unlink d5/d22/c7e 0 2024-03-18T14:51:10.565 INFO:tasks.workunit.client.1.smithi100.stdout:5/628: dread d29/d2d/d32/f68 [0,4194304] 0 2024-03-18T14:51:10.568 INFO:tasks.workunit.client.0.smithi012.stdout:4/936: rename d6/d1a/d92/fbc to d6/da/d58/dd2/f143 0 2024-03-18T14:51:10.568 INFO:tasks.workunit.client.0.smithi012.stdout:4/937: write d6/da/d58/f59 [5111755,104680] 0 2024-03-18T14:51:10.569 INFO:tasks.workunit.client.1.smithi100.stdout:6/628: dread d6/d12/f5c [0,4194304] 0 2024-03-18T14:51:10.569 INFO:tasks.workunit.client.1.smithi100.stdout:8/711: symlink d4/d5a/lfe 0 2024-03-18T14:51:10.571 INFO:tasks.workunit.client.0.smithi012.stdout:2/985: getdents d0/d6b/dbf 0 2024-03-18T14:51:10.574 INFO:tasks.workunit.client.0.smithi012.stdout:6/937: read d17/d38/d51/da8/d100/dd8/f88 [1332582,11700] 0 2024-03-18T14:51:10.576 INFO:tasks.workunit.client.1.smithi100.stdout:1/642: rename d8/d1e/d11/d27/d3f/daa/fb5 to d8/d2a/fc2 0 2024-03-18T14:51:10.579 INFO:tasks.workunit.client.0.smithi012.stdout:2/986: mknod d0/d6b/dbf/d33/d71/d90/dea/c139 0 2024-03-18T14:51:10.579 INFO:tasks.workunit.client.1.smithi100.stdout:7/658: dread d5/f52 [4194304,4194304] 0 2024-03-18T14:51:10.579 INFO:tasks.workunit.client.1.smithi100.stdout:2/661: dwrite d3/d6/f98 [0,4194304] 0 2024-03-18T14:51:10.579 INFO:tasks.workunit.client.1.smithi100.stdout:2/662: dread - d3/da/de/d20/d82/dad/fca zero size 2024-03-18T14:51:10.581 INFO:tasks.workunit.client.0.smithi012.stdout:6/938: mkdir d17/d38/d51/da8/d100/d119 0 2024-03-18T14:51:10.582 INFO:tasks.workunit.client.1.smithi100.stdout:1/643: dread d8/d2a/f47 [0,4194304] 0 2024-03-18T14:51:10.582 INFO:tasks.workunit.client.1.smithi100.stdout:1/644: dread - d8/d1e/d11/d8a/fa8 zero size 2024-03-18T14:51:10.585 INFO:tasks.workunit.client.0.smithi012.stdout:2/987: mknod d0/d6c/c13a 0 2024-03-18T14:51:10.585 INFO:tasks.workunit.client.0.smithi012.stdout:2/988: readlink d0/d6b/dbf/d31/l34 0 2024-03-18T14:51:10.585 INFO:tasks.workunit.client.1.smithi100.stdout:6/629: write d6/d7/f8d [3208270,119172] 0 2024-03-18T14:51:10.590 INFO:tasks.workunit.client.0.smithi012.stdout:2/989: rename d0/d6b/dbf/d33/dc9/fcd to d0/d6b/dbf/d33/dac/f13b 0 2024-03-18T14:51:10.595 INFO:tasks.workunit.client.1.smithi100.stdout:7/659: write d5/d14/d16/d2e/d62/f97 [553301,820] 0 2024-03-18T14:51:10.595 INFO:tasks.workunit.client.1.smithi100.stdout:7/660: write d5/d22/d24/fcd [572062,1965] 0 2024-03-18T14:51:10.595 INFO:tasks.workunit.client.1.smithi100.stdout:7/661: creat d5/d14/ffc x:0 0 0 2024-03-18T14:51:10.595 INFO:tasks.workunit.client.1.smithi100.stdout:7/662: creat d5/d14/d16/d2e/d62/ffd x:0 0 0 2024-03-18T14:51:10.596 INFO:tasks.workunit.client.0.smithi012.stdout:4/938: dwrite d6/f80 [0,4194304] 0 2024-03-18T14:51:10.598 INFO:tasks.workunit.client.0.smithi012.stdout:2/990: mkdir d0/d3c/d40/d47/d13c 0 2024-03-18T14:51:10.600 INFO:tasks.workunit.client.1.smithi100.stdout:5/629: symlink d29/d2d/d39/d71/lc9 0 2024-03-18T14:51:10.600 INFO:tasks.workunit.client.1.smithi100.stdout:5/630: getdents d29/d2d/d39/d71/d96/dc8 0 2024-03-18T14:51:10.600 INFO:tasks.workunit.client.1.smithi100.stdout:5/631: stat d29/d2d/d39/d65/f98 0 2024-03-18T14:51:10.603 INFO:tasks.workunit.client.1.smithi100.stdout:8/712: unlink d4/dc/dd/de/d21/d23/fd3 0 2024-03-18T14:51:10.606 INFO:tasks.workunit.client.1.smithi100.stdout:3/676: symlink d3/d16/d13/d1f/d3f/d4f/d60/led 0 2024-03-18T14:51:10.606 INFO:tasks.workunit.client.1.smithi100.stdout:3/677: write d3/d16/d13/d17/dab/fdb [426115,82879] 0 2024-03-18T14:51:10.609 INFO:tasks.workunit.client.0.smithi012.stdout:4/939: getdents d6/d1a/d30/d46/d6f 0 2024-03-18T14:51:10.611 INFO:tasks.workunit.client.1.smithi100.stdout:0/693: rename dd/f1f to dd/d31/d91/d56/ffa 0 2024-03-18T14:51:10.617 INFO:tasks.workunit.client.1.smithi100.stdout:5/632: read d29/d4a/f9c [2521043,22109] 0 2024-03-18T14:51:10.629 INFO:tasks.workunit.client.0.smithi012.stdout:4/940: symlink d6/da/d89/d94/l144 0 2024-03-18T14:51:10.629 INFO:tasks.workunit.client.0.smithi012.stdout:4/941: creat d6/d19/d6c/d96/f145 x:0 0 0 2024-03-18T14:51:10.629 INFO:tasks.workunit.client.0.smithi012.stdout:4/942: chown d6/d19/d6c/d96 13 1 2024-03-18T14:51:10.629 INFO:tasks.workunit.client.0.smithi012.stdout:4/943: chown d6/d1a/d30/d46/l4d 110837860 1 2024-03-18T14:51:10.632 INFO:tasks.workunit.client.1.smithi100.stdout:2/663: mkdir d3/da/de/d20/d82/dad/de2 0 2024-03-18T14:51:10.639 INFO:tasks.workunit.client.1.smithi100.stdout:7/663: mkdir d5/d14/d16/d2e/d62/dfe 0 2024-03-18T14:51:10.640 INFO:tasks.workunit.client.1.smithi100.stdout:8/713: mkdir d4/dc/dd/de/d21/d23/d47/dff 0 2024-03-18T14:51:10.640 INFO:tasks.workunit.client.1.smithi100.stdout:8/714: creat d4/dc/d31/d85/d8e/f100 x:0 0 0 2024-03-18T14:51:10.642 INFO:tasks.workunit.client.1.smithi100.stdout:2/664: mkdir d3/da/de/d20/d82/dad/dc5/de3 0 2024-03-18T14:51:10.646 INFO:tasks.workunit.client.1.smithi100.stdout:7/664: symlink d5/d14/d16/d2e/d62/dd1/d86/d85/d87/lff 0 2024-03-18T14:51:10.646 INFO:tasks.workunit.client.1.smithi100.stdout:7/665: chown d5/d14/d16/d2e/d62/dd1/d86 10193339 1 2024-03-18T14:51:10.646 INFO:tasks.workunit.client.1.smithi100.stdout:5/633: write d29/f2c [3674968,112321] 0 2024-03-18T14:51:10.647 INFO:tasks.workunit.client.1.smithi100.stdout:8/715: symlink d4/dc/d31/d85/d8e/l101 0 2024-03-18T14:51:10.648 INFO:tasks.workunit.client.1.smithi100.stdout:0/694: rmdir dd/d2b/d95/d79/d7d/da9/dac 39 2024-03-18T14:51:10.648 INFO:tasks.workunit.client.1.smithi100.stdout:2/665: mknod d3/da/de/d20/ce4 0 2024-03-18T14:51:10.649 INFO:tasks.workunit.client.1.smithi100.stdout:7/666: getdents d5/d14/d16/d2e/d62/dd1/d86/d85 0 2024-03-18T14:51:10.649 INFO:tasks.workunit.client.1.smithi100.stdout:7/667: write d5/d22/d3b/d47/fd5 [925159,7176] 0 2024-03-18T14:51:10.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/695: mknod dd/d31/d91/cfb 0 2024-03-18T14:51:10.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/696: write dd/d20/f4e [688631,88004] 0 2024-03-18T14:51:10.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/697: creat dd/d20/d44/ffc x:0 0 0 2024-03-18T14:51:10.651 INFO:tasks.workunit.client.1.smithi100.stdout:0/698: fsync dd/d2b/d95/d79/d7d/f9d 0 2024-03-18T14:51:10.656 INFO:tasks.workunit.client.1.smithi100.stdout:2/666: link d3/d6/f9 d3/d6/d5f/fe5 0 2024-03-18T14:51:10.661 INFO:tasks.workunit.client.0.smithi012.stdout:6/939: dwrite d17/d38/d51/da8/d100/dd8/f48 [0,4194304] 0 2024-03-18T14:51:10.662 INFO:tasks.workunit.client.1.smithi100.stdout:6/630: dwrite d6/d7/f8 [0,4194304] 0 2024-03-18T14:51:10.663 INFO:tasks.workunit.client.1.smithi100.stdout:5/634: dread d29/d2d/d39/d83/f8b [0,4194304] 0 2024-03-18T14:51:10.663 INFO:tasks.workunit.client.1.smithi100.stdout:1/645: dwrite d8/d1e/d11/d6d/d59/d93/fba [0,4194304] 0 2024-03-18T14:51:10.663 INFO:tasks.workunit.client.1.smithi100.stdout:3/678: dwrite d3/d16/d13/d17/fe5 [0,4194304] 0 2024-03-18T14:51:10.665 INFO:tasks.workunit.client.1.smithi100.stdout:3/679: dread d3/d16/d13/d1f/d3f/d4f/d60/f63 [0,4194304] 0 2024-03-18T14:51:10.665 INFO:tasks.workunit.client.1.smithi100.stdout:0/699: link dd/d31/d91/d56/c68 dd/d61/cfd 0 2024-03-18T14:51:10.665 INFO:tasks.workunit.client.1.smithi100.stdout:0/700: creat dd/d31/ffe x:0 0 0 2024-03-18T14:51:10.670 INFO:tasks.workunit.client.1.smithi100.stdout:9/680: sync 2024-03-18T14:51:10.671 INFO:tasks.workunit.client.1.smithi100.stdout:5/635: creat d29/d2d/d32/d92/daa/fca x:0 0 0 2024-03-18T14:51:10.673 INFO:tasks.workunit.client.0.smithi012.stdout:6/940: getdents d17/d19/d27/d58/d91 0 2024-03-18T14:51:10.674 INFO:tasks.workunit.client.1.smithi100.stdout:6/631: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [3129946,6228] 0 2024-03-18T14:51:10.678 INFO:tasks.workunit.client.0.smithi012.stdout:6/941: link d17/d26/f4a d17/d38/d51/d66/d7d/f11a 0 2024-03-18T14:51:10.682 INFO:tasks.workunit.client.1.smithi100.stdout:8/716: rmdir d4/dc/dd 39 2024-03-18T14:51:10.682 INFO:tasks.workunit.client.1.smithi100.stdout:8/717: fdatasync d4/dc/dd/de/d21/d23/d25/f9d 0 2024-03-18T14:51:10.682 INFO:tasks.workunit.client.1.smithi100.stdout:8/718: fdatasync d4/d5a/dbb/f6b 0 2024-03-18T14:51:10.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/944: dwrite d6/d1a/d64/f78 [0,4194304] 0 2024-03-18T14:51:10.687 INFO:tasks.workunit.client.0.smithi012.stdout:2/991: dwrite d0/d6b/dbf/ff3 [0,4194304] 0 2024-03-18T14:51:10.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/945: creat d6/d1a/d98/daf/db5/f146 x:0 0 0 2024-03-18T14:51:10.687 INFO:tasks.workunit.client.0.smithi012.stdout:4/946: fdatasync d6/d19/ffe 0 2024-03-18T14:51:10.692 INFO:tasks.workunit.client.1.smithi100.stdout:1/646: creat d8/d1e/d11/fc3 x:0 0 0 2024-03-18T14:51:10.693 INFO:tasks.workunit.client.0.smithi012.stdout:6/942: dread d17/d19/d27/d58/f77 [0,4194304] 0 2024-03-18T14:51:10.693 INFO:tasks.workunit.client.0.smithi012.stdout:6/943: stat d17/d38/d51/d66/faf 0 2024-03-18T14:51:10.698 INFO:tasks.workunit.client.1.smithi100.stdout:9/681: mknod da/dd/d3b/cda 0 2024-03-18T14:51:10.698 INFO:tasks.workunit.client.1.smithi100.stdout:9/682: fsync da/dc3/fd9 0 2024-03-18T14:51:10.698 INFO:tasks.workunit.client.1.smithi100.stdout:9/683: chown da/dd/d3b/c99 7293 1 2024-03-18T14:51:10.698 INFO:tasks.workunit.client.0.smithi012.stdout:4/947: write d6/db6/f124 [1798595,89842] 0 2024-03-18T14:51:10.699 INFO:tasks.workunit.client.1.smithi100.stdout:6/632: creat d6/d7/df/d26/d4e/d66/fc0 x:0 0 0 2024-03-18T14:51:10.699 INFO:tasks.workunit.client.1.smithi100.stdout:6/633: truncate d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa3 155356 0 2024-03-18T14:51:10.700 INFO:tasks.workunit.client.1.smithi100.stdout:3/680: rmdir d3/d16/d13/d17/dab 39 2024-03-18T14:51:10.701 INFO:tasks.workunit.client.1.smithi100.stdout:9/684: unlink da/dd/c2a 0 2024-03-18T14:51:10.702 INFO:tasks.workunit.client.1.smithi100.stdout:9/685: dread da/dd/fa1 [0,4194304] 0 2024-03-18T14:51:10.703 INFO:tasks.workunit.client.1.smithi100.stdout:6/634: creat d6/d12/d2c/fc1 x:0 0 0 2024-03-18T14:51:10.706 INFO:tasks.workunit.client.1.smithi100.stdout:3/681: creat d3/d16/d13/d17/d7d/fee x:0 0 0 2024-03-18T14:51:10.706 INFO:tasks.workunit.client.1.smithi100.stdout:3/682: dread - d3/dd/d21/d3e/fc0 zero size 2024-03-18T14:51:10.706 INFO:tasks.workunit.client.1.smithi100.stdout:7/668: dwrite d5/f72 [4194304,4194304] 0 2024-03-18T14:51:10.706 INFO:tasks.workunit.client.1.smithi100.stdout:7/669: write d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6/f65 [1441164,115422] 0 2024-03-18T14:51:10.707 INFO:tasks.workunit.client.0.smithi012.stdout:2/992: getdents d0/d10 0 2024-03-18T14:51:10.707 INFO:tasks.workunit.client.0.smithi012.stdout:2/993: truncate d0/d6b/dbf/d33/d77/df5/d103/d61/f6d 5157148 0 2024-03-18T14:51:10.707 INFO:tasks.workunit.client.0.smithi012.stdout:2/994: chown d0/da2/d59/fa4 2 1 2024-03-18T14:51:10.707 INFO:tasks.workunit.client.0.smithi012.stdout:2/995: dread - d0/d6b/dbf/d33/dc9/f133 zero size 2024-03-18T14:51:10.708 INFO:tasks.workunit.client.1.smithi100.stdout:9/686: mkdir da/dc3/ddb 0 2024-03-18T14:51:10.708 INFO:tasks.workunit.client.1.smithi100.stdout:9/687: chown da/d46/d4d/d67/fab 16961873 1 2024-03-18T14:51:10.708 INFO:tasks.workunit.client.1.smithi100.stdout:9/688: write da/d46/d4d/d62/f93 [1802496,13038] 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:9/689: readlink da/d2c/d70/lb9 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:9/690: fsync da/db/fd5 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:1/647: write d8/d1e/d11/d27/d3f/f51 [2928188,40921] 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:1/648: stat d8/d1e/d11/f77 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:1/649: read - d8/d1e/d11/d27/fa7 zero size 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:5/636: dwrite d29/d2d/d39/f4e [0,4194304] 0 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:5/637: chown f18 4 1 2024-03-18T14:51:10.709 INFO:tasks.workunit.client.1.smithi100.stdout:5/638: stat d29/d2d/d39/d61/f6b 0 2024-03-18T14:51:10.710 INFO:tasks.workunit.client.1.smithi100.stdout:6/635: mknod d6/d7/df/d26/d4e/db5/cc2 0 2024-03-18T14:51:10.710 INFO:tasks.workunit.client.1.smithi100.stdout:6/636: chown d6/d7/df/d72/d48/d76/fad 181 1 2024-03-18T14:51:10.714 INFO:tasks.workunit.client.0.smithi012.stdout:6/944: symlink d17/d38/d5b/l11b 0 2024-03-18T14:51:10.714 INFO:tasks.workunit.client.1.smithi100.stdout:3/683: truncate d3/d16/d13/d1f/d3f/d4f/f68 266595 0 2024-03-18T14:51:10.716 INFO:tasks.workunit.client.1.smithi100.stdout:7/670: mknod d5/d30/d4d/c100 0 2024-03-18T14:51:10.717 INFO:tasks.workunit.client.0.smithi012.stdout:4/948: mknod d6/d1a/d30/d46/d6f/d8b/d11f/c147 0 2024-03-18T14:51:10.717 INFO:tasks.workunit.client.0.smithi012.stdout:4/949: chown d6/d1a/d30/d46/d6f/d8b/f105 953441 1 2024-03-18T14:51:10.719 INFO:tasks.workunit.client.0.smithi012.stdout:2/996: truncate d0/d3c/fb5 4226834 0 2024-03-18T14:51:10.720 INFO:tasks.workunit.client.1.smithi100.stdout:1/650: write d8/fab [886470,55524] 0 2024-03-18T14:51:10.720 INFO:tasks.workunit.client.1.smithi100.stdout:1/651: chown d8/d2a/d37 741 1 2024-03-18T14:51:10.720 INFO:tasks.workunit.client.1.smithi100.stdout:1/652: readlink d8/d1e/d11/d27/d83/lae 0 2024-03-18T14:51:10.721 INFO:tasks.workunit.client.1.smithi100.stdout:2/667: dwrite d3/da/de/ff [0,4194304] 0 2024-03-18T14:51:10.724 INFO:tasks.workunit.client.0.smithi012.stdout:2/997: rename d0/d6c/db7/f12d to d0/d6b/dbf/d33/d71/d90/dea/dfe/f13d 0 2024-03-18T14:51:10.724 INFO:tasks.workunit.client.1.smithi100.stdout:0/701: dwrite dd/d20/d44/f5e [0,4194304] 0 2024-03-18T14:51:10.726 INFO:tasks.workunit.client.1.smithi100.stdout:5/639: unlink d29/d2d/d39/d65/d7d/l86 0 2024-03-18T14:51:10.726 INFO:tasks.workunit.client.1.smithi100.stdout:5/640: readlink d29/d2d/d32/l40 0 2024-03-18T14:51:10.729 INFO:tasks.workunit.client.1.smithi100.stdout:6/637: symlink d6/d12/lc3 0 2024-03-18T14:51:10.731 INFO:tasks.workunit.client.0.smithi012.stdout:4/950: write d6/da/d89/d95/fa4 [3095199,105442] 0 2024-03-18T14:51:10.733 INFO:tasks.workunit.client.1.smithi100.stdout:2/668: mknod d3/d6/d29/ce6 0 2024-03-18T14:51:10.735 INFO:tasks.workunit.client.1.smithi100.stdout:1/653: dread d8/d2a/f6b [4194304,4194304] 0 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/702: mkdir dd/d61/dcd/dff 0 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/703: creat dd/d61/f100 x:0 0 0 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/704: creat dd/d31/f101 x:0 0 0 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/705: truncate dd/d20/f27 1541648 0 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/706: chown dd/d2b/l7a 2053646 1 2024-03-18T14:51:10.737 INFO:tasks.workunit.client.1.smithi100.stdout:0/707: stat dd/d31/d91/d65/ff5 0 2024-03-18T14:51:10.738 INFO:tasks.workunit.client.0.smithi012.stdout:2/998: dread d0/d3c/d40/d47/d85/fdb [0,4194304] 0 2024-03-18T14:51:10.738 INFO:tasks.workunit.client.1.smithi100.stdout:5/641: symlink d29/d2d/d39/d83/dc1/lcb 0 2024-03-18T14:51:10.740 INFO:tasks.workunit.client.1.smithi100.stdout:6/638: truncate d6/d12/f75 1111436 0 2024-03-18T14:51:10.742 INFO:tasks.workunit.client.1.smithi100.stdout:1/654: link d8/d1e/d11/d27/la3 d8/d1e/d11/d6d/d59/lc4 0 2024-03-18T14:51:10.747 INFO:tasks.workunit.client.0.smithi012.stdout:4/951: dread d6/da/d58/fbd [0,4194304] 0 2024-03-18T14:51:10.747 INFO:tasks.workunit.client.0.smithi012.stdout:4/952: chown d6/da/d58/fbd 0 1 2024-03-18T14:51:10.751 INFO:tasks.workunit.client.1.smithi100.stdout:2/669: dread d3/d6/fa9 [0,4194304] 0 2024-03-18T14:51:10.752 INFO:tasks.workunit.client.1.smithi100.stdout:5/642: rmdir d29/d2d/d39/d71/da9 0 2024-03-18T14:51:10.759 INFO:tasks.workunit.client.1.smithi100.stdout:8/719: dwrite d4/dc/d31/f46 [0,4194304] 0 2024-03-18T14:51:10.759 INFO:tasks.workunit.client.1.smithi100.stdout:8/720: chown d4/dc/d1f/d33/f4e 763834 1 2024-03-18T14:51:10.759 INFO:tasks.workunit.client.1.smithi100.stdout:8/721: stat d4/dc/d1f/f32 0 2024-03-18T14:51:10.763 INFO:tasks.workunit.client.1.smithi100.stdout:0/708: write dd/d31/d91/f3f [4953110,60052] 0 2024-03-18T14:51:10.763 INFO:tasks.workunit.client.1.smithi100.stdout:0/709: write dd/f8f [5166522,47320] 0 2024-03-18T14:51:10.765 INFO:tasks.workunit.client.1.smithi100.stdout:2/670: dread d3/d6/d29/d36/d49/f7a [4194304,4194304] 0 2024-03-18T14:51:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:3/900: sync 2024-03-18T14:51:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:3/901: creat dc/d3c/d59/f123 x:0 0 0 2024-03-18T14:51:10.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/722: rename d4/dc/dd/de/d21/d23/d25/lf3 to d4/dc/dd/de/d19/d2c/da2/l102 0 2024-03-18T14:51:10.766 INFO:tasks.workunit.client.1.smithi100.stdout:0/710: creat dd/d20/db5/f102 x:0 0 0 2024-03-18T14:51:10.768 INFO:tasks.workunit.client.1.smithi100.stdout:5/643: dread d29/d2d/d39/f46 [0,4194304] 0 2024-03-18T14:51:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:3/902: rename dc/d3c/d59/db2 to dc/d3c/d4c/d57/d103/d9a/dd7/d124 0 2024-03-18T14:51:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:3/903: chown dc/d30/d3a/db7/ce1 6 1 2024-03-18T14:51:10.770 INFO:tasks.workunit.client.1.smithi100.stdout:2/671: symlink d3/da/de/d20/d82/dad/dc5/de3/le7 0 2024-03-18T14:51:10.771 INFO:tasks.workunit.client.1.smithi100.stdout:5/644: dread d29/d2d/f6f [0,4194304] 0 2024-03-18T14:51:10.771 INFO:tasks.workunit.client.1.smithi100.stdout:5/645: chown f17 217 1 2024-03-18T14:51:10.771 INFO:tasks.workunit.client.1.smithi100.stdout:5/646: chown f2 6 1 2024-03-18T14:51:10.771 INFO:tasks.workunit.client.0.smithi012.stdout:3/904: unlink dc/c23 0 2024-03-18T14:51:10.772 INFO:tasks.workunit.client.1.smithi100.stdout:2/672: stat d3/da/de/d48/cc4 0 2024-03-18T14:51:10.772 INFO:tasks.workunit.client.1.smithi100.stdout:2/673: chown d3/d1a/d76/fc7 117592 1 2024-03-18T14:51:10.772 INFO:tasks.workunit.client.1.smithi100.stdout:2/674: read d3/fba [2845,106642] 0 2024-03-18T14:51:10.772 INFO:tasks.workunit.client.1.smithi100.stdout:2/675: readlink d3/lb0 0 2024-03-18T14:51:10.772 INFO:tasks.workunit.client.1.smithi100.stdout:2/676: creat d3/d13/d40/dac/fe8 x:0 0 0 2024-03-18T14:51:10.774 INFO:tasks.workunit.client.1.smithi100.stdout:2/677: symlink d3/da/de/d20/le9 0 2024-03-18T14:51:10.785 INFO:tasks.workunit.client.1.smithi100.stdout:1/655: write d8/f7e [3439885,20634] 0 2024-03-18T14:51:10.787 INFO:tasks.workunit.client.1.smithi100.stdout:8/723: dread d4/dc/dd/de/d21/d23/d25/f71 [0,4194304] 0 2024-03-18T14:51:10.787 INFO:tasks.workunit.client.1.smithi100.stdout:1/656: mkdir d8/d2a/d37/d50/dc5 0 2024-03-18T14:51:10.788 INFO:tasks.workunit.client.1.smithi100.stdout:1/657: link d8/d30/l38 d8/d30/d61/d8d/lc6 0 2024-03-18T14:51:10.788 INFO:tasks.workunit.client.1.smithi100.stdout:1/658: creat d8/d2a/fc7 x:0 0 0 2024-03-18T14:51:10.789 INFO:tasks.workunit.client.1.smithi100.stdout:1/659: truncate d8/d1e/d11/d6d/f87 1713220 0 2024-03-18T14:51:10.789 INFO:tasks.workunit.client.1.smithi100.stdout:8/724: mkdir d4/dc/dd/de/d19/d103 0 2024-03-18T14:51:10.790 INFO:tasks.workunit.client.1.smithi100.stdout:4/633: sync 2024-03-18T14:51:10.791 INFO:tasks.workunit.client.1.smithi100.stdout:1/660: chown d8/d2a/d37/d50/d6f/d81 11957005 1 2024-03-18T14:51:10.791 INFO:tasks.workunit.client.1.smithi100.stdout:1/661: creat d8/d2a/d37/fc8 x:0 0 0 2024-03-18T14:51:10.793 INFO:tasks.workunit.client.1.smithi100.stdout:8/725: creat d4/dc/dd/de/d19/db0/dd5/f104 x:0 0 0 2024-03-18T14:51:10.793 INFO:tasks.workunit.client.1.smithi100.stdout:8/726: getdents d4/dc/dd/de/d21/d23/d47/dff 0 2024-03-18T14:51:10.794 INFO:tasks.workunit.client.1.smithi100.stdout:4/634: mknod d9/d13/d5a/d5b/cc1 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/635: symlink d9/d27/d62/d7b/d82/dac/lc2 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/636: read d9/d3a/f9c [926556,46573] 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/637: read d9/d3a/f99 [1899655,66371] 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/638: readlink d9/d27/l3e 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/639: truncate d9/f32 7535695 0 2024-03-18T14:51:10.798 INFO:tasks.workunit.client.1.smithi100.stdout:4/640: read d9/d17/f24 [1637663,41175] 0 2024-03-18T14:51:10.808 INFO:tasks.workunit.client.0.smithi012.stdout:6/945: dwrite d17/d38/d51/da8/d100/dd8/f88 [4194304,4194304] 0 2024-03-18T14:51:10.808 INFO:tasks.workunit.client.1.smithi100.stdout:9/691: dwrite da/d46/d4d/d62/fb7 [0,4194304] 0 2024-03-18T14:51:10.809 INFO:tasks.workunit.client.1.smithi100.stdout:3/684: dwrite d3/f29 [0,4194304] 0 2024-03-18T14:51:10.809 INFO:tasks.workunit.client.1.smithi100.stdout:7/671: dwrite d5/d14/d16/d2e/d62/dd1/f43 [0,4194304] 0 2024-03-18T14:51:10.811 INFO:tasks.workunit.client.1.smithi100.stdout:3/685: write d3/d16/d13/d17/f96 [640598,108310] 0 2024-03-18T14:51:10.813 INFO:tasks.workunit.client.1.smithi100.stdout:4/641: dread d9/d27/d62/f7d [0,4194304] 0 2024-03-18T14:51:10.813 INFO:tasks.workunit.client.0.smithi012.stdout:3/905: dwrite f3 [0,4194304] 0 2024-03-18T14:51:10.813 INFO:tasks.workunit.client.0.smithi012.stdout:4/953: dwrite d6/da/f123 [0,4194304] 0 2024-03-18T14:51:10.814 INFO:tasks.workunit.client.1.smithi100.stdout:9/692: mknod da/dd/d3b/cdc 0 2024-03-18T14:51:10.815 INFO:tasks.workunit.client.1.smithi100.stdout:1/662: dwrite d8/d1e/d11/d27/d83/fa1 [0,4194304] 0 2024-03-18T14:51:10.815 INFO:tasks.workunit.client.1.smithi100.stdout:1/663: readlink d8/d1e/d11/d2d/l79 0 2024-03-18T14:51:10.815 INFO:tasks.workunit.client.0.smithi012.stdout:2/999: dwrite d0/d6b/dbf/d33/d77/d10a/f128 [0,4194304] 0 2024-03-18T14:51:10.817 INFO:tasks.workunit.client.1.smithi100.stdout:0/711: fsync dd/d31/d91/f3f 0 2024-03-18T14:51:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.818+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e8410d680 con 0x7f2e6c06b660 2024-03-18T14:51:10.825 INFO:tasks.workunit.client.1.smithi100.stdout:7/672: symlink d5/d22/d3b/l101 0 2024-03-18T14:51:10.827 INFO:tasks.workunit.client.1.smithi100.stdout:3/686: symlink d3/d16/d13/d17/d7d/lef 0 2024-03-18T14:51:10.827 INFO:tasks.workunit.client.1.smithi100.stdout:3/687: readlink d3/d16/d13/d71/db9/laf 0 2024-03-18T14:51:10.827 INFO:tasks.workunit.client.1.smithi100.stdout:3/688: chown d3/d16/d13/d71/d74/fb5 235 1 2024-03-18T14:51:10.828 INFO:tasks.workunit.client.0.smithi012.stdout:4/954: read d6/f33 [3456411,5075] 0 2024-03-18T14:51:10.828 INFO:tasks.workunit.client.0.smithi012.stdout:4/955: write d6/d1a/d30/d46/d6f/d7c/fdb [1018979,20356] 0 2024-03-18T14:51:10.829 INFO:tasks.workunit.client.0.smithi012.stdout:3/906: link dc/d30/fac dc/d30/d3a/da0/dc5/deb/d102/f125 0 2024-03-18T14:51:10.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/712: dread dd/d20/f5c [0,4194304] 0 2024-03-18T14:51:10.832 INFO:tasks.workunit.client.1.smithi100.stdout:9/693: mkdir da/d2c/d70/ddd 0 2024-03-18T14:51:10.838 INFO:tasks.workunit.client.1.smithi100.stdout:1/664: mkdir d8/d2a/d37/d50/d6f/d71/dc9 0 2024-03-18T14:51:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.839+0000 7f2e72ffd700 1 -- 172.21.15.12:0/3530325510 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f2e8410d680 con 0x7f2e6c06b660 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (6m) 2m ago 8m 24.7M - 0.25.0 c8568f914cd2 3a6187ef0acb 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (8m) 2m ago 8m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (7m) 0s ago 7m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (6m) 2m ago 7m 80.3M - 9.4.7 954c08fa6188 a14a00878205 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (2m) 2m ago 2m 14.4M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (2m) 2m ago 2m 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (2m) 0s ago 2m 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (2m) 0s ago 2m 11.6M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (32s) 0s ago 6m 488M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 148db09fb780 2024-03-18T14:51:10.840 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (9m) 2m ago 9m 44.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (6m) 0s ago 6m 39.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (8m) 2m ago 8m 14.4M - 1.5.0 0da6a335fe13 4ebf4cd84beb 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (7m) 0s ago 7m 17.3M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (6m) 2m ago 6m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (5m) 2m ago 5m 46.1M 4289M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (5m) 2m ago 5m 46.7M 4289M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (4m) 0s ago 4m 274M 5312M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (4m) 0s ago 4m 221M 5312M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (4m) 0s ago 4m 266M 5312M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:51:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (6m) 2m ago 7m 53.0M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:51:10.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/956: mkdir d6/d19/d6c/d96/d148 0 2024-03-18T14:51:10.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/957: fdatasync d6/f8c 0 2024-03-18T14:51:10.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/958: truncate d6/da/d89/fce 4661211 0 2024-03-18T14:51:10.842 INFO:tasks.workunit.client.0.smithi012.stdout:4/959: write d6/da/d58/f9e [815992,28756] 0 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 msgr2=0x7f2e6c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 0x7f2e6c06db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400b0d0 tx=0x7f2e74009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 msgr2=0x7f2e8410bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e8410bd40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fd0 tx=0x7f2e7800b040 comp rx=0 tx=0).stop 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 shutdown_connections 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f2e6c06b660 0x7f2e6c06db10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e84066240 0x7f2e8410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 --2- 172.21.15.12:0/3530325510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e840fffa0 0x7f2e8410bd40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 >> 172.21.15.12:0/3530325510 conn(0x7f2e840f7430 msgr2=0x7f2e840fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 shutdown_connections 2024-03-18T14:51:10.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:10.841+0000 7f2e88f55700 1 -- 172.21.15.12:0/3530325510 wait complete. 2024-03-18T14:51:10.844 INFO:tasks.workunit.client.0.smithi012.stdout:3/907: symlink dc/d116/l126 0 2024-03-18T14:51:10.845 INFO:tasks.workunit.client.1.smithi100.stdout:9/694: write da/d46/d8b/f66 [197431,11905] 0 2024-03-18T14:51:10.845 INFO:tasks.workunit.client.1.smithi100.stdout:9/695: fsync da/d46/d4d/d67/fca 0 2024-03-18T14:51:10.845 INFO:tasks.workunit.client.1.smithi100.stdout:9/696: write da/dd/f8f [256687,103679] 0 2024-03-18T14:51:10.851 INFO:tasks.workunit.client.1.smithi100.stdout:4/642: dwrite d9/d27/f98 [0,4194304] 0 2024-03-18T14:51:10.851 INFO:tasks.workunit.client.1.smithi100.stdout:4/643: write d9/d3a/f9c [3137891,89059] 0 2024-03-18T14:51:10.851 INFO:tasks.workunit.client.0.smithi012.stdout:4/960: link d6/d19/d6c/f88 d6/d1a/d64/f149 0 2024-03-18T14:51:10.851 INFO:tasks.workunit.client.0.smithi012.stdout:6/946: dwrite d17/d38/f5a [0,4194304] 0 2024-03-18T14:51:10.851 INFO:tasks.workunit.client.0.smithi012.stdout:6/947: chown d17/d18/dc1/cef 0 1 2024-03-18T14:51:10.852 INFO:tasks.workunit.client.0.smithi012.stdout:4/961: symlink d6/d1a/d64/d12d/l14a 0 2024-03-18T14:51:10.855 INFO:tasks.workunit.client.1.smithi100.stdout:2/678: rename d3/d13 to d3/da/de/d20/dea 0 2024-03-18T14:51:10.862 INFO:tasks.workunit.client.0.smithi012.stdout:6/948: rename d17/d38/d51/da8/d100/dd8/fa5 to d17/d19/d27/db1/f11c 0 2024-03-18T14:51:10.864 INFO:tasks.workunit.client.0.smithi012.stdout:4/962: creat d6/d1a/d64/d12d/f14b x:0 0 0 2024-03-18T14:51:10.864 INFO:tasks.workunit.client.0.smithi012.stdout:4/963: dread - d6/d1a/d30/d46/d6f/d8b/f105 zero size 2024-03-18T14:51:10.865 INFO:tasks.workunit.client.1.smithi100.stdout:7/673: symlink d5/d30/d4d/d5a/l102 0 2024-03-18T14:51:10.866 INFO:tasks.workunit.client.1.smithi100.stdout:3/689: mkdir d3/d16/d13/d71/db9/df0 0 2024-03-18T14:51:10.867 INFO:tasks.workunit.client.0.smithi012.stdout:6/949: creat d17/d18/db8/f11d x:0 0 0 2024-03-18T14:51:10.870 INFO:tasks.workunit.client.0.smithi012.stdout:4/964: getdents d6/da/d89/d97/d12e 0 2024-03-18T14:51:10.872 INFO:tasks.workunit.client.1.smithi100.stdout:9/697: creat da/dd/d29/d9e/dcd/fde x:0 0 0 2024-03-18T14:51:10.876 INFO:tasks.workunit.client.0.smithi012.stdout:4/965: truncate d6/f28 943897 0 2024-03-18T14:51:10.878 INFO:tasks.workunit.client.1.smithi100.stdout:3/690: dread d3/d16/d13/d1f/d3f/d4f/f51 [0,4194304] 0 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/727: rename d4/dc/dd/de/d19/d2c/l95 to d4/dc/dd/de/d19/db0/dd5/l105 0 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/728: creat d4/dc/d97/f106 x:0 0 0 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/729: truncate d4/dc/d31/d85/d8e/ff6 288251 0 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/730: chown d4/dc/dd/de/d21/d23/d25/c70 133 1 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/731: fsync d4/dc/d1f/f5d 0 2024-03-18T14:51:10.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/732: getdents d4/dc/dc6 0 2024-03-18T14:51:10.884 INFO:tasks.workunit.client.1.smithi100.stdout:2/679: getdents d3/d6/d5f/d6a 0 2024-03-18T14:51:10.885 INFO:tasks.workunit.client.1.smithi100.stdout:7/674: creat d5/d30/f103 x:0 0 0 2024-03-18T14:51:10.886 INFO:tasks.workunit.client.1.smithi100.stdout:9/698: getdents da/dd/d3b 0 2024-03-18T14:51:10.888 INFO:tasks.workunit.client.1.smithi100.stdout:0/713: rename dd/d2b/d95/c9b to dd/d20/c103 0 2024-03-18T14:51:10.890 INFO:tasks.workunit.client.1.smithi100.stdout:9/699: dread da/d46/d4d/fad [4194304,4194304] 0 2024-03-18T14:51:10.890 INFO:tasks.workunit.client.1.smithi100.stdout:9/700: dread - da/d46/d4d/d67/faf zero size 2024-03-18T14:51:10.891 INFO:tasks.workunit.client.1.smithi100.stdout:2/680: mknod d3/da/db8/ceb 0 2024-03-18T14:51:10.891 INFO:tasks.workunit.client.1.smithi100.stdout:3/691: write d3/d16/d13/d1f/f59 [4055922,105981] 0 2024-03-18T14:51:10.892 INFO:tasks.workunit.client.0.smithi012.stdout:6/950: dwrite d17/d38/d51/f85 [0,4194304] 0 2024-03-18T14:51:10.892 INFO:tasks.workunit.client.0.smithi012.stdout:6/951: dread - d17/d19/d27/d58/f86 zero size 2024-03-18T14:51:10.892 INFO:tasks.workunit.client.0.smithi012.stdout:6/952: creat d17/d19/d27/db1/dd7/f11e x:0 0 0 2024-03-18T14:51:10.894 INFO:tasks.workunit.client.1.smithi100.stdout:1/665: rename d8/d2a/d37/fc8 to d8/d2a/d37/d50/d6f/d81/d91/fca 0 2024-03-18T14:51:10.894 INFO:tasks.workunit.client.1.smithi100.stdout:8/733: dread d4/dc/d1f/f9c [0,4194304] 0 2024-03-18T14:51:10.895 INFO:tasks.workunit.client.1.smithi100.stdout:0/714: creat dd/d31/d91/d56/d62/d6e/f104 x:0 0 0 2024-03-18T14:51:10.897 INFO:tasks.workunit.client.1.smithi100.stdout:9/701: getdents da/dd/d29/d71/d72/db2 0 2024-03-18T14:51:10.897 INFO:tasks.workunit.client.1.smithi100.stdout:9/702: chown da/dd/da0/fa6 157030641 1 2024-03-18T14:51:10.899 INFO:tasks.workunit.client.1.smithi100.stdout:2/681: creat d3/da/de/d20/d82/dad/de2/fec x:0 0 0 2024-03-18T14:51:10.901 INFO:tasks.workunit.client.0.smithi012.stdout:6/953: rmdir d17/d19/d27/db1 39 2024-03-18T14:51:10.904 INFO:tasks.workunit.client.1.smithi100.stdout:4/644: rename d9/d13/d39/f74 to d9/d27/da2/fc3 0 2024-03-18T14:51:10.904 INFO:tasks.workunit.client.1.smithi100.stdout:4/645: write d9/d3a/f99 [3249489,50649] 0 2024-03-18T14:51:10.905 INFO:tasks.workunit.client.1.smithi100.stdout:1/666: dread - d8/d2a/d37/d50/f85 zero size 2024-03-18T14:51:10.906 INFO:tasks.workunit.client.1.smithi100.stdout:3/692: dread d3/d16/fc [0,4194304] 0 2024-03-18T14:51:10.906 INFO:tasks.workunit.client.1.smithi100.stdout:3/693: chown d3/dd/d49 469 1 2024-03-18T14:51:10.907 INFO:tasks.workunit.client.0.smithi012.stdout:4/966: dwrite d6/d1a/f35 [0,4194304] 0 2024-03-18T14:51:10.907 INFO:tasks.workunit.client.1.smithi100.stdout:8/734: mknod d4/dc/dd/de/d21/d23/d47/dff/c107 0 2024-03-18T14:51:10.908 INFO:tasks.workunit.client.1.smithi100.stdout:9/703: link da/db/cc5 da/dd/d3c/d6b/cdf 0 2024-03-18T14:51:10.908 INFO:tasks.workunit.client.1.smithi100.stdout:9/704: read da/dd/d3c/d4b/f58 [2287332,113196] 0 2024-03-18T14:51:10.909 INFO:tasks.workunit.client.0.smithi012.stdout:4/967: rmdir d6/d1a/d30/d46/d6f/d8b/d11f 39 2024-03-18T14:51:10.909 INFO:tasks.workunit.client.0.smithi012.stdout:4/968: fdatasync d6/d1a/f31 0 2024-03-18T14:51:10.911 INFO:tasks.workunit.client.0.smithi012.stdout:4/969: mkdir d6/da/d89/da1/dbb/d14c 0 2024-03-18T14:51:10.916 INFO:tasks.workunit.client.1.smithi100.stdout:4/646: truncate d9/f10 12125428 0 2024-03-18T14:51:10.916 INFO:tasks.workunit.client.1.smithi100.stdout:4/647: write d9/d13/d39/f8f [258218,80581] 0 2024-03-18T14:51:10.916 INFO:tasks.workunit.client.1.smithi100.stdout:4/648: chown c4 3 1 2024-03-18T14:51:10.916 INFO:tasks.workunit.client.1.smithi100.stdout:4/649: chown d9/d27/fb2 327 1 2024-03-18T14:51:10.917 INFO:tasks.workunit.client.0.smithi012.stdout:4/970: truncate d6/da/d58/f9e 680688 0 2024-03-18T14:51:10.919 INFO:tasks.workunit.client.1.smithi100.stdout:1/667: creat d8/d30/d61/fcb x:0 0 0 2024-03-18T14:51:10.919 INFO:tasks.workunit.client.1.smithi100.stdout:1/668: dread - d8/d1e/d11/d2d/f6e zero size 2024-03-18T14:51:10.920 INFO:tasks.workunit.client.1.smithi100.stdout:3/694: mknod d3/d16/d13/d1f/cf1 0 2024-03-18T14:51:10.921 INFO:tasks.workunit.client.1.smithi100.stdout:3/695: dread - d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb zero size 2024-03-18T14:51:10.921 INFO:tasks.workunit.client.0.smithi012.stdout:4/971: mknod d6/da/d89/d94/dbe/dd6/c14d 0 2024-03-18T14:51:10.921 INFO:tasks.workunit.client.0.smithi012.stdout:4/972: readlink d6/d1a/l108 0 2024-03-18T14:51:10.923 INFO:tasks.workunit.client.1.smithi100.stdout:8/735: symlink d4/dc/dd/de/d21/d23/d47/l108 0 2024-03-18T14:51:10.924 INFO:tasks.workunit.client.1.smithi100.stdout:9/705: mkdir da/d46/dd6/de0 0 2024-03-18T14:51:10.927 INFO:tasks.workunit.client.1.smithi100.stdout:7/675: rename d5/d22/l36 to d5/l104 0 2024-03-18T14:51:10.928 INFO:tasks.workunit.client.1.smithi100.stdout:1/669: mknod d8/d1e/ccc 0 2024-03-18T14:51:10.928 INFO:tasks.workunit.client.1.smithi100.stdout:1/670: write d8/d1e/d11/d27/d3f/d76/fb0 [686397,50193] 0 2024-03-18T14:51:10.929 INFO:tasks.workunit.client.1.smithi100.stdout:3/696: write d3/d16/d13/f39 [3480279,34122] 0 2024-03-18T14:51:10.929 INFO:tasks.workunit.client.1.smithi100.stdout:0/715: dwrite dd/f85 [4194304,4194304] 0 2024-03-18T14:51:10.929 INFO:tasks.workunit.client.1.smithi100.stdout:4/650: dread d9/d3a/f69 [0,4194304] 0 2024-03-18T14:51:10.929 INFO:tasks.workunit.client.0.smithi012.stdout:6/954: dwrite d17/d26/d36/f7a [0,4194304] 0 2024-03-18T14:51:10.929 INFO:tasks.workunit.client.1.smithi100.stdout:9/706: truncate da/d46/fa3 1067533 0 2024-03-18T14:51:10.931 INFO:tasks.workunit.client.0.smithi012.stdout:4/973: write d6/f25 [3946749,109848] 0 2024-03-18T14:51:10.932 INFO:tasks.workunit.client.1.smithi100.stdout:6/639: sync 2024-03-18T14:51:10.932 INFO:tasks.workunit.client.1.smithi100.stdout:6/640: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f9c [290581,87734] 0 2024-03-18T14:51:10.932 INFO:tasks.workunit.client.1.smithi100.stdout:2/682: rename d3/da/de/d20/dea/d40/d53/f8f to d3/d6/d29/d36/ddf/fed 0 2024-03-18T14:51:10.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/683: creat d3/da/de/d20/d82/fee x:0 0 0 2024-03-18T14:51:10.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/684: chown d3/d6/d5f/d6a 6877 1 2024-03-18T14:51:10.933 INFO:tasks.workunit.client.1.smithi100.stdout:2/685: chown d3/da/de/d20/l86 23676 1 2024-03-18T14:51:10.933 INFO:tasks.workunit.client.1.smithi100.stdout:8/736: write d4/dc/d31/d85/ff5 [2669276,93822] 0 2024-03-18T14:51:10.934 INFO:tasks.workunit.client.1.smithi100.stdout:8/737: write d4/dc/dd/f44 [7320718,98999] 0 2024-03-18T14:51:10.934 INFO:tasks.workunit.client.1.smithi100.stdout:8/738: fdatasync d4/dc/d31/fd9 0 2024-03-18T14:51:10.934 INFO:tasks.workunit.client.1.smithi100.stdout:7/676: mkdir d5/d22/d24/d6e/d105 0 2024-03-18T14:51:10.935 INFO:tasks.workunit.client.1.smithi100.stdout:5/647: sync 2024-03-18T14:51:10.940 INFO:tasks.workunit.client.1.smithi100.stdout:3/697: mkdir d3/d16/d13/d71/df2 0 2024-03-18T14:51:10.940 INFO:tasks.workunit.client.1.smithi100.stdout:3/698: readlink d3/dd/d49/l4e 0 2024-03-18T14:51:10.941 INFO:tasks.workunit.client.0.smithi012.stdout:6/955: dread d17/d38/d51/d66/faf [0,4194304] 0 2024-03-18T14:51:10.947 INFO:tasks.workunit.client.1.smithi100.stdout:0/716: creat dd/d2b/d95/d79/d7d/da9/dac/f105 x:0 0 0 2024-03-18T14:51:10.948 INFO:tasks.workunit.client.0.smithi012.stdout:6/956: mknod d17/d38/d51/df7/c11f 0 2024-03-18T14:51:10.949 INFO:tasks.workunit.client.0.smithi012.stdout:4/974: dwrite d6/da/d89/d95/fa4 [0,4194304] 0 2024-03-18T14:51:10.949 INFO:tasks.workunit.client.0.smithi012.stdout:4/975: truncate d6/d1a/ff9 484780 0 2024-03-18T14:51:10.949 INFO:tasks.workunit.client.1.smithi100.stdout:4/651: symlink d9/d13/lc4 0 2024-03-18T14:51:10.950 INFO:tasks.workunit.client.1.smithi100.stdout:9/707: mknod da/d51/d55/d7e/ce1 0 2024-03-18T14:51:10.950 INFO:tasks.workunit.client.1.smithi100.stdout:9/708: creat da/dcb/fe2 x:0 0 0 2024-03-18T14:51:10.951 INFO:tasks.workunit.client.0.smithi012.stdout:6/957: mkdir d17/d26/d36/d4d/d82/dcd/d120 0 2024-03-18T14:51:10.953 INFO:tasks.workunit.client.0.smithi012.stdout:4/976: link d6/d19/d6c/d96/dda/f129 d6/f14e 0 2024-03-18T14:51:10.953 INFO:tasks.workunit.client.0.smithi012.stdout:4/977: write d6/d1a/d30/d46/d6f/d8b/f105 [388584,29702] 0 2024-03-18T14:51:10.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/641: rename d6/d7/df/d26/d4e/d66/d36/d4f/d71/f79 to d6/d12/d2c/fc4 0 2024-03-18T14:51:10.961 INFO:tasks.workunit.client.0.smithi012.stdout:4/978: link d6/da/d89/d94/da2/cfd d6/d1a/d30/d46/d6f/d8b/d11f/c14f 0 2024-03-18T14:51:10.963 INFO:tasks.workunit.client.1.smithi100.stdout:2/686: symlink d3/d1a/lef 0 2024-03-18T14:51:10.967 INFO:tasks.workunit.client.1.smithi100.stdout:8/739: truncate d4/dc/f8a 1638101 0 2024-03-18T14:51:10.967 INFO:tasks.workunit.client.1.smithi100.stdout:8/740: truncate d4/dc/dd/de/ff7 1035227 0 2024-03-18T14:51:10.968 INFO:tasks.workunit.client.1.smithi100.stdout:7/677: mknod d5/d90/da1/c106 0 2024-03-18T14:51:10.969 INFO:tasks.workunit.client.0.smithi012.stdout:4/979: symlink d6/d1a/l150 0 2024-03-18T14:51:10.969 INFO:tasks.workunit.client.0.smithi012.stdout:4/980: chown d6/d27/c73 6426027 1 2024-03-18T14:51:10.970 INFO:tasks.workunit.client.1.smithi100.stdout:1/671: truncate d8/f29 1651886 0 2024-03-18T14:51:10.972 INFO:tasks.workunit.client.1.smithi100.stdout:3/699: mknod d3/dd/d49/cf3 0 2024-03-18T14:51:10.974 INFO:tasks.workunit.client.1.smithi100.stdout:4/652: dwrite d9/d27/d62/f9a [0,4194304] 0 2024-03-18T14:51:10.974 INFO:tasks.workunit.client.1.smithi100.stdout:3/700: fdatasync d3/d16/f14 0 2024-03-18T14:51:10.975 INFO:tasks.workunit.client.0.smithi012.stdout:4/981: mkdir d6/d1a/d30/d151 0 2024-03-18T14:51:10.975 INFO:tasks.workunit.client.0.smithi012.stdout:4/982: truncate d6/d1a/d98/daf/db5/f146 646443 0 2024-03-18T14:51:10.977 INFO:tasks.workunit.client.1.smithi100.stdout:0/717: mknod dd/d31/d91/d65/c106 0 2024-03-18T14:51:10.977 INFO:tasks.workunit.client.1.smithi100.stdout:0/718: write dd/d2b/d95/d79/d7d/da9/db0/fbe [1012842,2579] 0 2024-03-18T14:51:10.977 INFO:tasks.workunit.client.1.smithi100.stdout:0/719: chown dd 0 1 2024-03-18T14:51:10.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/687: dread d3/d6/f67 [0,4194304] 0 2024-03-18T14:51:10.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/688: fsync d3/da/de/fab 0 2024-03-18T14:51:10.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/689: fsync d3/da/de/d48/f8e 0 2024-03-18T14:51:10.979 INFO:tasks.workunit.client.1.smithi100.stdout:2/690: truncate d3/d6/d29/d36/d49/fb4 3999166 0 2024-03-18T14:51:10.980 INFO:tasks.workunit.client.1.smithi100.stdout:2/691: dread d3/fba [0,4194304] 0 2024-03-18T14:51:10.981 INFO:tasks.workunit.client.1.smithi100.stdout:9/709: mknod da/dd/d29/d71/d72/db2/ce3 0 2024-03-18T14:51:10.983 INFO:tasks.workunit.client.1.smithi100.stdout:5/648: rename d29/d2d/d32/l90 to d29/d2d/dbe/lcc 0 2024-03-18T14:51:10.983 INFO:tasks.workunit.client.1.smithi100.stdout:5/649: truncate d29/d2d/d39/d71/fb3 600154 0 2024-03-18T14:51:10.983 INFO:tasks.workunit.client.1.smithi100.stdout:5/650: write d29/d2d/d39/d83/dc1/dbb/f5f [1012005,124745] 0 2024-03-18T14:51:10.983 INFO:tasks.workunit.client.1.smithi100.stdout:5/651: readlink d29/d4a/l93 0 2024-03-18T14:51:10.987 INFO:tasks.workunit.client.1.smithi100.stdout:8/741: link d4/l3f d4/dc/d1f/d59/d6c/l109 0 2024-03-18T14:51:10.990 INFO:tasks.workunit.client.0.smithi012.stdout:1/885: sync 2024-03-18T14:51:10.990 INFO:tasks.workunit.client.1.smithi100.stdout:0/720: dread dd/f8f [0,4194304] 0 2024-03-18T14:51:10.990 INFO:tasks.workunit.client.1.smithi100.stdout:1/672: creat d8/d2a/d37/d50/d6f/d71/db7/fcd x:0 0 0 2024-03-18T14:51:10.992 INFO:tasks.workunit.client.0.smithi012.stdout:0/964: sync 2024-03-18T14:51:10.994 INFO:tasks.workunit.client.1.smithi100.stdout:4/653: truncate d9/d27/f98 539952 0 2024-03-18T14:51:10.996 INFO:tasks.workunit.client.1.smithi100.stdout:7/678: dwrite d5/d14/ff3 [0,4194304] 0 2024-03-18T14:51:10.996 INFO:tasks.workunit.client.1.smithi100.stdout:7/679: dread - d5/d22/d3b/d9f/da0/db1/fda zero size 2024-03-18T14:51:10.996 INFO:tasks.workunit.client.1.smithi100.stdout:7/680: dread - d5/d14/d16/ff4 zero size 2024-03-18T14:51:10.997 INFO:tasks.workunit.client.0.smithi012.stdout:4/983: dwrite d6/d19/ffe [0,4194304] 0 2024-03-18T14:51:10.999 INFO:tasks.workunit.client.0.smithi012.stdout:0/965: link d1/d16/d19/d8d/fad d1/d16/d19/dd0/d129/d67/f138 0 2024-03-18T14:51:10.999 INFO:tasks.workunit.client.0.smithi012.stdout:0/966: creat d1/d16/d19/f139 x:0 0 0 2024-03-18T14:51:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fbd59a700 1 -- 172.21.15.12:0/510396401 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb000e070 con 0x7f4fc00fbcd0 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 -- 172.21.15.12:0/510396401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fbcd0 msgr2=0x7f4fc00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/510396401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc00fc0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0004940 tx=0x7f4fb0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 -- 172.21.15.12:0/510396401 shutdown_connections 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/510396401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fcf80 0x7f4fc00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/510396401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc00fc0e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.002+0000 7f4fc4b00700 1 -- 172.21.15.12:0/510396401 >> 172.21.15.12:0/510396401 conn(0x7f4fc00f7430 msgr2=0x7f4fc00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 -- 172.21.15.12:0/510396401 shutdown_connections 2024-03-18T14:51:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 -- 172.21.15.12:0/510396401 wait complete. 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 Processor -- start 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 -- start start 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc0110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00663b0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.003+0000 7f4fc4b00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00664f0 con 0x7f4fc00fbcd0 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50336/0 (socket says 172.21.15.12:50336) 2024-03-18T14:51:11.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 -- 172.21.15.12:0/712650625 learned_addr learned my addr 172.21.15.12:0/712650625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbe59c700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc0110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 -- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fbcd0 msgr2=0x7f4fc0110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc0110630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 -- 172.21.15.12:0/712650625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb0005040 con 0x7f4fc00fcf80 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fbdd9b700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c8c0 tx=0x7f4fb40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb400a760 con 0x7f4fc00fcf80 2024-03-18T14:51:11.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0113180 con 0x7f4fc00fcf80 2024-03-18T14:51:11.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb400ad90 con 0x7f4fc00fcf80 2024-03-18T14:51:11.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.004+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb40113a0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.005+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc0113740 con 0x7f4fc00fcf80 2024-03-18T14:51:11.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.007+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc005f5f0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.007+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f4fb4012400 con 0x7f4fc00fcf80 2024-03-18T14:51:11.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.008+0000 7f4faeffd700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 0x7f4fa806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.008+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4fb408a1a0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.009+0000 7f4fbe59c700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 0x7f4fa806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.010 INFO:tasks.workunit.client.1.smithi100.stdout:0/721: dwrite dd/d31/d91/d56/d59/f7c [0,4194304] 0 2024-03-18T14:51:11.010 INFO:tasks.workunit.client.1.smithi100.stdout:0/722: fsync dd/d31/d91/d56/d62/fe2 0 2024-03-18T14:51:11.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.009+0000 7f4fbe59c700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 0x7f4fa806db60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009fd0 tx=0x7f4fb0009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:11.014 INFO:tasks.workunit.client.0.smithi012.stdout:1/886: dwrite d8/db/d10/d31/d43/d5e/da5/f10a [0,4194304] 0 2024-03-18T14:51:11.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.014+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f4fb4053ee0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.019 INFO:tasks.workunit.client.1.smithi100.stdout:2/692: truncate d3/da/de/d20/dea/d40/fa7 612072 0 2024-03-18T14:51:11.019 INFO:tasks.workunit.client.1.smithi100.stdout:2/693: readlink d3/d1a/lef 0 2024-03-18T14:51:11.021 INFO:tasks.workunit.client.1.smithi100.stdout:0/723: dread dd/d61/f72 [0,4194304] 0 2024-03-18T14:51:11.021 INFO:tasks.workunit.client.1.smithi100.stdout:9/710: stat da/l57 0 2024-03-18T14:51:11.022 INFO:tasks.workunit.client.0.smithi012.stdout:6/958: dwrite d17/d18/f23 [0,4194304] 0 2024-03-18T14:51:11.024 INFO:tasks.workunit.client.1.smithi100.stdout:5/652: symlink d29/d2d/d39/d83/dc1/dbb/d9e/lcd 0 2024-03-18T14:51:11.026 INFO:tasks.workunit.client.1.smithi100.stdout:1/673: getdents d8/d2a/d37/d50/d6f/d81 0 2024-03-18T14:51:11.033 INFO:tasks.workunit.client.0.smithi012.stdout:4/984: rename d6/d1a/d64/d12d/l14a to d6/d19/da5/d127/l152 0 2024-03-18T14:51:11.038 INFO:tasks.workunit.client.1.smithi100.stdout:3/701: mkdir d3/dd/d21/dcd/df4 0 2024-03-18T14:51:11.038 INFO:tasks.workunit.client.1.smithi100.stdout:3/702: dread - d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb zero size 2024-03-18T14:51:11.039 INFO:tasks.workunit.client.1.smithi100.stdout:4/654: read d9/d27/f98 [289758,24714] 0 2024-03-18T14:51:11.040 INFO:tasks.workunit.client.0.smithi012.stdout:0/967: mkdir d1/d13a 0 2024-03-18T14:51:11.041 INFO:tasks.workunit.client.0.smithi012.stdout:4/985: dread d6/d1a/d64/f78 [0,4194304] 0 2024-03-18T14:51:11.046 INFO:tasks.workunit.client.0.smithi012.stdout:1/887: symlink d8/db/d10/d31/d43/d5e/da5/l11b 0 2024-03-18T14:51:11.049 INFO:tasks.workunit.client.1.smithi100.stdout:4/655: dread d9/d3a/f99 [0,4194304] 0 2024-03-18T14:51:11.050 INFO:tasks.workunit.client.0.smithi012.stdout:4/986: write d6/f8c [1799011,123645] 0 2024-03-18T14:51:11.051 INFO:tasks.workunit.client.1.smithi100.stdout:4/656: read d9/d13/d25/f48 [219691,87411] 0 2024-03-18T14:51:11.053 INFO:tasks.workunit.client.0.smithi012.stdout:6/959: mknod d17/d38/d51/c121 0 2024-03-18T14:51:11.055 INFO:tasks.workunit.client.1.smithi100.stdout:0/724: creat dd/d2b/d95/d79/d7d/f107 x:0 0 0 2024-03-18T14:51:11.057 INFO:tasks.workunit.client.0.smithi012.stdout:0/968: unlink d1/d16/d31/f50 0 2024-03-18T14:51:11.057 INFO:tasks.workunit.client.0.smithi012.stdout:0/969: dread - d1/d20/d23/d6f/d93/f100 zero size 2024-03-18T14:51:11.059 INFO:tasks.workunit.client.1.smithi100.stdout:9/711: creat da/dd/d29/d9e/fe4 x:0 0 0 2024-03-18T14:51:11.059 INFO:tasks.workunit.client.1.smithi100.stdout:9/712: chown da/d46/d4d/d67/f6e 45 1 2024-03-18T14:51:11.061 INFO:tasks.workunit.client.0.smithi012.stdout:1/888: write d8/db/f7d [3803223,30789] 0 2024-03-18T14:51:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:1/889: write d8/d1e/d27/f38 [2524656,23001] 0 2024-03-18T14:51:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:1/890: creat d8/dbf/f11c x:0 0 0 2024-03-18T14:51:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:1/891: chown d8/db/d10/d31/d45/f6d 5 1 2024-03-18T14:51:11.072 INFO:tasks.workunit.client.0.smithi012.stdout:1/892: stat d8/db/d10/d56/f8f 0 2024-03-18T14:51:11.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/653: creat d29/d2d/dbe/dc2/fce x:0 0 0 2024-03-18T14:51:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/987: dwrite d6/da/d89/d97/fa9 [0,4194304] 0 2024-03-18T14:51:11.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/988: truncate d6/d19/d39/fec 97430 0 2024-03-18T14:51:11.089 INFO:tasks.workunit.client.1.smithi100.stdout:9/713: dwrite da/fa4 [4194304,4194304] 0 2024-03-18T14:51:11.089 INFO:tasks.workunit.client.1.smithi100.stdout:9/714: dread - da/dcb/fe2 zero size 2024-03-18T14:51:11.089 INFO:tasks.workunit.client.1.smithi100.stdout:9/715: creat da/d2c/d2f/fe5 x:0 0 0 2024-03-18T14:51:11.089 INFO:tasks.workunit.client.1.smithi100.stdout:9/716: creat da/db/fe6 x:0 0 0 2024-03-18T14:51:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:4/989: dread d6/f11e [0,4194304] 0 2024-03-18T14:51:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:6/960: dwrite d17/d19/d27/d58/d91/ff6 [0,4194304] 0 2024-03-18T14:51:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:6/961: write d17/d38/d51/fa9 [1844533,98736] 0 2024-03-18T14:51:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:6/962: chown d17/d26/c29 52 1 2024-03-18T14:51:11.090 INFO:tasks.workunit.client.0.smithi012.stdout:6/963: chown d17/d38/d51/d66/d7d 0 1 2024-03-18T14:51:11.092 INFO:tasks.workunit.client.0.smithi012.stdout:6/964: read d17/d38/d51/fa9 [386580,82681] 0 2024-03-18T14:51:11.109 INFO:tasks.workunit.client.0.smithi012.stdout:6/965: dwrite d17/d19/d27/d58/d91/fca [0,4194304] 0 2024-03-18T14:51:11.158 INFO:tasks.workunit.client.1.smithi100.stdout:1/674: rename d8/d1e/d11/d27/d3e/l6a to d8/d2a/d37/d50/d6f/d71/lce 0 2024-03-18T14:51:11.161 INFO:tasks.workunit.client.0.smithi012.stdout:5/896: sync 2024-03-18T14:51:11.161 INFO:tasks.workunit.client.0.smithi012.stdout:5/897: fdatasync d3/d29/d36/fe3 0 2024-03-18T14:51:11.161 INFO:tasks.workunit.client.0.smithi012.stdout:7/982: sync 2024-03-18T14:51:11.166 INFO:tasks.workunit.client.1.smithi100.stdout:1/675: dread d8/d2a/f44 [0,4194304] 0 2024-03-18T14:51:11.166 INFO:tasks.workunit.client.1.smithi100.stdout:1/676: write d8/d2a/fa9 [536471,69335] 0 2024-03-18T14:51:11.167 INFO:tasks.workunit.client.1.smithi100.stdout:2/694: getdents d3/da/de/d20/d82 0 2024-03-18T14:51:11.168 INFO:tasks.workunit.client.1.smithi100.stdout:2/695: truncate d3/da/de/d6e/fcd 881416 0 2024-03-18T14:51:11.187 INFO:tasks.workunit.client.0.smithi012.stdout:0/970: dwrite d1/d20/d23/d82/fb8 [4194304,4194304] 0 2024-03-18T14:51:11.188 INFO:tasks.workunit.client.1.smithi100.stdout:2/696: dwrite d3/da/de/fa5 [0,4194304] 0 2024-03-18T14:51:11.210 INFO:tasks.workunit.client.1.smithi100.stdout:9/717: creat da/d2c/fe7 x:0 0 0 2024-03-18T14:51:11.212 INFO:tasks.workunit.client.0.smithi012.stdout:4/990: rename d6/d1a/d64/f78 to d6/da/f153 0 2024-03-18T14:51:11.329 INFO:tasks.workunit.client.1.smithi100.stdout:1/677: truncate d8/d2a/f41 1872482 0 2024-03-18T14:51:11.340 INFO:tasks.workunit.client.1.smithi100.stdout:2/697: mknod d3/d6/d5f/d6a/cf0 0 2024-03-18T14:51:11.344 INFO:tasks.workunit.client.1.smithi100.stdout:9/718: creat da/d46/dd6/fe8 x:0 0 0 2024-03-18T14:51:11.345 INFO:tasks.workunit.client.1.smithi100.stdout:9/719: write da/dcb/fe2 [533005,121466] 0 2024-03-18T14:51:11.350 INFO:tasks.workunit.client.1.smithi100.stdout:2/698: creat d3/da/de/d20/dea/ff1 x:0 0 0 2024-03-18T14:51:11.351 INFO:tasks.workunit.client.1.smithi100.stdout:2/699: stat d3/da/de/d48/c81 0 2024-03-18T14:51:11.351 INFO:tasks.workunit.client.1.smithi100.stdout:2/700: creat d3/dc3/ff2 x:0 0 0 2024-03-18T14:51:11.351 INFO:tasks.workunit.client.1.smithi100.stdout:2/701: chown d3/da/de/d20/f23 67 1 2024-03-18T14:51:11.357 INFO:tasks.workunit.client.1.smithi100.stdout:2/702: write d3/da/de/d20/dea/d4f/fc2 [3087300,75667] 0 2024-03-18T14:51:11.358 INFO:tasks.workunit.client.1.smithi100.stdout:2/703: dread d3/da/f33 [0,4194304] 0 2024-03-18T14:51:11.365 INFO:tasks.workunit.client.1.smithi100.stdout:2/704: unlink d3/da/de/d20/d82/dad/dc5/de3/le7 0 2024-03-18T14:51:11.368 INFO:tasks.workunit.client.0.smithi012.stdout:7/983: unlink dd/d13/d23/d2f/d53/c87 0 2024-03-18T14:51:11.376 INFO:tasks.workunit.client.0.smithi012.stdout:4/991: creat d6/d1a/d98/daf/f154 x:0 0 0 2024-03-18T14:51:11.381 INFO:tasks.workunit.client.0.smithi012.stdout:6/966: rename d17/d38/d51/da8/d100/dd8/c4c to d17/d19/d27/c122 0 2024-03-18T14:51:11.381 INFO:tasks.workunit.client.0.smithi012.stdout:6/967: rename d17/d19/d27/d6e/d84 to d17/d19/d27/d6e/d84/d123 22 2024-03-18T14:51:11.388 INFO:tasks.workunit.client.0.smithi012.stdout:4/992: mknod d6/da/d89/d94/da2/c155 0 2024-03-18T14:51:11.388 INFO:tasks.workunit.client.0.smithi012.stdout:4/993: stat d6/da/d89/d97/dc5/c11b 0 2024-03-18T14:51:11.390 INFO:tasks.workunit.client.0.smithi012.stdout:7/984: dwrite dd/d13/d23/d76/d6b/d14a/ff9 [0,4194304] 0 2024-03-18T14:51:11.393 INFO:tasks.workunit.client.0.smithi012.stdout:7/985: symlink dd/d13/d23/d76/d109/l14b 0 2024-03-18T14:51:11.393 INFO:tasks.workunit.client.0.smithi012.stdout:7/986: read dd/d13/d23/d76/f68 [2085847,42419] 0 2024-03-18T14:51:11.395 INFO:tasks.workunit.client.0.smithi012.stdout:6/968: dwrite d17/d19/d27/db1/dd7/f10b [0,4194304] 0 2024-03-18T14:51:11.396 INFO:tasks.workunit.client.0.smithi012.stdout:6/969: fdatasync d17/d19/f2b 0 2024-03-18T14:51:11.400 INFO:tasks.workunit.client.0.smithi012.stdout:6/970: mkdir d17/d38/d51/d66/dee/d103/d124 0 2024-03-18T14:51:11.400 INFO:tasks.workunit.client.0.smithi012.stdout:6/971: fdatasync d17/d18/f94 0 2024-03-18T14:51:11.414 INFO:tasks.workunit.client.0.smithi012.stdout:7/987: symlink dd/d13/l14c 0 2024-03-18T14:51:11.414 INFO:tasks.workunit.client.0.smithi012.stdout:7/988: chown dd/d13/d23/fdc 147522258 1 2024-03-18T14:51:11.415 INFO:tasks.workunit.client.0.smithi012.stdout:7/989: getdents dd/d13/d23/d60/d140/dc7/d136 0 2024-03-18T14:51:11.415 INFO:tasks.workunit.client.0.smithi012.stdout:7/990: write dd/d13/d23/d60/d140/dc7/f12e [865381,74868] 0 2024-03-18T14:51:11.415 INFO:tasks.workunit.client.0.smithi012.stdout:7/991: write dd/d13/d23/d76/d6f/f133 [419440,30046] 0 2024-03-18T14:51:11.419 INFO:tasks.workunit.client.1.smithi100.stdout:3/703: unlink d3/ca0 0 2024-03-18T14:51:11.421 INFO:tasks.workunit.client.0.smithi012.stdout:6/972: dwrite d17/d19/d27/d6e/f107 [0,4194304] 0 2024-03-18T14:51:11.421 INFO:tasks.workunit.client.0.smithi012.stdout:6/973: rename d17/d38 to d17/d38/d51/d66/d7d/d125 22 2024-03-18T14:51:11.425 INFO:tasks.workunit.client.0.smithi012.stdout:6/974: mkdir d17/d38/d51/d66/dee/d103/d126 0 2024-03-18T14:51:11.429 INFO:tasks.workunit.client.0.smithi012.stdout:6/975: dread d17/d19/d27/fd6 [0,4194304] 0 2024-03-18T14:51:11.429 INFO:tasks.workunit.client.0.smithi012.stdout:6/976: truncate d17/d38/d51/d72/f74 5004155 0 2024-03-18T14:51:11.429 INFO:tasks.workunit.client.0.smithi012.stdout:6/977: chown d17/c9b 132992484 1 2024-03-18T14:51:11.431 INFO:tasks.workunit.client.0.smithi012.stdout:6/978: mkdir d17/d38/d51/d66/dee/d127 0 2024-03-18T14:51:11.433 INFO:tasks.workunit.client.0.smithi012.stdout:6/979: unlink d17/d26/c76 0 2024-03-18T14:51:11.447 INFO:tasks.workunit.client.0.smithi012.stdout:7/992: dwrite dd/d13/d23/d76/d109/dfa/f123 [0,4194304] 0 2024-03-18T14:51:11.449 INFO:tasks.workunit.client.1.smithi100.stdout:3/704: dwrite d3/d16/d13/d71/d74/fea [0,4194304] 0 2024-03-18T14:51:11.449 INFO:tasks.workunit.client.1.smithi100.stdout:3/705: fdatasync d3/d16/d13/d1f/d3f/d4f/f68 0 2024-03-18T14:51:11.450 INFO:tasks.workunit.client.0.smithi012.stdout:6/980: dwrite d17/d26/fbc [0,4194304] 0 2024-03-18T14:51:11.452 INFO:tasks.workunit.client.0.smithi012.stdout:7/993: symlink dd/d13/d23/d60/d140/dc7/d136/l14d 0 2024-03-18T14:51:11.453 INFO:tasks.workunit.client.0.smithi012.stdout:7/994: chown dd/d13/d23/f55 1 1 2024-03-18T14:51:11.453 INFO:tasks.workunit.client.0.smithi012.stdout:7/995: fsync dd/d13/d23/fce 0 2024-03-18T14:51:11.456 INFO:tasks.workunit.client.1.smithi100.stdout:3/706: dread d3/d16/d13/d17/f4c [0,4194304] 0 2024-03-18T14:51:11.457 INFO:tasks.workunit.client.1.smithi100.stdout:9/720: symlink da/dd/d29/d71/le9 0 2024-03-18T14:51:11.461 INFO:tasks.workunit.client.0.smithi012.stdout:6/981: creat d17/d38/d51/d66/dee/d103/f128 x:0 0 0 2024-03-18T14:51:11.461 INFO:tasks.workunit.client.0.smithi012.stdout:6/982: dread - d17/d38/d5b/ff0 zero size 2024-03-18T14:51:11.463 INFO:tasks.workunit.client.1.smithi100.stdout:3/707: mknod d3/d16/d13/d1f/d3f/d4f/de0/de6/cf5 0 2024-03-18T14:51:11.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/721: dread da/db/f4c [0,4194304] 0 2024-03-18T14:51:11.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/722: dread - da/dd/d29/d71/f98 zero size 2024-03-18T14:51:11.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/723: write da/db/fe6 [999395,60188] 0 2024-03-18T14:51:11.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/724: dread - da/dd/da0/fa6 zero size 2024-03-18T14:51:11.464 INFO:tasks.workunit.client.1.smithi100.stdout:9/725: stat da/d46/d4d/d67 0 2024-03-18T14:51:11.465 INFO:tasks.workunit.client.1.smithi100.stdout:9/726: creat da/d2c/d2f/fea x:0 0 0 2024-03-18T14:51:11.467 INFO:tasks.workunit.client.0.smithi012.stdout:6/983: truncate d17/d19/d27/d58/f6f 3966132 0 2024-03-18T14:51:11.467 INFO:tasks.workunit.client.0.smithi012.stdout:6/984: chown d17/d26/d36/d4d/cc5 127304846 1 2024-03-18T14:51:11.471 INFO:tasks.workunit.client.0.smithi012.stdout:6/985: mkdir d17/d38/d118/d129 0 2024-03-18T14:51:11.471 INFO:tasks.workunit.client.0.smithi012.stdout:6/986: stat d17/d26/d36/d4d/d82/d97/ffb 0 2024-03-18T14:51:11.473 INFO:tasks.workunit.client.0.smithi012.stdout:6/987: link d17/d19/d27/db1/dd7/fe1 d17/d26/d36/d4d/f12a 0 2024-03-18T14:51:11.473 INFO:tasks.workunit.client.0.smithi012.stdout:6/988: write d17/d38/d51/d66/d7d/fff [47684,27019] 0 2024-03-18T14:51:11.476 INFO:tasks.workunit.client.0.smithi012.stdout:5/898: link d3/d84/dfa/f10c d3/d15/d42/dc6/d73/d6b/f12f 0 2024-03-18T14:51:11.477 INFO:tasks.workunit.client.0.smithi012.stdout:5/899: creat d3/d15/d42/dc6/dfe/f130 x:0 0 0 2024-03-18T14:51:11.484 INFO:tasks.workunit.client.0.smithi012.stdout:6/989: dread d17/d26/f8c [0,4194304] 0 2024-03-18T14:51:11.485 INFO:tasks.workunit.client.1.smithi100.stdout:3/708: dwrite d3/dd/d49/fb7 [0,4194304] 0 2024-03-18T14:51:11.485 INFO:tasks.workunit.client.1.smithi100.stdout:3/709: chown d3/dd/d21/d3e/c43 115171282 1 2024-03-18T14:51:11.486 INFO:tasks.workunit.client.0.smithi012.stdout:6/990: link d17/d38/d51/da8/cbe d17/d38/d51/da8/d112/c12b 0 2024-03-18T14:51:11.486 INFO:tasks.workunit.client.0.smithi012.stdout:6/991: readlink d17/d38/d51/da8/d100/dd8/lb6 0 2024-03-18T14:51:11.486 INFO:tasks.workunit.client.0.smithi012.stdout:6/992: write d17/d19/f3e [1864078,12158] 0 2024-03-18T14:51:11.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:11 smithi012 ceph-mon[29182]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:11.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:11 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:11.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:11 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:11.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:11 smithi012 ceph-mon[29182]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:11.494 INFO:tasks.workunit.client.1.smithi100.stdout:5/654: rename d29/d2d/d39/lab to d29/d2d/d39/d74/lcf 0 2024-03-18T14:51:11.494 INFO:tasks.workunit.client.1.smithi100.stdout:5/655: dread - d29/d2d/d39/d74/fa5 zero size 2024-03-18T14:51:11.497 INFO:tasks.workunit.client.1.smithi100.stdout:3/710: read d3/dd/d49/fb7 [897066,44415] 0 2024-03-18T14:51:11.498 INFO:tasks.workunit.client.0.smithi012.stdout:5/900: dwrite d3/f9e [0,4194304] 0 2024-03-18T14:51:11.498 INFO:tasks.workunit.client.1.smithi100.stdout:4/657: rmdir d9/d13 39 2024-03-18T14:51:11.500 INFO:tasks.workunit.client.0.smithi012.stdout:0/971: write d1/f10 [3159976,91583] 0 2024-03-18T14:51:11.502 INFO:tasks.workunit.client.0.smithi012.stdout:6/993: dread d17/d19/f3e [0,4194304] 0 2024-03-18T14:51:11.502 INFO:tasks.workunit.client.0.smithi012.stdout:6/994: fsync d17/d19/d27/d58/d91/fc3 0 2024-03-18T14:51:11.504 INFO:tasks.workunit.client.1.smithi100.stdout:1/678: rename d8/d1e/d11/d6d/f7f to d8/d30/d61/fcf 0 2024-03-18T14:51:11.504 INFO:tasks.workunit.client.1.smithi100.stdout:1/679: dread - d8/d30/d61/fcb zero size 2024-03-18T14:51:11.504 INFO:tasks.workunit.client.1.smithi100.stdout:1/680: creat d8/d2a/d37/d50/d6f/d71/fd0 x:0 0 0 2024-03-18T14:51:11.504 INFO:tasks.workunit.client.1.smithi100.stdout:1/681: fsync d8/d1e/d11/d27/f4d 0 2024-03-18T14:51:11.505 INFO:tasks.workunit.client.1.smithi100.stdout:5/656: mkdir d29/dd0 0 2024-03-18T14:51:11.508 INFO:tasks.workunit.client.0.smithi012.stdout:5/901: creat d3/db6/df0/f131 x:0 0 0 2024-03-18T14:51:11.508 INFO:tasks.workunit.client.1.smithi100.stdout:1/682: write d8/d1e/d11/f77 [4897989,2082] 0 2024-03-18T14:51:11.508 INFO:tasks.workunit.client.1.smithi100.stdout:1/683: truncate d8/d2a/d37/d50/d6f/d71/fad 1043731 0 2024-03-18T14:51:11.509 INFO:tasks.workunit.client.1.smithi100.stdout:3/711: symlink d3/d16/d13/d17/lf6 0 2024-03-18T14:51:11.510 INFO:tasks.workunit.client.0.smithi012.stdout:1/893: symlink d8/db/d10/d31/d43/d5e/l11d 0 2024-03-18T14:51:11.515 INFO:tasks.workunit.client.1.smithi100.stdout:4/658: symlink d9/d13/d39/dc0/lc5 0 2024-03-18T14:51:11.518 INFO:tasks.workunit.client.1.smithi100.stdout:4/659: dread d9/d17/f23 [0,4194304] 0 2024-03-18T14:51:11.518 INFO:tasks.workunit.client.0.smithi012.stdout:0/972: symlink d1/d20/d23/d26/l13b 0 2024-03-18T14:51:11.518 INFO:tasks.workunit.client.0.smithi012.stdout:1/894: read d8/db/d10/d16/da1/d2e/d9c/fb1 [2171057,53724] 0 2024-03-18T14:51:11.525 INFO:tasks.workunit.client.0.smithi012.stdout:6/995: mknod d17/d19/d27/c12c 0 2024-03-18T14:51:11.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.526+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4fc004bed0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.527+0000 7f4faeffd700 1 -- 172.21.15.12:0/712650625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f4fb40575a0 con 0x7f4fc00fcf80 2024-03-18T14:51:11.530 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 1, 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:51:11.531 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 13, 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:51:11.532 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:51:11.532 INFO:tasks.workunit.client.0.smithi012.stdout:5/902: dwrite d3/d29/d35/f5a [0,4194304] 0 2024-03-18T14:51:11.533 INFO:tasks.workunit.client.1.smithi100.stdout:2/705: rename d3/da/de/d20/d82/dad/de2/fec to d3/da/de/d6e/ff3 0 2024-03-18T14:51:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 msgr2=0x7f4fa806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 0x7f4fa806db60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009fd0 tx=0x7f4fb0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 msgr2=0x7f4fc0112b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c8c0 tx=0x7f4fb40044c0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 shutdown_connections 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f4fa806b6b0 0x7f4fa806db60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc00fbcd0 0x7f4fc0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 --2- 172.21.15.12:0/712650625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc00fcf80 0x7f4fc0112b80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 >> 172.21.15.12:0/712650625 conn(0x7f4fc00f7430 msgr2=0x7f4fc00f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 shutdown_connections 2024-03-18T14:51:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.532+0000 7f4fc4b00700 1 -- 172.21.15.12:0/712650625 wait complete. 2024-03-18T14:51:11.534 INFO:tasks.workunit.client.1.smithi100.stdout:5/657: symlink d29/d2d/d39/d60/dbf/ld1 0 2024-03-18T14:51:11.536 INFO:tasks.workunit.client.1.smithi100.stdout:1/684: mkdir d8/d1e/d11/d27/d3f/daa/dd1 0 2024-03-18T14:51:11.537 INFO:tasks.workunit.client.0.smithi012.stdout:1/895: getdents d8/dbf 0 2024-03-18T14:51:11.539 INFO:tasks.workunit.client.1.smithi100.stdout:3/712: creat d3/ff7 x:0 0 0 2024-03-18T14:51:11.541 INFO:tasks.workunit.client.0.smithi012.stdout:5/903: truncate d3/d15/d42/dc6/d73/f7e 2062300 0 2024-03-18T14:51:11.543 INFO:tasks.workunit.client.1.smithi100.stdout:4/660: unlink d9/d27/d62/d7b/l9d 0 2024-03-18T14:51:11.543 INFO:tasks.workunit.client.1.smithi100.stdout:4/661: write d9/d13/d39/f8f [976006,47364] 0 2024-03-18T14:51:11.543 INFO:tasks.workunit.client.1.smithi100.stdout:4/662: fdatasync d9/d3a/f69 0 2024-03-18T14:51:11.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:11 smithi100 ceph-mon[37807]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:11.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:11 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:11.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:11 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:11.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:11 smithi100 ceph-mon[37807]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:11.545 INFO:tasks.workunit.client.0.smithi012.stdout:1/896: truncate d8/db/d10/d31/d43/d5e/f62 432867 0 2024-03-18T14:51:11.547 INFO:tasks.workunit.client.1.smithi100.stdout:9/727: rename da/dd/f73 to da/db/db0/feb 0 2024-03-18T14:51:11.549 INFO:tasks.workunit.client.0.smithi012.stdout:0/973: dwrite d1/f9c [0,4194304] 0 2024-03-18T14:51:11.549 INFO:tasks.workunit.client.0.smithi012.stdout:0/974: fdatasync d1/d20/d23/d26/f90 0 2024-03-18T14:51:11.549 INFO:tasks.workunit.client.0.smithi012.stdout:0/975: creat d1/d20/d23/d3b/dc1/f13c x:0 0 0 2024-03-18T14:51:11.549 INFO:tasks.workunit.client.0.smithi012.stdout:6/996: dwrite d17/d26/d36/d105/f113 [0,4194304] 0 2024-03-18T14:51:11.549 INFO:tasks.workunit.client.0.smithi012.stdout:6/997: creat d17/d26/d36/d105/f12d x:0 0 0 2024-03-18T14:51:11.551 INFO:tasks.workunit.client.1.smithi100.stdout:5/658: mknod d29/d2d/d8f/cd2 0 2024-03-18T14:51:11.551 INFO:tasks.workunit.client.1.smithi100.stdout:5/659: write d29/fa0 [250454,1540] 0 2024-03-18T14:51:11.552 INFO:tasks.workunit.client.1.smithi100.stdout:1/685: mknod d8/d30/cd2 0 2024-03-18T14:51:11.554 INFO:tasks.workunit.client.0.smithi012.stdout:1/897: link d8/db/d10/d16/da1/d2e/l6a d8/db/d10/d31/d43/d5e/d65/l11e 0 2024-03-18T14:51:11.556 INFO:tasks.workunit.client.1.smithi100.stdout:4/663: link d9/d13/d5a/fbc d9/d3a/fc6 0 2024-03-18T14:51:11.556 INFO:tasks.workunit.client.1.smithi100.stdout:4/664: chown d9/d27/l3e 134318583 1 2024-03-18T14:51:11.560 INFO:tasks.workunit.client.1.smithi100.stdout:2/706: rename d3/da/c2e to d3/da/de/d48/cf4 0 2024-03-18T14:51:11.564 INFO:tasks.workunit.client.0.smithi012.stdout:1/898: dread d8/d1e/f2f [0,4194304] 0 2024-03-18T14:51:11.565 INFO:tasks.workunit.client.0.smithi012.stdout:7/996: rename dd/d13/d23/d2f/fb9 to dd/d13/d23/d60/d9c/dfd/f14e 0 2024-03-18T14:51:11.572 INFO:tasks.workunit.client.1.smithi100.stdout:7/681: mkdir d5/d14/d16/d2e/d62/dd1/d107 0 2024-03-18T14:51:11.574 INFO:tasks.workunit.client.0.smithi012.stdout:6/998: creat d17/d38/d51/da8/d100/dd8/f12e x:0 0 0 2024-03-18T14:51:11.576 INFO:tasks.workunit.client.0.smithi012.stdout:1/899: creat d8/db/f11f x:0 0 0 2024-03-18T14:51:11.579 INFO:tasks.workunit.client.1.smithi100.stdout:2/707: getdents d3/d6/d29/d36/d4a 0 2024-03-18T14:51:11.579 INFO:tasks.workunit.client.1.smithi100.stdout:1/686: dwrite d8/d1e/d11/d27/d73/d89/fa4 [4194304,4194304] 0 2024-03-18T14:51:11.579 INFO:tasks.workunit.client.1.smithi100.stdout:1/687: stat d8/d30/d61/f63 0 2024-03-18T14:51:11.579 INFO:tasks.workunit.client.1.smithi100.stdout:1/688: truncate d8/d2a/fa9 1748627 0 2024-03-18T14:51:11.579 INFO:tasks.workunit.client.1.smithi100.stdout:1/689: write d8/d2a/fa9 [2574073,42079] 0 2024-03-18T14:51:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:0/976: dwrite d1/d16/d19/d8d/fad [0,4194304] 0 2024-03-18T14:51:11.580 INFO:tasks.workunit.client.0.smithi012.stdout:6/999: dread d17/d19/d98/fd2 [0,4194304] 0 2024-03-18T14:51:11.580 INFO:tasks.workunit.client.1.smithi100.stdout:4/665: dwrite d9/d13/d25/f4d [4194304,4194304] 0 2024-03-18T14:51:11.580 INFO:tasks.workunit.client.1.smithi100.stdout:4/666: dread - d9/d3a/f64 zero size 2024-03-18T14:51:11.580 INFO:tasks.workunit.client.1.smithi100.stdout:4/667: truncate d9/d27/da2/fc3 818781 0 2024-03-18T14:51:11.581 INFO:tasks.workunit.client.1.smithi100.stdout:7/682: symlink d5/d22/d3b/d47/l108 0 2024-03-18T14:51:11.583 INFO:tasks.workunit.client.0.smithi012.stdout:5/904: rename d3/l46 to d3/db6/df0/l132 0 2024-03-18T14:51:11.585 INFO:tasks.workunit.client.1.smithi100.stdout:3/713: rename d3/d16/d13/d71 to d3/dd/d21/d3e/df8 0 2024-03-18T14:51:11.585 INFO:tasks.workunit.client.1.smithi100.stdout:3/714: chown d3/fb6 1762563569 1 2024-03-18T14:51:11.586 INFO:tasks.workunit.client.0.smithi012.stdout:1/900: mkdir d8/db/d10/d31/d43/d5e/d65/d120 0 2024-03-18T14:51:11.588 INFO:tasks.workunit.client.1.smithi100.stdout:2/708: write d3/da/de/d20/f23 [154158,30755] 0 2024-03-18T14:51:11.597 INFO:tasks.workunit.client.0.smithi012.stdout:5/905: mkdir d3/d29/d35/dcc/d133 0 2024-03-18T14:51:11.597 INFO:tasks.workunit.client.0.smithi012.stdout:5/906: chown d3/d84/f9f 7047 1 2024-03-18T14:51:11.597 INFO:tasks.workunit.client.0.smithi012.stdout:5/907: read - d3/d29/d35/d5b/f12e zero size 2024-03-18T14:51:11.603 INFO:tasks.workunit.client.0.smithi012.stdout:7/997: dread dd/d13/d23/d2f/f32 [0,4194304] 0 2024-03-18T14:51:11.606 INFO:tasks.workunit.client.1.smithi100.stdout:4/668: mknod d9/d13/d5a/cc7 0 2024-03-18T14:51:11.606 INFO:tasks.workunit.client.1.smithi100.stdout:4/669: chown d9/d13/d5a/d5b/da0/cb0 16 1 2024-03-18T14:51:11.607 INFO:tasks.workunit.client.0.smithi012.stdout:7/998: mknod dd/d13/d23/d2f/d53/d8d/c14f 0 2024-03-18T14:51:11.609 INFO:tasks.workunit.client.1.smithi100.stdout:5/660: rename d29/d4a/f80 to d29/d2d/d39/d71/fd3 0 2024-03-18T14:51:11.613 INFO:tasks.workunit.client.1.smithi100.stdout:1/690: dwrite d8/d2a/d37/d50/d6f/d71/db7/fcd [0,4194304] 0 2024-03-18T14:51:11.616 INFO:tasks.workunit.client.1.smithi100.stdout:2/709: link d3/d6/d29/fda d3/d6/d29/d36/d4a/ff5 0 2024-03-18T14:51:11.619 INFO:tasks.workunit.client.1.smithi100.stdout:5/661: rename f17 to d29/d2d/fd4 0 2024-03-18T14:51:11.620 INFO:tasks.workunit.client.0.smithi012.stdout:7/999: dread dd/d13/d23/d76/f3e [0,4194304] 0 2024-03-18T14:51:11.622 INFO:tasks.workunit.client.1.smithi100.stdout:1/691: symlink d8/d2a/d37/d50/d6f/d71/dc9/ld3 0 2024-03-18T14:51:11.622 INFO:tasks.workunit.client.1.smithi100.stdout:1/692: stat d8/d1e/d11/c46 0 2024-03-18T14:51:11.624 INFO:tasks.workunit.client.1.smithi100.stdout:2/710: mknod d3/da/cf6 0 2024-03-18T14:51:11.624 INFO:tasks.workunit.client.1.smithi100.stdout:2/711: write d3/d6/d5f/f68 [1201266,112777] 0 2024-03-18T14:51:11.625 INFO:tasks.workunit.client.1.smithi100.stdout:2/712: rmdir d3/da/de/d20/dea/d40/d53 39 2024-03-18T14:51:11.628 INFO:tasks.workunit.client.1.smithi100.stdout:2/713: rmdir d3/da/de/d20/dea 39 2024-03-18T14:51:11.632 INFO:tasks.workunit.client.1.smithi100.stdout:5/662: read d29/d2d/d39/d60/f70 [605104,42324] 0 2024-03-18T14:51:11.634 INFO:tasks.workunit.client.1.smithi100.stdout:5/663: mknod d29/d2d/dbe/cd5 0 2024-03-18T14:51:11.635 INFO:tasks.workunit.client.1.smithi100.stdout:4/670: dwrite d9/d13/d5a/d5b/f9e [0,4194304] 0 2024-03-18T14:51:11.635 INFO:tasks.workunit.client.0.smithi012.stdout:1/901: dwrite d8/db/d10/f33 [0,4194304] 0 2024-03-18T14:51:11.635 INFO:tasks.workunit.client.0.smithi012.stdout:1/902: fdatasync d8/db/d10/d31/d81/f11a 0 2024-03-18T14:51:11.637 INFO:tasks.workunit.client.0.smithi012.stdout:0/977: dwrite d1/d16/d6e/f9b [0,4194304] 0 2024-03-18T14:51:11.637 INFO:tasks.workunit.client.1.smithi100.stdout:7/683: dwrite d5/d14/fab [0,4194304] 0 2024-03-18T14:51:11.638 INFO:tasks.workunit.client.0.smithi012.stdout:5/908: dwrite d3/d29/d9d/dba/fd7 [0,4194304] 0 2024-03-18T14:51:11.640 INFO:tasks.workunit.client.1.smithi100.stdout:5/664: creat d29/d2d/d32/fd6 x:0 0 0 2024-03-18T14:51:11.650 INFO:tasks.workunit.client.1.smithi100.stdout:1/693: dwrite d8/d1e/d11/fc3 [0,4194304] 0 2024-03-18T14:51:11.650 INFO:tasks.workunit.client.0.smithi012.stdout:1/903: dread d8/db/d10/d16/da1/d2e/d9c/d9d/fdb [0,4194304] 0 2024-03-18T14:51:11.650 INFO:tasks.workunit.client.0.smithi012.stdout:1/904: write d8/d1e/d27/d44/dc2/f10b [535580,53743] 0 2024-03-18T14:51:11.652 INFO:tasks.workunit.client.1.smithi100.stdout:7/684: creat d5/d22/d24/d6e/d105/f109 x:0 0 0 2024-03-18T14:51:11.653 INFO:tasks.workunit.client.0.smithi012.stdout:5/909: mkdir d3/d15/d42/dc6/d65/d134 0 2024-03-18T14:51:11.655 INFO:tasks.workunit.client.1.smithi100.stdout:5/665: dread d29/d2d/d32/d92/daa/fa1 [0,4194304] 0 2024-03-18T14:51:11.655 INFO:tasks.workunit.client.1.smithi100.stdout:5/666: dread - d29/d2d/d32/d92/f9a zero size 2024-03-18T14:51:11.656 INFO:tasks.workunit.client.1.smithi100.stdout:2/714: dwrite d3/d6/fa9 [0,4194304] 0 2024-03-18T14:51:11.657 INFO:tasks.workunit.client.0.smithi012.stdout:1/905: symlink d8/db/d10/d31/d43/d5e/d65/d70/dc7/l121 0 2024-03-18T14:51:11.658 INFO:tasks.workunit.client.0.smithi012.stdout:0/978: dwrite d1/d2d/f62 [0,4194304] 0 2024-03-18T14:51:11.662 INFO:tasks.workunit.client.0.smithi012.stdout:5/910: mkdir d3/db6/df0/d135 0 2024-03-18T14:51:11.662 INFO:tasks.workunit.client.0.smithi012.stdout:5/911: stat d3/d15/d42/dc6/d65/fa1 0 2024-03-18T14:51:11.662 INFO:tasks.workunit.client.0.smithi012.stdout:5/912: write d3/d15/d42/dc6/d73/d6b/d8f/d113/f11d [760516,59605] 0 2024-03-18T14:51:11.664 INFO:tasks.workunit.client.1.smithi100.stdout:1/694: dwrite d8/d1e/d11/d6d/f88 [0,4194304] 0 2024-03-18T14:51:11.666 INFO:tasks.workunit.client.1.smithi100.stdout:7/685: creat d5/d14/d16/d2e/d62/d66/d4f/dcb/f10a x:0 0 0 2024-03-18T14:51:11.667 INFO:tasks.workunit.client.0.smithi012.stdout:1/906: rename d8/dbf/f11c to d8/f122 0 2024-03-18T14:51:11.669 INFO:tasks.workunit.client.1.smithi100.stdout:5/667: mkdir d29/d2d/d39/d83/dc1/dbb/d9e/dd7 0 2024-03-18T14:51:11.674 INFO:tasks.workunit.client.1.smithi100.stdout:2/715: creat d3/d6/ff7 x:0 0 0 2024-03-18T14:51:11.674 INFO:tasks.workunit.client.1.smithi100.stdout:2/716: chown d3/da/de/d20/dea/d4f 906621 1 2024-03-18T14:51:11.675 INFO:tasks.workunit.client.0.smithi012.stdout:1/907: dread d8/db/d10/d16/da1/d2e/d9c/d9d/fdb [0,4194304] 0 2024-03-18T14:51:11.679 INFO:tasks.workunit.client.1.smithi100.stdout:1/695: rename d8/f29 to d8/d1e/d11/d27/d83/fd4 0 2024-03-18T14:51:11.679 INFO:tasks.workunit.client.0.smithi012.stdout:5/913: read d3/d15/d42/dc6/d73/d6b/f98 [810286,68527] 0 2024-03-18T14:51:11.681 INFO:tasks.workunit.client.0.smithi012.stdout:3/908: sync 2024-03-18T14:51:11.683 INFO:tasks.workunit.client.1.smithi100.stdout:7/686: unlink d5/d14/ff3 0 2024-03-18T14:51:11.684 INFO:tasks.workunit.client.1.smithi100.stdout:2/717: mknod d3/d1a/d76/cf8 0 2024-03-18T14:51:11.685 INFO:tasks.workunit.client.0.smithi012.stdout:5/914: dread d3/d29/d36/fe4 [0,4194304] 0 2024-03-18T14:51:11.685 INFO:tasks.workunit.client.1.smithi100.stdout:1/696: mkdir d8/d1e/d11/d27/d3f/daa/dd5 0 2024-03-18T14:51:11.686 INFO:tasks.workunit.client.1.smithi100.stdout:6/642: sync 2024-03-18T14:51:11.686 INFO:tasks.workunit.client.1.smithi100.stdout:8/742: sync 2024-03-18T14:51:11.687 INFO:tasks.workunit.client.1.smithi100.stdout:0/725: sync 2024-03-18T14:51:11.687 INFO:tasks.workunit.client.1.smithi100.stdout:0/726: dread - dd/d2b/d95/d79/d7d/da9/db0/fb6 zero size 2024-03-18T14:51:11.687 INFO:tasks.workunit.client.1.smithi100.stdout:0/727: chown dd/d2b/d95/dc1 9614288 1 2024-03-18T14:51:11.688 INFO:tasks.workunit.client.0.smithi012.stdout:3/909: mkdir dc/d3c/d4c/d57/d103/d9a/d64/df4/d127 0 2024-03-18T14:51:11.688 INFO:tasks.workunit.client.0.smithi012.stdout:0/979: dwrite d1/faa [0,4194304] 0 2024-03-18T14:51:11.688 INFO:tasks.workunit.client.0.smithi012.stdout:0/980: chown d1/d20/d23/d3b/d33/d51/ff3 1588 1 2024-03-18T14:51:11.688 INFO:tasks.workunit.client.1.smithi100.stdout:1/697: read d8/fab [83358,92321] 0 2024-03-18T14:51:11.688 INFO:tasks.workunit.client.1.smithi100.stdout:1/698: creat d8/d1e/d11/d27/fd6 x:0 0 0 2024-03-18T14:51:11.689 INFO:tasks.workunit.client.1.smithi100.stdout:7/687: symlink d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6/l10b 0 2024-03-18T14:51:11.691 INFO:tasks.workunit.client.0.smithi012.stdout:5/915: creat d3/d15/d42/dc6/d86/f136 x:0 0 0 2024-03-18T14:51:11.691 INFO:tasks.workunit.client.0.smithi012.stdout:5/916: truncate d3/d15/d42/f105 615615 0 2024-03-18T14:51:11.694 INFO:tasks.workunit.client.0.smithi012.stdout:3/910: dread dc/d3c/d4c/d57/fa1 [4194304,4194304] 0 2024-03-18T14:51:11.695 INFO:tasks.workunit.client.1.smithi100.stdout:5/668: dwrite d29/d2d/d39/f84 [0,4194304] 0 2024-03-18T14:51:11.701 INFO:tasks.workunit.client.1.smithi100.stdout:6/643: dread d6/d7/df/f55 [0,4194304] 0 2024-03-18T14:51:11.704 INFO:tasks.workunit.client.1.smithi100.stdout:7/688: mkdir d5/d22/d24/d39/d10c 0 2024-03-18T14:51:11.705 INFO:tasks.workunit.client.0.smithi012.stdout:0/981: rmdir d1/d20/d23/d3b/d33/d51 39 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 -- 172.21.15.12:0/1731119801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc06e7a0 msgr2=0x7febcc06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 --2- 172.21.15.12:0/1731119801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc06e7a0 0x7febcc06ebb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febc4009ce0 tx=0x7febc4004d10 comp rx=0 tx=0).stop 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 -- 172.21.15.12:0/1731119801 shutdown_connections 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 --2- 172.21.15.12:0/1731119801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 0x7febcc108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 --2- 172.21.15.12:0/1731119801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc06e7a0 0x7febcc06ebb0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febc4009ce0 tx=0x7febc4004d10 comp rx=0 tx=0).stop 2024-03-18T14:51:11.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 -- 172.21.15.12:0/1731119801 >> 172.21.15.12:0/1731119801 conn(0x7febcc06a380 msgr2=0x7febcc06c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:11.709 INFO:tasks.workunit.client.1.smithi100.stdout:8/743: unlink d4/dc/dd/de/d19/l7d 0 2024-03-18T14:51:11.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 -- 172.21.15.12:0/1731119801 shutdown_connections 2024-03-18T14:51:11.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.707+0000 7febd4177700 1 -- 172.21.15.12:0/1731119801 wait complete. 2024-03-18T14:51:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.708+0000 7febd4177700 1 Processor -- start 2024-03-18T14:51:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.708+0000 7febd4177700 1 -- start start 2024-03-18T14:51:11.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.708+0000 7febd4177700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 0x7febcc110950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd4177700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd4177700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc111350 con 0x7febcc110e90 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd4177700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc10d930 con 0x7febcc06f0f0 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd1712700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd1712700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50348/0 (socket says 172.21.15.12:50348) 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd1712700 1 -- 172.21.15.12:0/889771834 learned_addr learned my addr 172.21.15.12:0/889771834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.709+0000 7febd1f13700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 0x7febcc110950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.710+0000 7febd1712700 1 -- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 msgr2=0x7febcc110950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.710+0000 7febd1712700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 0x7febcc110950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.710+0000 7febd1712700 1 -- 172.21.15.12:0/889771834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc4009940 con 0x7febcc110e90 2024-03-18T14:51:11.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.710+0000 7febd1712700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febc800c830 tx=0x7febc8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:11.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.711+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc80106f0 con 0x7febcc110e90 2024-03-18T14:51:11.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.711+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febc8010d20 con 0x7febcc110e90 2024-03-18T14:51:11.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.711+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc80113a0 con 0x7febcc110e90 2024-03-18T14:51:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.712+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc10dbc0 con 0x7febcc110e90 2024-03-18T14:51:11.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.712+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc122b90 con 0x7febcc110e90 2024-03-18T14:51:11.713 INFO:tasks.workunit.client.0.smithi012.stdout:0/982: creat d1/d20/dcd/f13d x:0 0 0 2024-03-18T14:51:11.714 INFO:tasks.workunit.client.0.smithi012.stdout:1/908: dwrite d8/db/f2d [0,4194304] 0 2024-03-18T14:51:11.714 INFO:tasks.workunit.client.0.smithi012.stdout:1/909: chown d8/f116 862 1 2024-03-18T14:51:11.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.714+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc04c710 con 0x7febcc110e90 2024-03-18T14:51:11.715 INFO:tasks.workunit.client.1.smithi100.stdout:0/728: dwrite dd/f1a [0,4194304] 0 2024-03-18T14:51:11.716 INFO:tasks.workunit.client.1.smithi100.stdout:1/699: write d8/f52 [2141935,1057] 0 2024-03-18T14:51:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.715+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7febc8012400 con 0x7febcc110e90 2024-03-18T14:51:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.716+0000 7febc27fc700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 0x7febb806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:11.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.716+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7febc808a080 con 0x7febcc110e90 2024-03-18T14:51:11.717 INFO:tasks.workunit.client.1.smithi100.stdout:5/669: mkdir d29/dd8 0 2024-03-18T14:51:11.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.717+0000 7febd1f13700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 0x7febb806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:11.719 INFO:tasks.workunit.client.0.smithi012.stdout:0/983: creat d1/d16/d19/dd0/d129/d67/d10c/f13e x:0 0 0 2024-03-18T14:51:11.721 INFO:tasks.workunit.client.1.smithi100.stdout:6/644: creat d6/d7/fc5 x:0 0 0 2024-03-18T14:51:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.721+0000 7febd1f13700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 0x7febb806db30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f80 tx=0x7febc4004d10 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:11.723 INFO:tasks.workunit.client.1.smithi100.stdout:7/689: unlink d5/d22/d24/d6e/d7a/c9d 0 2024-03-18T14:51:11.725 INFO:tasks.workunit.client.0.smithi012.stdout:0/984: creat d1/d13a/f13f x:0 0 0 2024-03-18T14:51:11.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:11.725+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7febc8057970 con 0x7febcc110e90 2024-03-18T14:51:11.731 INFO:tasks.workunit.client.0.smithi012.stdout:0/985: creat d1/d16/d19/dd0/d129/d67/f140 x:0 0 0 2024-03-18T14:51:11.731 INFO:tasks.workunit.client.0.smithi012.stdout:5/917: dwrite d3/d15/d42/f126 [0,4194304] 0 2024-03-18T14:51:11.732 INFO:tasks.workunit.client.0.smithi012.stdout:1/910: dread d8/d1e/d27/f74 [0,4194304] 0 2024-03-18T14:51:11.736 INFO:tasks.workunit.client.0.smithi012.stdout:0/986: rmdir d1/d16/d19/dd0/d129/d67/dbc 39 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/918: creat d3/d29/d35/dcc/d133/f137 x:0 0 0 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/919: write d3/d15/d42/dc6/d73/d6b/df3/fde [736451,29578] 0 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/920: chown d3/d1e 93 1 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/921: chown d3/c39 15361879 1 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.0.smithi012.stdout:5/922: write d3/db6/df0/f131 [350450,84763] 0 2024-03-18T14:51:11.742 INFO:tasks.workunit.client.1.smithi100.stdout:5/670: symlink d29/ld9 0 2024-03-18T14:51:11.743 INFO:tasks.workunit.client.0.smithi012.stdout:0/987: mknod d1/d20/d23/d26/c141 0 2024-03-18T14:51:11.743 INFO:tasks.workunit.client.0.smithi012.stdout:0/988: chown d1/d16/d19/dd0/d129/d67/dbc/cd4 55855 1 2024-03-18T14:51:11.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/645: mknod d6/d7/df/d26/d4e/d66/d36/d4f/d8a/cc6 0 2024-03-18T14:51:11.745 INFO:tasks.workunit.client.1.smithi100.stdout:6/646: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [4176881,117896] 0 2024-03-18T14:51:11.745 INFO:tasks.workunit.client.0.smithi012.stdout:3/911: dwrite dc/d3c/d4c/d57/d103/d9a/d54/f58 [0,4194304] 0 2024-03-18T14:51:11.746 INFO:tasks.workunit.client.1.smithi100.stdout:7/690: creat d5/d22/d3b/df5/f10d x:0 0 0 2024-03-18T14:51:11.746 INFO:tasks.workunit.client.1.smithi100.stdout:7/691: creat d5/d30/f10e x:0 0 0 2024-03-18T14:51:11.746 INFO:tasks.workunit.client.1.smithi100.stdout:7/692: fsync d5/f52 0 2024-03-18T14:51:11.746 INFO:tasks.workunit.client.1.smithi100.stdout:8/744: dwrite d4/dc/dd/de/d21/d23/d47/f86 [0,4194304] 0 2024-03-18T14:51:11.747 INFO:tasks.workunit.client.0.smithi012.stdout:0/989: rmdir d1/d20/d7b/d124 0 2024-03-18T14:51:11.748 INFO:tasks.workunit.client.1.smithi100.stdout:1/700: dwrite d8/d30/f40 [0,4194304] 0 2024-03-18T14:51:11.752 INFO:tasks.workunit.client.1.smithi100.stdout:5/671: dread d29/f37 [0,4194304] 0 2024-03-18T14:51:11.753 INFO:tasks.workunit.client.1.smithi100.stdout:5/672: write d29/f37 [3354182,75511] 0 2024-03-18T14:51:11.753 INFO:tasks.workunit.client.0.smithi012.stdout:3/912: truncate dc/d3c/d72/fc0 1356316 0 2024-03-18T14:51:11.754 INFO:tasks.workunit.client.0.smithi012.stdout:5/923: dread d3/d15/d42/dc6/f47 [0,4194304] 0 2024-03-18T14:51:11.754 INFO:tasks.workunit.client.0.smithi012.stdout:5/924: chown d3/d15/cdc 804217 1 2024-03-18T14:51:11.755 INFO:tasks.workunit.client.1.smithi100.stdout:1/701: write d8/d1e/d11/d27/d83/fd4 [1410685,126231] 0 2024-03-18T14:51:11.756 INFO:tasks.workunit.client.1.smithi100.stdout:0/729: dwrite dd/d31/d91/d56/d71/dcc/dcf/fdf [0,4194304] 0 2024-03-18T14:51:11.756 INFO:tasks.workunit.client.1.smithi100.stdout:0/730: dread - dd/d20/d44/feb zero size 2024-03-18T14:51:11.756 INFO:tasks.workunit.client.0.smithi012.stdout:1/911: dwrite d8/db/d10/d16/d6b/d7e/fad [0,4194304] 0 2024-03-18T14:51:11.756 INFO:tasks.workunit.client.0.smithi012.stdout:0/990: mknod d1/d16/d19/dd0/d129/d67/d10c/c142 0 2024-03-18T14:51:11.763 INFO:tasks.workunit.client.0.smithi012.stdout:3/913: symlink dc/d3c/d4c/d57/d103/d9a/dd7/de3/l128 0 2024-03-18T14:51:11.765 INFO:tasks.workunit.client.1.smithi100.stdout:8/745: mknod d4/dc/dd/c10a 0 2024-03-18T14:51:11.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/746: truncate d4/d5a/dbb/f6b 1245599 0 2024-03-18T14:51:11.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/747: dread - d4/dc/dd/de/d21/d23/d47/fa4 zero size 2024-03-18T14:51:11.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/748: chown d4/d3c/lce 371 1 2024-03-18T14:51:11.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/749: stat d4/dc/dd/de/d21/d23/d47/d54 0 2024-03-18T14:51:11.766 INFO:tasks.workunit.client.1.smithi100.stdout:8/750: dread d4/dc/dd/de/d21/f72 [0,4194304] 0 2024-03-18T14:51:11.768 INFO:tasks.workunit.client.1.smithi100.stdout:6/647: getdents d6/d12/d2c 0 2024-03-18T14:51:11.773 INFO:tasks.workunit.client.1.smithi100.stdout:5/673: symlink d29/d2d/d39/d60/lda 0 2024-03-18T14:51:11.773 INFO:tasks.workunit.client.1.smithi100.stdout:5/674: chown l1d 228151 1 2024-03-18T14:51:11.778 INFO:tasks.workunit.client.0.smithi012.stdout:3/914: dread dc/d3c/d4c/d57/d103/d9a/f5d [0,4194304] 0 2024-03-18T14:51:11.778 INFO:tasks.workunit.client.0.smithi012.stdout:3/915: chown dc/d30/d3a/da0/dc5/dc7/dd8/c121 868059 1 2024-03-18T14:51:11.778 INFO:tasks.workunit.client.0.smithi012.stdout:1/912: mknod d8/db/d10/d31/d43/d5e/d65/d84/dc1/c123 0 2024-03-18T14:51:11.782 INFO:tasks.workunit.client.1.smithi100.stdout:1/702: dwrite d8/d1e/d11/d27/d3f/f49 [0,4194304] 0 2024-03-18T14:51:11.782 INFO:tasks.workunit.client.1.smithi100.stdout:1/703: fsync d8/d1e/d11/d27/d3f/f74 0 2024-03-18T14:51:11.783 INFO:tasks.workunit.client.0.smithi012.stdout:5/925: rmdir d3/db6 39 2024-03-18T14:51:11.783 INFO:tasks.workunit.client.0.smithi012.stdout:0/991: dwrite d1/d20/d23/d3b/dc1/f13c [0,4194304] 0 2024-03-18T14:51:11.785 INFO:tasks.workunit.client.0.smithi012.stdout:3/916: rename ca to dc/d3c/d4c/c129 0 2024-03-18T14:51:11.786 INFO:tasks.workunit.client.0.smithi012.stdout:5/926: mkdir d3/dad/d138 0 2024-03-18T14:51:11.786 INFO:tasks.workunit.client.0.smithi012.stdout:5/927: write d3/d1e/f93 [711012,65367] 0 2024-03-18T14:51:11.787 INFO:tasks.workunit.client.0.smithi012.stdout:5/928: write d3/d15/d42/dc6/d73/d6b/df3/fde [672518,35579] 0 2024-03-18T14:51:11.787 INFO:tasks.workunit.client.0.smithi012.stdout:5/929: readlink d3/d29/d35/d5b/lc2 0 2024-03-18T14:51:11.789 INFO:tasks.workunit.client.1.smithi100.stdout:7/693: getdents d5/d22/d24/d39 0 2024-03-18T14:51:11.791 INFO:tasks.workunit.client.1.smithi100.stdout:8/751: symlink d4/dc/d97/l10b 0 2024-03-18T14:51:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:0/992: symlink d1/d16/d19/dd0/l143 0 2024-03-18T14:51:11.799 INFO:tasks.workunit.client.0.smithi012.stdout:1/913: rename d8/db/d10/d31/d81/fa4 to d8/db/d10/d31/f124 0 2024-03-18T14:51:11.799 INFO:tasks.workunit.client.0.smithi012.stdout:1/914: write d8/f122 [518738,49365] 0 2024-03-18T14:51:11.799 INFO:tasks.workunit.client.0.smithi012.stdout:1/915: stat d8/d1e/d27/d44/f4f 0 2024-03-18T14:51:11.799 INFO:tasks.workunit.client.0.smithi012.stdout:1/916: chown d8/db/d10/d16/caa 16 1 2024-03-18T14:51:11.801 INFO:tasks.workunit.client.1.smithi100.stdout:8/752: read d4/dc/d1f/d33/f8d [3490450,130560] 0 2024-03-18T14:51:11.802 INFO:tasks.workunit.client.0.smithi012.stdout:3/917: unlink dc/d3c/c6f 0 2024-03-18T14:51:11.803 INFO:tasks.workunit.client.1.smithi100.stdout:6/648: symlink d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d63/lc7 0 2024-03-18T14:51:11.805 INFO:tasks.workunit.client.1.smithi100.stdout:5/675: symlink d29/ldb 0 2024-03-18T14:51:11.806 INFO:tasks.workunit.client.0.smithi012.stdout:5/930: rmdir d3/d15/d42/dc6/d73/d6b 39 2024-03-18T14:51:11.810 INFO:tasks.workunit.client.0.smithi012.stdout:0/993: dwrite d1/f9 [0,4194304] 0 2024-03-18T14:51:11.815 INFO:tasks.workunit.client.1.smithi100.stdout:1/704: creat d8/d1e/d11/d6d/d59/fd7 x:0 0 0 2024-03-18T14:51:11.818 INFO:tasks.workunit.client.1.smithi100.stdout:7/694: mknod d5/d22/d24/d39/c10f 0 2024-03-18T14:51:11.820 INFO:tasks.workunit.client.1.smithi100.stdout:8/753: mkdir d4/dc/dd/d48/d10c 0 2024-03-18T14:51:11.820 INFO:tasks.workunit.client.1.smithi100.stdout:8/754: chown d4/f6 21971 1 2024-03-18T14:51:11.824 INFO:tasks.workunit.client.1.smithi100.stdout:6/649: mknod d6/d7/df/d26/d4e/d66/d36/d4f/d71/cc8 0 2024-03-18T14:51:11.828 INFO:tasks.workunit.client.1.smithi100.stdout:5/676: unlink d29/d2d/d39/d83/dc1/dbb/d9e/lcd 0 2024-03-18T14:51:11.828 INFO:tasks.workunit.client.1.smithi100.stdout:5/677: dread d29/d2d/d39/d83/f85 [0,4194304] 0 2024-03-18T14:51:11.829 INFO:tasks.workunit.client.1.smithi100.stdout:1/705: symlink d8/d2a/ld8 0 2024-03-18T14:51:11.833 INFO:tasks.workunit.client.1.smithi100.stdout:6/650: mkdir d6/d7/df/d72/dc9 0 2024-03-18T14:51:11.835 INFO:tasks.workunit.client.1.smithi100.stdout:5/678: symlink d29/d2d/d39/d60/ldc 0 2024-03-18T14:51:11.835 INFO:tasks.workunit.client.1.smithi100.stdout:5/679: truncate d29/fa0 952317 0 2024-03-18T14:51:11.836 INFO:tasks.workunit.client.1.smithi100.stdout:3/715: sync 2024-03-18T14:51:11.836 INFO:tasks.workunit.client.1.smithi100.stdout:9/728: sync 2024-03-18T14:51:11.837 INFO:tasks.workunit.client.1.smithi100.stdout:3/716: chown d3/fa1 125493 1 2024-03-18T14:51:11.837 INFO:tasks.workunit.client.1.smithi100.stdout:3/717: creat d3/da5/ff9 x:0 0 0 2024-03-18T14:51:11.837 INFO:tasks.workunit.client.1.smithi100.stdout:9/729: read da/d88/f8c [1029626,76400] 0 2024-03-18T14:51:11.837 INFO:tasks.workunit.client.1.smithi100.stdout:9/730: chown da/d51/d55/d7e/f9d 2391649 1 2024-03-18T14:51:11.838 INFO:tasks.workunit.client.1.smithi100.stdout:6/651: rename d6/d12/f75 to d6/d9f/fca 0 2024-03-18T14:51:11.838 INFO:tasks.workunit.client.1.smithi100.stdout:6/652: chown d6/d7/df/d26/d4e/d66/d36/d4f/d71 118126991 1 2024-03-18T14:51:11.842 INFO:tasks.workunit.client.0.smithi012.stdout:0/994: dwrite d1/d16/d19/dd0/d129/d67/f138 [0,4194304] 0 2024-03-18T14:51:11.845 INFO:tasks.workunit.client.0.smithi012.stdout:1/917: creat d8/db/d10/d31/d43/d5e/f125 x:0 0 0 2024-03-18T14:51:11.846 INFO:tasks.workunit.client.1.smithi100.stdout:9/731: truncate da/d46/d8b/f3f 302958 0 2024-03-18T14:51:11.846 INFO:tasks.workunit.client.1.smithi100.stdout:9/732: stat da/dd/d3b/c80 0 2024-03-18T14:51:11.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/695: dwrite d5/d22/d3b/d9f/da0/db1/fda [0,4194304] 0 2024-03-18T14:51:11.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/696: readlink d5/d14/d16/d2e/l31 0 2024-03-18T14:51:11.849 INFO:tasks.workunit.client.0.smithi012.stdout:3/918: link dc/d30/d35/l8c dc/d3c/d4c/d57/d103/d9a/dd7/l12a 0 2024-03-18T14:51:11.850 INFO:tasks.workunit.client.1.smithi100.stdout:8/755: dwrite d4/dc/dd/f11 [0,4194304] 0 2024-03-18T14:51:11.850 INFO:tasks.workunit.client.1.smithi100.stdout:8/756: stat d4/dc/d1f/d59/fbd 0 2024-03-18T14:51:11.850 INFO:tasks.workunit.client.1.smithi100.stdout:3/718: dread d3/d16/d13/f3b [0,4194304] 0 2024-03-18T14:51:11.853 INFO:tasks.workunit.client.0.smithi012.stdout:5/931: rename d3/d15/d42/dc6/d73/d6b/df3/d8b/ce8 to d3/d15/d5f/c139 0 2024-03-18T14:51:11.855 INFO:tasks.workunit.client.1.smithi100.stdout:9/733: unlink da/d2c/d2f/c4e 0 2024-03-18T14:51:11.857 INFO:tasks.workunit.client.0.smithi012.stdout:5/932: dread d3/d15/d42/dc6/f4a [0,4194304] 0 2024-03-18T14:51:11.859 INFO:tasks.workunit.client.1.smithi100.stdout:1/706: dwrite d8/d1e/d11/d27/d3e/f4b [0,4194304] 0 2024-03-18T14:51:11.859 INFO:tasks.workunit.client.1.smithi100.stdout:6/653: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa3 [0,4194304] 0 2024-03-18T14:51:11.862 INFO:tasks.workunit.client.0.smithi012.stdout:3/919: dread f3 [0,4194304] 0 2024-03-18T14:51:11.868 INFO:tasks.workunit.client.1.smithi100.stdout:2/718: sync 2024-03-18T14:51:11.868 INFO:tasks.workunit.client.1.smithi100.stdout:4/671: sync 2024-03-18T14:51:11.868 INFO:tasks.workunit.client.1.smithi100.stdout:2/719: write d3/d6/d29/f35 [2093924,33521] 0 2024-03-18T14:51:11.868 INFO:tasks.workunit.client.1.smithi100.stdout:2/720: readlink d3/d1a/d76/l8a 0 2024-03-18T14:51:11.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/995: mknod d1/d20/d23/d3b/d33/d64/c144 0 2024-03-18T14:51:11.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/996: readlink d1/d20/d23/d7f/l8c 0 2024-03-18T14:51:11.870 INFO:tasks.workunit.client.0.smithi012.stdout:0/997: truncate d1/d2d/f117 833786 0 2024-03-18T14:51:11.874 INFO:tasks.workunit.client.0.smithi012.stdout:3/920: dwrite dc/d3c/d4c/ff9 [0,4194304] 0 2024-03-18T14:51:11.879 INFO:tasks.workunit.client.1.smithi100.stdout:8/757: mknod d4/dc/d31/d85/dd6/df9/c10d 0 2024-03-18T14:51:11.879 INFO:tasks.workunit.client.1.smithi100.stdout:8/758: write d4/dc/d31/f46 [4417066,29036] 0 2024-03-18T14:51:11.882 INFO:tasks.workunit.client.1.smithi100.stdout:9/734: creat da/dd/fec x:0 0 0 2024-03-18T14:51:11.883 INFO:tasks.workunit.client.0.smithi012.stdout:5/933: dread d3/d15/d42/dc6/d73/f7d [0,4194304] 0 2024-03-18T14:51:11.883 INFO:tasks.workunit.client.0.smithi012.stdout:5/934: chown d3/d15/d42/f105 3028983 1 2024-03-18T14:51:11.887 INFO:tasks.workunit.client.0.smithi012.stdout:3/921: getdents dc/d30/d3a/da0/dc5/dc7/dfc 0 2024-03-18T14:51:11.888 INFO:tasks.workunit.client.1.smithi100.stdout:1/707: mknod d8/d1e/d11/d27/d73/cd9 0 2024-03-18T14:51:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:3/922: dread f3 [0,4194304] 0 2024-03-18T14:51:11.891 INFO:tasks.workunit.client.0.smithi012.stdout:3/923: write dc/d3c/d72/df3/d109/d11a/f120 [4863338,737] 0 2024-03-18T14:51:11.894 INFO:tasks.workunit.client.1.smithi100.stdout:6/654: rename d6/d7/df/d26/c3e to d6/d7/ccb 0 2024-03-18T14:51:11.894 INFO:tasks.workunit.client.0.smithi012.stdout:0/998: rmdir d1/d16/d19 39 2024-03-18T14:51:11.896 INFO:tasks.workunit.client.1.smithi100.stdout:6/655: dread d6/d12/d2c/f4c [0,4194304] 0 2024-03-18T14:51:11.896 INFO:tasks.workunit.client.1.smithi100.stdout:4/672: unlink d9/d13/c37 0 2024-03-18T14:51:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/935: creat d3/d29/d9d/dc4/f13a x:0 0 0 2024-03-18T14:51:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/936: write d3/d84/f9f [887156,99801] 0 2024-03-18T14:51:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/937: chown d3/d29/d36/f45 1069551827 1 2024-03-18T14:51:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/938: fsync d3/d15/d42/dc6/d73/d6b/f12f 0 2024-03-18T14:51:11.898 INFO:tasks.workunit.client.0.smithi012.stdout:5/939: write d3/d15/d42/dc6/d65/da5/f119 [268585,121798] 0 2024-03-18T14:51:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:5/940: chown d3/d15/d42/dc6/d73/f124 37428160 1 2024-03-18T14:51:11.899 INFO:tasks.workunit.client.0.smithi012.stdout:5/941: stat d3/d1e/f12b 0 2024-03-18T14:51:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:3/924: mkdir dc/d3c/d4c/d57/d103/d9a/d64/df4/d127/d12b 0 2024-03-18T14:51:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:3/925: dread - dc/d30/d3a/da0/dc5/ffe zero size 2024-03-18T14:51:11.900 INFO:tasks.workunit.client.0.smithi012.stdout:3/926: chown dc/d3c/d4c/d57/d103/d9a/dd7/d124/cef 37598 1 2024-03-18T14:51:11.902 INFO:tasks.workunit.client.0.smithi012.stdout:0/999: mknod d1/d20/d23/d3b/c145 0 2024-03-18T14:51:11.904 INFO:tasks.workunit.client.1.smithi100.stdout:7/697: rmdir d5/d22/d24/d6e/d7a 39 2024-03-18T14:51:11.904 INFO:tasks.workunit.client.1.smithi100.stdout:7/698: readlink d5/d14/d16/d2e/d62/dd1/la3 0 2024-03-18T14:51:11.904 INFO:tasks.workunit.client.1.smithi100.stdout:7/699: fdatasync d5/d22/d24/dac/fca 0 2024-03-18T14:51:11.908 INFO:tasks.workunit.client.1.smithi100.stdout:8/759: symlink d4/d5a/l10e 0 2024-03-18T14:51:11.908 INFO:tasks.workunit.client.1.smithi100.stdout:0/731: sync 2024-03-18T14:51:11.908 INFO:tasks.workunit.client.1.smithi100.stdout:0/732: write dd/d20/db5/f102 [99048,37783] 0 2024-03-18T14:51:11.910 INFO:tasks.workunit.client.1.smithi100.stdout:9/735: mknod da/d46/d4d/ced 0 2024-03-18T14:51:11.910 INFO:tasks.workunit.client.1.smithi100.stdout:9/736: stat da/d2c/f89 0 2024-03-18T14:51:11.910 INFO:tasks.workunit.client.1.smithi100.stdout:9/737: creat da/dcb/fee x:0 0 0 2024-03-18T14:51:11.915 INFO:tasks.workunit.client.1.smithi100.stdout:1/708: stat d8/d30/l38 0 2024-03-18T14:51:11.915 INFO:tasks.workunit.client.1.smithi100.stdout:1/709: write d8/d30/d61/d8d/fc1 [4865754,51792] 0 2024-03-18T14:51:11.919 INFO:tasks.workunit.client.1.smithi100.stdout:6/656: rename d6/d7/df/d26/d4e/c9a to d6/d12/d2c/ccc 0 2024-03-18T14:51:11.919 INFO:tasks.workunit.client.1.smithi100.stdout:6/657: rename d6/d7/df/d26/d4e/d66/d36/d4f/d71 to d6/d7/df/d26/d4e/d66/d36/d4f/d71/dcd 22 2024-03-18T14:51:11.920 INFO:tasks.workunit.client.0.smithi012.stdout:5/942: rmdir d3/dad 39 2024-03-18T14:51:11.920 INFO:tasks.workunit.client.0.smithi012.stdout:5/943: fsync d3/d15/d42/dc6/d73/f107 0 2024-03-18T14:51:11.922 INFO:tasks.workunit.client.1.smithi100.stdout:4/673: mkdir d9/dc8 0 2024-03-18T14:51:11.923 INFO:tasks.workunit.client.1.smithi100.stdout:8/760: mknod d4/dc/dd/de/c10f 0 2024-03-18T14:51:11.925 INFO:tasks.workunit.client.1.smithi100.stdout:0/733: link dd/d20/db5/lda dd/d2b/d95/d79/d7d/da9/db0/l108 0 2024-03-18T14:51:11.927 INFO:tasks.workunit.client.1.smithi100.stdout:9/738: mknod da/dd/d3c/d6b/cef 0 2024-03-18T14:51:11.928 INFO:tasks.workunit.client.0.smithi012.stdout:3/927: dwrite dc/d3c/d59/fc8 [0,4194304] 0 2024-03-18T14:51:11.928 INFO:tasks.workunit.client.1.smithi100.stdout:7/700: dwrite d5/d14/d16/d2e/d62/f97 [0,4194304] 0 2024-03-18T14:51:11.932 INFO:tasks.workunit.client.1.smithi100.stdout:6/658: dread d6/d7/df/d26/f56 [0,4194304] 0 2024-03-18T14:51:11.932 INFO:tasks.workunit.client.1.smithi100.stdout:6/659: fsync d6/d12/d2c/fc1 0 2024-03-18T14:51:11.932 INFO:tasks.workunit.client.1.smithi100.stdout:6/660: write d6/d12/d78/fb2 [765218,111731] 0 2024-03-18T14:51:11.932 INFO:tasks.workunit.client.1.smithi100.stdout:4/674: rmdir d9/d13/d5a/d5b/da0 39 2024-03-18T14:51:11.934 INFO:tasks.workunit.client.0.smithi012.stdout:5/944: dread d3/d29/d36/d9b/ddf/f104 [0,4194304] 0 2024-03-18T14:51:11.935 INFO:tasks.workunit.client.0.smithi012.stdout:3/928: symlink dc/d3c/l12c 0 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/739: symlink da/d51/d55/d7e/lf0 0 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/740: dread - da/dd/da0/fa6 zero size 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/741: write da/dd/d3c/fc9 [441119,75045] 0 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/742: dread - da/dd/d29/d71/d72/fb6 zero size 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/743: fsync da/dcb/fd3 0 2024-03-18T14:51:11.938 INFO:tasks.workunit.client.1.smithi100.stdout:9/744: write da/d46/d4d/d67/f75 [64659,69105] 0 2024-03-18T14:51:11.939 INFO:tasks.workunit.client.0.smithi012.stdout:5/945: symlink d3/db6/l13b 0 2024-03-18T14:51:11.940 INFO:tasks.workunit.client.1.smithi100.stdout:7/701: mkdir d5/d14/d16/d2e/d62/dc3/d110 0 2024-03-18T14:51:11.940 INFO:tasks.workunit.client.1.smithi100.stdout:7/702: dread - d5/d22/d3b/d9f/da0/db1/dd3/ff1 zero size 2024-03-18T14:51:11.940 INFO:tasks.workunit.client.1.smithi100.stdout:7/703: fsync d5/d14/d16/d2e/d62/ffd 0 2024-03-18T14:51:11.942 INFO:tasks.workunit.client.1.smithi100.stdout:6/661: symlink d6/d7/df/d26/d4e/db5/dba/lce 0 2024-03-18T14:51:11.943 INFO:tasks.workunit.client.1.smithi100.stdout:8/761: dwrite d4/dc/d31/d85/fb2 [0,4194304] 0 2024-03-18T14:51:11.943 INFO:tasks.workunit.client.1.smithi100.stdout:8/762: creat d4/dc/d97/f110 x:0 0 0 2024-03-18T14:51:11.943 INFO:tasks.workunit.client.1.smithi100.stdout:8/763: chown d4/f6 598 1 2024-03-18T14:51:11.944 INFO:tasks.workunit.client.0.smithi012.stdout:5/946: fsync d3/d84/f9f 0 2024-03-18T14:51:11.944 INFO:tasks.workunit.client.1.smithi100.stdout:4/675: dread d9/da6/fb6 [0,4194304] 0 2024-03-18T14:51:11.944 INFO:tasks.workunit.client.1.smithi100.stdout:4/676: write d9/f2b [3033936,74182] 0 2024-03-18T14:51:11.946 INFO:tasks.workunit.client.1.smithi100.stdout:4/677: write d9/d13/d39/f8f [790135,36045] 0 2024-03-18T14:51:11.950 INFO:tasks.workunit.client.0.smithi012.stdout:3/929: read dc/d3c/f6c [1443812,46020] 0 2024-03-18T14:51:11.957 INFO:tasks.workunit.client.0.smithi012.stdout:5/947: creat d3/d15/d42/f13c x:0 0 0 2024-03-18T14:51:11.959 INFO:tasks.workunit.client.1.smithi100.stdout:8/764: mknod d4/dc/d1f/d59/d6c/ddb/c111 0 2024-03-18T14:51:11.959 INFO:tasks.workunit.client.1.smithi100.stdout:6/662: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f22 [493562,130881] 0 2024-03-18T14:51:11.960 INFO:tasks.workunit.client.1.smithi100.stdout:7/704: write d5/d14/d16/d2e/d62/dd1/f51 [1263842,43674] 0 2024-03-18T14:51:11.960 INFO:tasks.workunit.client.1.smithi100.stdout:7/705: write d5/d30/f10e [365941,127355] 0 2024-03-18T14:51:11.960 INFO:tasks.workunit.client.0.smithi012.stdout:5/948: rmdir d3/d15/d42/dc6/d73/d6b/df3 39 2024-03-18T14:51:11.962 INFO:tasks.workunit.client.0.smithi012.stdout:3/930: dread dc/d30/f51 [0,4194304] 0 2024-03-18T14:51:11.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/765: truncate d4/dc/d1f/d33/f8d 680654 0 2024-03-18T14:51:11.963 INFO:tasks.workunit.client.1.smithi100.stdout:4/678: dwrite d9/d27/f36 [0,4194304] 0 2024-03-18T14:51:11.964 INFO:tasks.workunit.client.1.smithi100.stdout:6/663: getdents d6/d7/df/d26/d4e/d66/d36/d4f/d8a 0 2024-03-18T14:51:11.964 INFO:tasks.workunit.client.1.smithi100.stdout:9/745: dwrite da/d51/d55/f5c [0,4194304] 0 2024-03-18T14:51:11.965 INFO:tasks.workunit.client.0.smithi012.stdout:5/949: mkdir d3/d15/d42/d13d 0 2024-03-18T14:51:11.967 INFO:tasks.workunit.client.0.smithi012.stdout:3/931: write dc/d30/d3a/f117 [713856,101785] 0 2024-03-18T14:51:11.968 INFO:tasks.workunit.client.0.smithi012.stdout:3/932: readlink dc/d3c/d72/l7d 0 2024-03-18T14:51:11.968 INFO:tasks.workunit.client.1.smithi100.stdout:9/746: dread da/dcb/fe2 [0,4194304] 0 2024-03-18T14:51:11.968 INFO:tasks.workunit.client.1.smithi100.stdout:9/747: write da/d46/d8b/f3f [892319,64162] 0 2024-03-18T14:51:11.970 INFO:tasks.workunit.client.1.smithi100.stdout:8/766: unlink d4/dc/dd/de/d19/d2c/c4b 0 2024-03-18T14:51:11.971 INFO:tasks.workunit.client.1.smithi100.stdout:8/767: chown d4/dc/dd/c83 1975509 1 2024-03-18T14:51:11.971 INFO:tasks.workunit.client.1.smithi100.stdout:8/768: readlink d4/l6e 0 2024-03-18T14:51:11.976 INFO:tasks.workunit.client.0.smithi012.stdout:5/950: link d3/d1e/d33/l69 d3/d29/d35/d122/l13e 0 2024-03-18T14:51:11.977 INFO:tasks.workunit.client.1.smithi100.stdout:4/679: rename d9/c6e to d9/d27/d62/d7b/d82/dac/cc9 0 2024-03-18T14:51:11.979 INFO:tasks.workunit.client.1.smithi100.stdout:5/680: sync 2024-03-18T14:51:11.979 INFO:tasks.workunit.client.1.smithi100.stdout:4/680: dread d9/d13/d25/f48 [0,4194304] 0 2024-03-18T14:51:11.979 INFO:tasks.workunit.client.1.smithi100.stdout:4/681: fdatasync d9/d13/f33 0 2024-03-18T14:51:11.979 INFO:tasks.workunit.client.1.smithi100.stdout:4/682: write d9/d3a/f64 [287114,125702] 0 2024-03-18T14:51:11.980 INFO:tasks.workunit.client.0.smithi012.stdout:3/933: link dc/d91/la2 dc/d3c/d4c/d57/l12d 0 2024-03-18T14:51:11.980 INFO:tasks.workunit.client.0.smithi012.stdout:3/934: truncate dc/d30/fd6 585335 0 2024-03-18T14:51:11.983 INFO:tasks.workunit.client.1.smithi100.stdout:0/734: dwrite dd/d31/d91/d56/d62/f84 [0,4194304] 0 2024-03-18T14:51:11.985 INFO:tasks.workunit.client.1.smithi100.stdout:8/769: dread d4/dc/d31/d85/ff5 [0,4194304] 0 2024-03-18T14:51:11.988 INFO:tasks.workunit.client.0.smithi012.stdout:5/951: truncate d3/d29/d36/f6d 2021558 0 2024-03-18T14:51:11.990 INFO:tasks.workunit.client.0.smithi012.stdout:3/935: write dc/d3c/d59/f5a [3417148,61165] 0 2024-03-18T14:51:11.992 INFO:tasks.workunit.client.0.smithi012.stdout:5/952: mkdir d3/dad/df2/d102/d13f 0 2024-03-18T14:51:11.996 INFO:tasks.workunit.client.1.smithi100.stdout:9/748: mkdir da/dd/df1 0 2024-03-18T14:51:11.997 INFO:tasks.workunit.client.0.smithi012.stdout:5/953: truncate d3/d29/d35/dcc/ff8 355253 0 2024-03-18T14:51:11.999 INFO:tasks.workunit.client.1.smithi100.stdout:6/664: rename d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d63 to d6/d7/df/d26/d4e/d66/d36/d4f/dcf 0 2024-03-18T14:51:12.017 INFO:tasks.workunit.client.1.smithi100.stdout:9/749: dwrite da/f4f [0,4194304] 0 2024-03-18T14:51:12.017 INFO:tasks.workunit.client.1.smithi100.stdout:9/750: readlink da/dd/d3b/l45 0 2024-03-18T14:51:12.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.071+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febcc122de0 con 0x7febb806b680 2024-03-18T14:51:12.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.073+0000 7febc27fc700 1 -- 172.21.15.12:0/889771834 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7febcc122de0 con 0x7febb806b680 2024-03-18T14:51:12.073 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "1/2 daemons upgraded", 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "", 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:51:12.074 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:51:12.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 msgr2=0x7febb806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:12.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 0x7febb806db30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febc4000f80 tx=0x7febc4004d10 comp rx=0 tx=0).stop 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 msgr2=0x7febcc10d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febc800c830 tx=0x7febc8004470 comp rx=0 tx=0).stop 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 shutdown_connections 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7febb806b680 0x7febb806db30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc06f0f0 0x7febcc110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 --2- 172.21.15.12:0/889771834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7febcc110e90 0x7febcc10d3f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 >> 172.21.15.12:0/889771834 conn(0x7febcc06a380 msgr2=0x7febcc1073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 shutdown_connections 2024-03-18T14:51:12.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:12.075+0000 7febd4177700 1 -- 172.21.15.12:0/889771834 wait complete. 2024-03-18T14:51:12.082 INFO:tasks.workunit.client.0.smithi012.stdout:4/994: sync 2024-03-18T14:51:12.083 INFO:tasks.workunit.client.1.smithi100.stdout:5/681: unlink f12 0 2024-03-18T14:51:12.084 INFO:tasks.workunit.client.0.smithi012.stdout:5/954: mkdir d3/d29/d140 0 2024-03-18T14:51:12.084 INFO:tasks.workunit.client.0.smithi012.stdout:5/955: write d3/d1e/f114 [87347,68041] 0 2024-03-18T14:51:12.084 INFO:tasks.workunit.client.0.smithi012.stdout:5/956: write d3/d29/f57 [571043,107744] 0 2024-03-18T14:51:12.084 INFO:tasks.workunit.client.0.smithi012.stdout:1/918: sync 2024-03-18T14:51:12.085 INFO:tasks.workunit.client.1.smithi100.stdout:4/683: mkdir d9/d13/d39/d76/da3/dca 0 2024-03-18T14:51:12.086 INFO:tasks.workunit.client.1.smithi100.stdout:6/665: dwrite d6/d12/f5c [0,4194304] 0 2024-03-18T14:51:12.086 INFO:tasks.workunit.client.1.smithi100.stdout:3/719: sync 2024-03-18T14:51:12.087 INFO:tasks.workunit.client.1.smithi100.stdout:3/720: chown d3/d16/d13/d17/l3a 1068 1 2024-03-18T14:51:12.087 INFO:tasks.workunit.client.1.smithi100.stdout:2/721: sync 2024-03-18T14:51:12.087 INFO:tasks.workunit.client.1.smithi100.stdout:2/722: write d3/d6/d29/f35 [1633951,52459] 0 2024-03-18T14:51:12.088 INFO:tasks.workunit.client.0.smithi012.stdout:4/995: symlink d6/d1a/d64/l156 0 2024-03-18T14:51:12.091 INFO:tasks.workunit.client.0.smithi012.stdout:5/957: unlink d3/d15/d42/dc6/d73/d6b/df3/fde 0 2024-03-18T14:51:12.091 INFO:tasks.workunit.client.0.smithi012.stdout:5/958: fsync d3/d1e/d33/f4b 0 2024-03-18T14:51:12.093 INFO:tasks.workunit.client.1.smithi100.stdout:4/684: write d9/d13/d25/f4d [5540569,122137] 0 2024-03-18T14:51:12.093 INFO:tasks.workunit.client.1.smithi100.stdout:4/685: readlink d9/d13/d39/d76/lb8 0 2024-03-18T14:51:12.095 INFO:tasks.workunit.client.1.smithi100.stdout:0/735: creat dd/d31/d91/d56/d71/f109 x:0 0 0 2024-03-18T14:51:12.096 INFO:tasks.workunit.client.0.smithi012.stdout:1/919: dread d8/d1e/d27/d44/f59 [0,4194304] 0 2024-03-18T14:51:12.096 INFO:tasks.workunit.client.0.smithi012.stdout:5/959: fdatasync d3/d1e/f114 0 2024-03-18T14:51:12.097 INFO:tasks.workunit.client.1.smithi100.stdout:8/770: mknod d4/d5a/c112 0 2024-03-18T14:51:12.099 INFO:tasks.workunit.client.1.smithi100.stdout:9/751: rename da/d2c/c2d to da/d51/cf2 0 2024-03-18T14:51:12.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/920: mkdir d8/d1e/d27/d44/d109/d126 0 2024-03-18T14:51:12.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/921: readlink d8/db/d10/d16/da1/l1f 0 2024-03-18T14:51:12.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/922: write d8/d1e/d27/fe2 [251447,111386] 0 2024-03-18T14:51:12.101 INFO:tasks.workunit.client.0.smithi012.stdout:1/923: fsync d8/d1e/d27/ffa 0 2024-03-18T14:51:12.102 INFO:tasks.workunit.client.0.smithi012.stdout:1/924: readlink d8/db/d10/l79 0 2024-03-18T14:51:12.104 INFO:tasks.workunit.client.0.smithi012.stdout:5/960: unlink d3/f3a 0 2024-03-18T14:51:12.104 INFO:tasks.workunit.client.0.smithi012.stdout:5/961: chown d3/d1e/d33/l69 94 1 2024-03-18T14:51:12.110 INFO:tasks.workunit.client.0.smithi012.stdout:5/962: truncate d3/d1e/f13 1134307 0 2024-03-18T14:51:12.111 INFO:tasks.workunit.client.1.smithi100.stdout:3/721: symlink d3/d16/d13/d1f/d3f/lfa 0 2024-03-18T14:51:12.111 INFO:tasks.workunit.client.1.smithi100.stdout:3/722: truncate d3/d16/dca/fcf 591790 0 2024-03-18T14:51:12.113 INFO:tasks.workunit.client.0.smithi012.stdout:5/963: mknod d3/d29/d9d/dc4/c141 0 2024-03-18T14:51:12.115 INFO:tasks.workunit.client.1.smithi100.stdout:2/723: link d3/da/de/d20/dea/f38 d3/dd4/ff9 0 2024-03-18T14:51:12.116 INFO:tasks.workunit.client.0.smithi012.stdout:1/925: dwrite d8/d1e/d27/d44/dc2/fee [0,4194304] 0 2024-03-18T14:51:12.116 INFO:tasks.workunit.client.0.smithi012.stdout:1/926: truncate d8/db/d10/d31/d81/f11a 9312138 0 2024-03-18T14:51:12.117 INFO:tasks.workunit.client.1.smithi100.stdout:4/686: rmdir d9/d27/d62/d7b/d82/dac 39 2024-03-18T14:51:12.117 INFO:tasks.workunit.client.0.smithi012.stdout:5/964: chown d3/d15/d42/dc6/d73/f76 1528 1 2024-03-18T14:51:12.118 INFO:tasks.workunit.client.1.smithi100.stdout:0/736: symlink dd/d20/d44/df9/l10a 0 2024-03-18T14:51:12.119 INFO:tasks.workunit.client.1.smithi100.stdout:0/737: stat dd/d20/l70 0 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.0.smithi012.stdout:1/927: mknod d8/db/d10/d16/da1/c127 0 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.0.smithi012.stdout:1/928: chown d8/d1e/d27/d44/fa0 218342 1 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.0.smithi012.stdout:1/929: fdatasync d8/db/d10/d16/da1/d2e/fb4 0 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/706: sync 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.1.smithi100.stdout:1/710: sync 2024-03-18T14:51:12.124 INFO:tasks.workunit.client.1.smithi100.stdout:7/707: chown d5/d22/d24/dac 136609116 1 2024-03-18T14:51:12.126 INFO:tasks.workunit.client.0.smithi012.stdout:5/965: symlink d3/d15/d42/dc6/d65/l142 0 2024-03-18T14:51:12.126 INFO:tasks.workunit.client.0.smithi012.stdout:4/996: dwrite d6/d1a/ff9 [0,4194304] 0 2024-03-18T14:51:12.130 INFO:tasks.workunit.client.0.smithi012.stdout:1/930: unlink d8/db/d10/d31/d43/d5e/d65/d84/dc1/fdc 0 2024-03-18T14:51:12.130 INFO:tasks.workunit.client.1.smithi100.stdout:5/682: rename d29/d2d/d32/db2 to d29/d2d/d39/d61/ddd 0 2024-03-18T14:51:12.131 INFO:tasks.workunit.client.1.smithi100.stdout:3/723: write d3/dd/d21/fa6 [5463555,118845] 0 2024-03-18T14:51:12.132 INFO:tasks.workunit.client.0.smithi012.stdout:5/966: creat d3/db6/df0/f143 x:0 0 0 2024-03-18T14:51:12.138 INFO:tasks.workunit.client.1.smithi100.stdout:2/724: link d3/d6/ff7 d3/da/de/d20/d82/dad/de2/ffa 0 2024-03-18T14:51:12.138 INFO:tasks.workunit.client.1.smithi100.stdout:2/725: truncate d3/d6/d29/d36/d49/f51 1560406 0 2024-03-18T14:51:12.139 INFO:tasks.workunit.client.1.smithi100.stdout:2/726: chown d3/d6/d5f/c6b 108 1 2024-03-18T14:51:12.139 INFO:tasks.workunit.client.0.smithi012.stdout:4/997: mkdir d6/da/d89/da1/dbb/ddc/d157 0 2024-03-18T14:51:12.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/687: creat d9/d13/d5a/fcb x:0 0 0 2024-03-18T14:51:12.140 INFO:tasks.workunit.client.1.smithi100.stdout:4/688: write d9/f2b [849980,24388] 0 2024-03-18T14:51:12.142 INFO:tasks.workunit.client.1.smithi100.stdout:0/738: creat dd/d31/f10b x:0 0 0 2024-03-18T14:51:12.143 INFO:tasks.workunit.client.1.smithi100.stdout:8/771: mkdir d4/dc/dd/de/d19/dc0/d113 0 2024-03-18T14:51:12.144 INFO:tasks.workunit.client.0.smithi012.stdout:5/967: mknod d3/d29/d140/c144 0 2024-03-18T14:51:12.144 INFO:tasks.workunit.client.1.smithi100.stdout:1/711: mkdir d8/d1e/d11/d8a/dda 0 2024-03-18T14:51:12.145 INFO:tasks.workunit.client.1.smithi100.stdout:7/708: dwrite d5/d14/d16/d2e/d62/dd1/f74 [0,4194304] 0 2024-03-18T14:51:12.145 INFO:tasks.workunit.client.1.smithi100.stdout:7/709: stat d5/d30/d4d/db4 0 2024-03-18T14:51:12.146 INFO:tasks.workunit.client.1.smithi100.stdout:5/683: creat d29/d2d/d39/fde x:0 0 0 2024-03-18T14:51:12.146 INFO:tasks.workunit.client.1.smithi100.stdout:2/727: symlink d3/d6/d29/d36/d49/lfb 0 2024-03-18T14:51:12.148 INFO:tasks.workunit.client.1.smithi100.stdout:4/689: creat d9/d13/d39/d76/fcc x:0 0 0 2024-03-18T14:51:12.148 INFO:tasks.workunit.client.1.smithi100.stdout:4/690: write d9/d27/fa9 [717671,14639] 0 2024-03-18T14:51:12.148 INFO:tasks.workunit.client.0.smithi012.stdout:5/968: dread d3/d15/d42/dc6/f3e [0,4194304] 0 2024-03-18T14:51:12.149 INFO:tasks.workunit.client.1.smithi100.stdout:4/691: write d9/d13/d5a/d5b/f70 [113479,79810] 0 2024-03-18T14:51:12.150 INFO:tasks.workunit.client.1.smithi100.stdout:0/739: rename dd/d20/c51 to dd/d31/d91/d56/c10c 0 2024-03-18T14:51:12.150 INFO:tasks.workunit.client.1.smithi100.stdout:0/740: readlink dd/d31/d91/d56/d59/laa 0 2024-03-18T14:51:12.150 INFO:tasks.workunit.client.1.smithi100.stdout:8/772: creat d4/dc/dd/de/d19/dc0/d113/f114 x:0 0 0 2024-03-18T14:51:12.152 INFO:tasks.workunit.client.1.smithi100.stdout:1/712: creat d8/d1e/fdb x:0 0 0 2024-03-18T14:51:12.152 INFO:tasks.workunit.client.1.smithi100.stdout:1/713: chown d8/d2a/d37/d50/d6f/d81/d91/fca 146 1 2024-03-18T14:51:12.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/724: dwrite d3/fc1 [0,4194304] 0 2024-03-18T14:51:12.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/725: write d3/fc2 [1236925,91338] 0 2024-03-18T14:51:12.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/726: fsync d3/ff 0 2024-03-18T14:51:12.155 INFO:tasks.workunit.client.1.smithi100.stdout:5/684: link d29/d2d/d39/c58 d29/d2d/d39/d71/d96/cdf 0 2024-03-18T14:51:12.155 INFO:tasks.workunit.client.1.smithi100.stdout:5/685: creat d29/d2d/d39/d71/d96/fe0 x:0 0 0 2024-03-18T14:51:12.155 INFO:tasks.workunit.client.1.smithi100.stdout:5/686: dread - d29/d2d/dbe/dc2/fce zero size 2024-03-18T14:51:12.157 INFO:tasks.workunit.client.1.smithi100.stdout:2/728: getdents d3/d6/d5f/d6a 0 2024-03-18T14:51:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:4/998: dread d6/d1a/d30/d46/f6b [0,4194304] 0 2024-03-18T14:51:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:4/999: chown d6/d1a/d30/d46/d6f/d7c/l10b 32722 1 2024-03-18T14:51:12.164 INFO:tasks.workunit.client.1.smithi100.stdout:4/692: mkdir d9/d13/d5a/d5b/da0/dcd 0 2024-03-18T14:51:12.165 INFO:tasks.workunit.client.1.smithi100.stdout:0/741: symlink dd/d20/d44/l10d 0 2024-03-18T14:51:12.165 INFO:tasks.workunit.client.1.smithi100.stdout:0/742: dread - dd/fa6 zero size 2024-03-18T14:51:12.169 INFO:tasks.workunit.client.1.smithi100.stdout:1/714: write d8/d1e/d11/d27/d83/fa1 [3996693,77577] 0 2024-03-18T14:51:12.172 INFO:tasks.workunit.client.1.smithi100.stdout:3/727: symlink d3/dd/d21/d3e/df8/db9/df0/lfb 0 2024-03-18T14:51:12.177 INFO:tasks.workunit.client.1.smithi100.stdout:5/687: symlink d29/d2d/d39/d83/dc1/dbb/d9e/le1 0 2024-03-18T14:51:12.177 INFO:tasks.workunit.client.1.smithi100.stdout:5/688: stat d29/d2d/d39/d61/ddd 0 2024-03-18T14:51:12.177 INFO:tasks.workunit.client.1.smithi100.stdout:5/689: creat d29/d2d/d39/d65/d7d/fe2 x:0 0 0 2024-03-18T14:51:12.177 INFO:tasks.workunit.client.1.smithi100.stdout:5/690: write d29/d2d/d32/d92/fb9 [1085614,7980] 0 2024-03-18T14:51:12.178 INFO:tasks.workunit.client.1.smithi100.stdout:2/729: dwrite d3/dc3/fd1 [0,4194304] 0 2024-03-18T14:51:12.179 INFO:tasks.workunit.client.1.smithi100.stdout:8/773: dwrite d4/fcc [0,4194304] 0 2024-03-18T14:51:12.180 INFO:tasks.workunit.client.1.smithi100.stdout:0/743: mknod dd/d31/d91/d56/d71/dcc/c10e 0 2024-03-18T14:51:12.183 INFO:tasks.workunit.client.1.smithi100.stdout:3/728: mknod d3/dd/d21/d3e/df8/db9/df0/cfc 0 2024-03-18T14:51:12.183 INFO:tasks.workunit.client.1.smithi100.stdout:3/729: readlink d3/l1c 0 2024-03-18T14:51:12.183 INFO:tasks.workunit.client.1.smithi100.stdout:3/730: chown d3/dd/d21/d3e/df8/df2 7 1 2024-03-18T14:51:12.187 INFO:tasks.workunit.client.0.smithi012.stdout:1/931: dwrite d8/db/f41 [0,4194304] 0 2024-03-18T14:51:12.194 INFO:tasks.workunit.client.0.smithi012.stdout:1/932: rename d8/db/d10/d16/d54/cb2 to d8/d1e/d27/d44/dc2/c128 0 2024-03-18T14:51:12.194 INFO:tasks.workunit.client.0.smithi012.stdout:1/933: chown d8/db/d10/d16/d6b/lbd 133522212 1 2024-03-18T14:51:12.196 INFO:tasks.workunit.client.1.smithi100.stdout:2/730: symlink d3/lfc 0 2024-03-18T14:51:12.196 INFO:tasks.workunit.client.1.smithi100.stdout:2/731: chown d3/d6/d29/d36 1420812 1 2024-03-18T14:51:12.196 INFO:tasks.workunit.client.0.smithi012.stdout:1/934: symlink d8/db/d10/d31/d45/l129 0 2024-03-18T14:51:12.198 INFO:tasks.workunit.client.0.smithi012.stdout:1/935: mkdir d8/db/d10/d31/d43/d5e/d65/d120/d12a 0 2024-03-18T14:51:12.201 INFO:tasks.workunit.client.0.smithi012.stdout:1/936: mkdir d8/db/d10/d31/d43/d5e/da5/d12b 0 2024-03-18T14:51:12.202 INFO:tasks.workunit.client.1.smithi100.stdout:4/693: rmdir d9/d27 39 2024-03-18T14:51:12.204 INFO:tasks.workunit.client.0.smithi012.stdout:1/937: symlink d8/db/d10/d16/da1/db9/l12c 0 2024-03-18T14:51:12.204 INFO:tasks.workunit.client.0.smithi012.stdout:1/938: write d8/d1e/d27/f38 [2216968,89932] 0 2024-03-18T14:51:12.209 INFO:tasks.workunit.client.1.smithi100.stdout:8/774: unlink d4/dc/d31/d66/fde 0 2024-03-18T14:51:12.210 INFO:tasks.workunit.client.1.smithi100.stdout:0/744: truncate dd/d20/d44/f5e 1589309 0 2024-03-18T14:51:12.213 INFO:tasks.workunit.client.1.smithi100.stdout:2/732: creat d3/da/de/d20/ffd x:0 0 0 2024-03-18T14:51:12.213 INFO:tasks.workunit.client.1.smithi100.stdout:2/733: chown d3/da/de/d20/dea/d40/f83 2628192 1 2024-03-18T14:51:12.214 INFO:tasks.workunit.client.1.smithi100.stdout:8/775: creat d4/dc/d1f/d59/d6c/f115 x:0 0 0 2024-03-18T14:51:12.215 INFO:tasks.workunit.client.1.smithi100.stdout:0/745: mknod dd/d20/ddc/c10f 0 2024-03-18T14:51:12.215 INFO:tasks.workunit.client.1.smithi100.stdout:0/746: fsync dd/d2b/d95/d79/d7d/da9/db0/fe4 0 2024-03-18T14:51:12.216 INFO:tasks.workunit.client.1.smithi100.stdout:1/715: rename d8/d1e/d11/d6d/d59/d93 to d8/d1e/d11/d27/d3f/daa/dd5/ddc 0 2024-03-18T14:51:12.218 INFO:tasks.workunit.client.1.smithi100.stdout:2/734: creat d3/ffe x:0 0 0 2024-03-18T14:51:12.218 INFO:tasks.workunit.client.1.smithi100.stdout:2/735: stat d3/da/de/d20/dea/l75 0 2024-03-18T14:51:12.218 INFO:tasks.workunit.client.1.smithi100.stdout:5/691: rename d29/d2d/d39/d61/ddd/cc5 to d29/d2d/ce3 0 2024-03-18T14:51:12.219 INFO:tasks.workunit.client.1.smithi100.stdout:2/736: dread d3/da/de/d20/dea/d40/fa7 [0,4194304] 0 2024-03-18T14:51:12.219 INFO:tasks.workunit.client.1.smithi100.stdout:1/716: symlink d8/d1e/d11/d8a/ldd 0 2024-03-18T14:51:12.221 INFO:tasks.workunit.client.1.smithi100.stdout:3/731: rename d3/fa1 to d3/d16/d13/d1f/d3f/d4f/de0/ffd 0 2024-03-18T14:51:12.222 INFO:tasks.workunit.client.1.smithi100.stdout:1/717: mkdir d8/d30/d61/d8d/dde 0 2024-03-18T14:51:12.223 INFO:tasks.workunit.client.0.smithi012.stdout:1/939: dwrite d8/db/dbe/f112 [0,4194304] 0 2024-03-18T14:51:12.224 INFO:tasks.workunit.client.1.smithi100.stdout:4/694: dwrite d9/f1b [0,4194304] 0 2024-03-18T14:51:12.224 INFO:tasks.workunit.client.1.smithi100.stdout:2/737: rename d3/d6/d29/d36/d49/f5b to d3/da/fff 0 2024-03-18T14:51:12.225 INFO:tasks.workunit.client.1.smithi100.stdout:5/692: getdents d29/d2d/d39/d61 0 2024-03-18T14:51:12.227 INFO:tasks.workunit.client.1.smithi100.stdout:1/718: truncate d8/d2a/f41 1805918 0 2024-03-18T14:51:12.227 INFO:tasks.workunit.client.1.smithi100.stdout:8/776: write d4/dc/dd/de/d21/d23/f27 [1188722,11251] 0 2024-03-18T14:51:12.228 INFO:tasks.workunit.client.1.smithi100.stdout:0/747: dread dd/d31/d91/d56/d71/dcc/dcf/fdf [0,4194304] 0 2024-03-18T14:51:12.232 INFO:tasks.workunit.client.0.smithi012.stdout:1/940: write d8/db/d10/d16/d54/fa3 [3221202,78070] 0 2024-03-18T14:51:12.233 INFO:tasks.workunit.client.1.smithi100.stdout:4/695: creat d9/d13/d39/d76/da3/dca/fce x:0 0 0 2024-03-18T14:51:12.234 INFO:tasks.workunit.client.1.smithi100.stdout:3/732: rename d3/dd/d49/c6e to d3/d16/d13/d1f/d3f/d4f/de0/cfe 0 2024-03-18T14:51:12.236 INFO:tasks.workunit.client.1.smithi100.stdout:5/693: write d29/d2d/d39/f4e [2808231,115060] 0 2024-03-18T14:51:12.237 INFO:tasks.workunit.client.1.smithi100.stdout:1/719: link d8/d1e/d11/d8a/lbe d8/d2a/d37/d50/d6f/d71/ldf 0 2024-03-18T14:51:12.237 INFO:tasks.workunit.client.1.smithi100.stdout:1/720: dread - d8/d1e/d11/d6d/d59/fd7 zero size 2024-03-18T14:51:12.238 INFO:tasks.workunit.client.1.smithi100.stdout:0/748: read dd/d20/f5c [284486,64011] 0 2024-03-18T14:51:12.239 INFO:tasks.workunit.client.1.smithi100.stdout:0/749: read - dd/d20/db5/fab zero size 2024-03-18T14:51:12.239 INFO:tasks.workunit.client.1.smithi100.stdout:0/750: chown dd/d61/df8 14464 1 2024-03-18T14:51:12.239 INFO:tasks.workunit.client.1.smithi100.stdout:0/751: truncate dd/f55 4545304 0 2024-03-18T14:51:12.240 INFO:tasks.workunit.client.1.smithi100.stdout:8/777: dread d4/dc/d31/fd9 [0,4194304] 0 2024-03-18T14:51:12.242 INFO:tasks.workunit.client.0.smithi012.stdout:1/941: creat d8/db/d10/d31/d45/d108/f12d x:0 0 0 2024-03-18T14:51:12.242 INFO:tasks.workunit.client.0.smithi012.stdout:1/942: truncate d8/db/d10/d31/d43/d5e/da5/fb8 5862194 0 2024-03-18T14:51:12.244 INFO:tasks.workunit.client.1.smithi100.stdout:5/694: symlink d29/d2d/d39/d61/ddd/le4 0 2024-03-18T14:51:12.245 INFO:tasks.workunit.client.1.smithi100.stdout:5/695: fsync d29/d2d/d39/d83/dc1/dbb/f6e 0 2024-03-18T14:51:12.245 INFO:tasks.workunit.client.1.smithi100.stdout:5/696: read d29/f37 [407436,113668] 0 2024-03-18T14:51:12.246 INFO:tasks.workunit.client.1.smithi100.stdout:6/666: sync 2024-03-18T14:51:12.248 INFO:tasks.workunit.client.1.smithi100.stdout:3/733: symlink d3/dd/d21/lff 0 2024-03-18T14:51:12.252 INFO:tasks.workunit.client.1.smithi100.stdout:3/734: rmdir d3/d16/d13/d1f/d3f/d4f/de0/de6 39 2024-03-18T14:51:12.252 INFO:tasks.workunit.client.1.smithi100.stdout:3/735: truncate d3/d16/d13/d1f/d37/fb3 816749 0 2024-03-18T14:51:12.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/752: rmdir dd/d20/d44/dce 0 2024-03-18T14:51:12.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/753: readlink dd/d2b/l53 0 2024-03-18T14:51:12.256 INFO:tasks.workunit.client.1.smithi100.stdout:3/736: rename d3/d16/d13/d1f/l77 to d3/d16/d13/d17/d7d/l100 0 2024-03-18T14:51:12.258 INFO:tasks.workunit.client.1.smithi100.stdout:4/696: dwrite d9/d27/d62/f73 [0,4194304] 0 2024-03-18T14:51:12.258 INFO:tasks.workunit.client.1.smithi100.stdout:4/697: fdatasync d9/d27/f3d 0 2024-03-18T14:51:12.258 INFO:tasks.workunit.client.1.smithi100.stdout:0/754: rename dd/d61/cfd to dd/d2b/d95/d79/d7d/da9/db0/c110 0 2024-03-18T14:51:12.258 INFO:tasks.workunit.client.1.smithi100.stdout:0/755: write dd/d20/db5/f102 [685691,22547] 0 2024-03-18T14:51:12.259 INFO:tasks.workunit.client.1.smithi100.stdout:1/721: dwrite d8/d2a/d37/d50/f85 [0,4194304] 0 2024-03-18T14:51:12.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/756: rename dd/d2b/l7a to dd/d31/d91/d65/l111 0 2024-03-18T14:51:12.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/757: creat dd/d2b/d95/d79/d7d/da9/dac/f112 x:0 0 0 2024-03-18T14:51:12.266 INFO:tasks.workunit.client.1.smithi100.stdout:1/722: rename d8/d30/d61/fcf to d8/d2a/d37/fe0 0 2024-03-18T14:51:12.267 INFO:tasks.workunit.client.1.smithi100.stdout:1/723: chown d8/d1e/d11/d27/f4d 479450 1 2024-03-18T14:51:12.267 INFO:tasks.workunit.client.1.smithi100.stdout:1/724: creat d8/d1e/d11/d2d/fe1 x:0 0 0 2024-03-18T14:51:12.269 INFO:tasks.workunit.client.1.smithi100.stdout:9/752: sync 2024-03-18T14:51:12.273 INFO:tasks.workunit.client.1.smithi100.stdout:4/698: dread d9/d3a/f9c [0,4194304] 0 2024-03-18T14:51:12.274 INFO:tasks.workunit.client.1.smithi100.stdout:4/699: write d9/d13/d39/d76/da3/fb1 [454991,67824] 0 2024-03-18T14:51:12.275 INFO:tasks.workunit.client.1.smithi100.stdout:4/700: mknod d9/d13/d39/d76/da3/dca/ccf 0 2024-03-18T14:51:12.276 INFO:tasks.workunit.client.1.smithi100.stdout:4/701: chown d9/d27/l67 116256875 1 2024-03-18T14:51:12.281 INFO:tasks.workunit.client.1.smithi100.stdout:4/702: write d9/f3b [190583,27387] 0 2024-03-18T14:51:12.282 INFO:tasks.workunit.client.1.smithi100.stdout:4/703: unlink d9/d13/d39/d76/da3/dca/fce 0 2024-03-18T14:51:12.283 INFO:tasks.workunit.client.1.smithi100.stdout:4/704: link d9/d3a/c54 d9/d13/dae/cd0 0 2024-03-18T14:51:12.284 INFO:tasks.workunit.client.1.smithi100.stdout:4/705: truncate d9/f10 6525659 0 2024-03-18T14:51:12.285 INFO:tasks.workunit.client.1.smithi100.stdout:4/706: mknod d9/d27/d62/d7b/cd1 0 2024-03-18T14:51:12.286 INFO:tasks.workunit.client.1.smithi100.stdout:4/707: symlink d9/ld2 0 2024-03-18T14:51:12.286 INFO:tasks.workunit.client.1.smithi100.stdout:4/708: write d9/d3a/f86 [902216,4762] 0 2024-03-18T14:51:12.287 INFO:tasks.workunit.client.1.smithi100.stdout:4/709: unlink d9/d17/l4f 0 2024-03-18T14:51:12.291 INFO:tasks.workunit.client.1.smithi100.stdout:9/753: write da/d46/d4d/d62/fb7 [2518500,41209] 0 2024-03-18T14:51:12.295 INFO:tasks.workunit.client.1.smithi100.stdout:4/710: read d9/d27/d62/f7d [6400439,30854] 0 2024-03-18T14:51:12.296 INFO:tasks.workunit.client.1.smithi100.stdout:4/711: creat d9/d13/d5a/d5b/da0/dcd/fd3 x:0 0 0 2024-03-18T14:51:12.297 INFO:tasks.workunit.client.1.smithi100.stdout:4/712: unlink d9/d27/f36 0 2024-03-18T14:51:12.298 INFO:tasks.workunit.client.1.smithi100.stdout:4/713: creat d9/d27/d62/d7b/fd4 x:0 0 0 2024-03-18T14:51:12.299 INFO:tasks.workunit.client.1.smithi100.stdout:4/714: symlink d9/d27/da2/ld5 0 2024-03-18T14:51:12.302 INFO:tasks.workunit.client.1.smithi100.stdout:4/715: rename d9/f22 to d9/dc8/fd6 0 2024-03-18T14:51:12.302 INFO:tasks.workunit.client.1.smithi100.stdout:4/716: write d9/d13/d39/d76/da3/fb1 [1948316,38209] 0 2024-03-18T14:51:12.302 INFO:tasks.workunit.client.1.smithi100.stdout:9/754: dread da/d51/d55/f7b [4194304,4194304] 0 2024-03-18T14:51:12.304 INFO:tasks.workunit.client.1.smithi100.stdout:9/755: creat da/d51/d55/d7e/ff3 x:0 0 0 2024-03-18T14:51:12.304 INFO:tasks.workunit.client.1.smithi100.stdout:9/756: chown da/d51/f69 2266629 1 2024-03-18T14:51:12.304 INFO:tasks.workunit.client.1.smithi100.stdout:6/667: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f5d [0,4194304] 0 2024-03-18T14:51:12.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/668: stat d6/d7/df/d26/d4e/l50 0 2024-03-18T14:51:12.305 INFO:tasks.workunit.client.1.smithi100.stdout:6/669: creat d6/d7/df/d26/d4e/d66/fd0 x:0 0 0 2024-03-18T14:51:12.305 INFO:tasks.workunit.client.0.smithi012.stdout:1/943: dwrite d8/db/f7d [0,4194304] 0 2024-03-18T14:51:12.306 INFO:tasks.workunit.client.1.smithi100.stdout:0/758: dwrite dd/f2e [4194304,4194304] 0 2024-03-18T14:51:12.306 INFO:tasks.workunit.client.1.smithi100.stdout:8/778: dwrite d4/dc/d1f/f5d [0,4194304] 0 2024-03-18T14:51:12.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/725: dwrite d8/d1e/f86 [0,4194304] 0 2024-03-18T14:51:12.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/726: chown d8/d2a/d37/f5f 4441 1 2024-03-18T14:51:12.306 INFO:tasks.workunit.client.1.smithi100.stdout:1/727: chown d8/d30/d61 1443054 1 2024-03-18T14:51:12.311 INFO:tasks.workunit.client.0.smithi012.stdout:1/944: mknod d8/d1e/d27/d44/dc2/de8/c12e 0 2024-03-18T14:51:12.311 INFO:tasks.workunit.client.1.smithi100.stdout:4/717: dread d9/d13/d39/d76/da3/fb1 [0,4194304] 0 2024-03-18T14:51:12.314 INFO:tasks.workunit.client.1.smithi100.stdout:9/757: dread da/d46/d4d/f56 [0,4194304] 0 2024-03-18T14:51:12.318 INFO:tasks.workunit.client.1.smithi100.stdout:0/759: mkdir dd/d31/d91/d56/de0/d113 0 2024-03-18T14:51:12.320 INFO:tasks.workunit.client.1.smithi100.stdout:8/779: creat d4/dc/d31/d85/dd6/df9/f116 x:0 0 0 2024-03-18T14:51:12.321 INFO:tasks.workunit.client.1.smithi100.stdout:6/670: dwrite d6/d7/fc5 [0,4194304] 0 2024-03-18T14:51:12.321 INFO:tasks.workunit.client.1.smithi100.stdout:6/671: write d6/d12/d78/fb2 [1545355,8606] 0 2024-03-18T14:51:12.336 INFO:tasks.workunit.client.1.smithi100.stdout:4/718: rename d9/d13/d25/d51/f5d to d9/fd7 0 2024-03-18T14:51:12.336 INFO:tasks.workunit.client.1.smithi100.stdout:4/719: chown d9/d27/d42 7 1 2024-03-18T14:51:12.337 INFO:tasks.workunit.client.1.smithi100.stdout:9/758: mkdir da/d51/df4 0 2024-03-18T14:51:12.338 INFO:tasks.workunit.client.1.smithi100.stdout:0/760: mkdir dd/d2b/d95/d79/d114 0 2024-03-18T14:51:12.345 INFO:tasks.workunit.client.1.smithi100.stdout:8/780: rename d4/fcc to d4/dc/dd/d48/f117 0 2024-03-18T14:51:12.348 INFO:tasks.workunit.client.1.smithi100.stdout:6/672: rename d6/d7/df/d26/f1c to d6/d7/df/d26/d4e/d66/d36/d4f/dcf/fd1 0 2024-03-18T14:51:12.350 INFO:tasks.workunit.client.1.smithi100.stdout:8/781: symlink d4/d5a/dbb/l118 0 2024-03-18T14:51:12.350 INFO:tasks.workunit.client.1.smithi100.stdout:8/782: dread - d4/dc/d1f/d59/fb1 zero size 2024-03-18T14:51:12.352 INFO:tasks.workunit.client.1.smithi100.stdout:0/761: dwrite dd/d31/d91/f3f [0,4194304] 0 2024-03-18T14:51:12.352 INFO:tasks.workunit.client.1.smithi100.stdout:4/720: rename d9/d13/d39/dc0 to d9/d13/d5a/d5b/da0/dd8 0 2024-03-18T14:51:12.353 INFO:tasks.workunit.client.1.smithi100.stdout:6/673: symlink d6/d7/df/d26/d4e/d66/d36/d4f/ld2 0 2024-03-18T14:51:12.355 INFO:tasks.workunit.client.1.smithi100.stdout:0/762: symlink dd/d31/d91/d56/d71/dcc/dcf/l115 0 2024-03-18T14:51:12.359 INFO:tasks.workunit.client.1.smithi100.stdout:6/674: creat d6/d7/df/d26/d4e/db5/dba/fd3 x:0 0 0 2024-03-18T14:51:12.359 INFO:tasks.workunit.client.1.smithi100.stdout:6/675: write d6/f6c [2262981,67048] 0 2024-03-18T14:51:12.364 INFO:tasks.workunit.client.1.smithi100.stdout:6/676: creat d6/fd4 x:0 0 0 2024-03-18T14:51:12.365 INFO:tasks.workunit.client.1.smithi100.stdout:0/763: rmdir dd/d31/d91/d56/d59 39 2024-03-18T14:51:12.366 INFO:tasks.workunit.client.0.smithi012.stdout:1/945: write d8/db/d10/d31/d43/d5e/f62 [194165,110806] 0 2024-03-18T14:51:12.367 INFO:tasks.workunit.client.1.smithi100.stdout:4/721: dwrite d9/f1b [0,4194304] 0 2024-03-18T14:51:12.367 INFO:tasks.workunit.client.1.smithi100.stdout:9/759: rename da/d46/d4d/d62 to da/dd/d29/d71/df5 0 2024-03-18T14:51:12.367 INFO:tasks.workunit.client.1.smithi100.stdout:9/760: chown da/dd/d29/d71/df5/fd0 3388 1 2024-03-18T14:51:12.368 INFO:tasks.workunit.client.1.smithi100.stdout:6/677: creat d6/d7/df/d26/d4e/fd5 x:0 0 0 2024-03-18T14:51:12.371 INFO:tasks.workunit.client.1.smithi100.stdout:4/722: rmdir d9/d27/d62 39 2024-03-18T14:51:12.371 INFO:tasks.workunit.client.1.smithi100.stdout:4/723: chown d9/d13/d39/d79/f97 896985 1 2024-03-18T14:51:12.371 INFO:tasks.workunit.client.1.smithi100.stdout:4/724: chown d9/d17/f23 12 1 2024-03-18T14:51:12.371 INFO:tasks.workunit.client.1.smithi100.stdout:4/725: read d9/d27/f98 [362827,46633] 0 2024-03-18T14:51:12.371 INFO:tasks.workunit.client.1.smithi100.stdout:0/764: read dd/d20/d44/f5e [962186,43871] 0 2024-03-18T14:51:12.372 INFO:tasks.workunit.client.1.smithi100.stdout:9/761: mknod da/db/db0/cf6 0 2024-03-18T14:51:12.373 INFO:tasks.workunit.client.1.smithi100.stdout:4/726: mknod d9/d13/d39/d76/da3/cd9 0 2024-03-18T14:51:12.374 INFO:tasks.workunit.client.1.smithi100.stdout:0/765: rename dd/d31/d91/d56/d62/cf4 to dd/d2b/d95/d79/d7d/da9/dac/c116 0 2024-03-18T14:51:12.375 INFO:tasks.workunit.client.1.smithi100.stdout:7/710: sync 2024-03-18T14:51:12.375 INFO:tasks.workunit.client.1.smithi100.stdout:9/762: mkdir da/dd/d29/d71/d72/df7 0 2024-03-18T14:51:12.377 INFO:tasks.workunit.client.1.smithi100.stdout:4/727: unlink l6 0 2024-03-18T14:51:12.378 INFO:tasks.workunit.client.1.smithi100.stdout:0/766: mknod dd/d31/d91/d56/d59/c117 0 2024-03-18T14:51:12.380 INFO:tasks.workunit.client.1.smithi100.stdout:7/711: mkdir d5/d22/d24/d6e/d7a/dc2/d111 0 2024-03-18T14:51:12.382 INFO:tasks.workunit.client.1.smithi100.stdout:9/763: creat da/d51/d55/d7e/ff8 x:0 0 0 2024-03-18T14:51:12.382 INFO:tasks.workunit.client.1.smithi100.stdout:4/728: symlink d9/d27/d62/lda 0 2024-03-18T14:51:12.382 INFO:tasks.workunit.client.1.smithi100.stdout:4/729: chown d9/d13/c6f 196203 1 2024-03-18T14:51:12.383 INFO:tasks.workunit.client.1.smithi100.stdout:9/764: getdents da/d88 0 2024-03-18T14:51:12.385 INFO:tasks.workunit.client.0.smithi012.stdout:1/946: dwrite d8/db/d10/d56/dcc/ff9 [0,4194304] 0 2024-03-18T14:51:12.388 INFO:tasks.workunit.client.1.smithi100.stdout:4/730: mknod d9/d13/d5a/cdb 0 2024-03-18T14:51:12.390 INFO:tasks.workunit.client.1.smithi100.stdout:0/767: write dd/d2b/f89 [3204579,106946] 0 2024-03-18T14:51:12.391 INFO:tasks.workunit.client.1.smithi100.stdout:0/768: fsync dd/f49 0 2024-03-18T14:51:12.391 INFO:tasks.workunit.client.1.smithi100.stdout:9/765: unlink da/d46/fa3 0 2024-03-18T14:51:12.391 INFO:tasks.workunit.client.1.smithi100.stdout:9/766: dread - da/dcb/fee zero size 2024-03-18T14:51:12.392 INFO:tasks.workunit.client.1.smithi100.stdout:9/767: dread da/dd/d3b/fbc [0,4194304] 0 2024-03-18T14:51:12.392 INFO:tasks.workunit.client.1.smithi100.stdout:9/768: write da/d88/f8c [2528701,11458] 0 2024-03-18T14:51:12.393 INFO:tasks.workunit.client.1.smithi100.stdout:7/712: dwrite d5/d14/d16/d44/f92 [0,4194304] 0 2024-03-18T14:51:12.393 INFO:tasks.workunit.client.1.smithi100.stdout:7/713: stat d5/l2f 0 2024-03-18T14:51:12.394 INFO:tasks.workunit.client.1.smithi100.stdout:0/769: symlink dd/l118 0 2024-03-18T14:51:12.394 INFO:tasks.workunit.client.1.smithi100.stdout:0/770: write dd/d61/fe8 [241175,52491] 0 2024-03-18T14:51:12.395 INFO:tasks.workunit.client.1.smithi100.stdout:0/771: creat dd/d31/d91/d56/d62/d6e/f119 x:0 0 0 2024-03-18T14:51:12.395 INFO:tasks.workunit.client.1.smithi100.stdout:0/772: write dd/d20/d44/f9e [862202,102244] 0 2024-03-18T14:51:12.400 INFO:tasks.workunit.client.1.smithi100.stdout:9/769: unlink da/l1e 0 2024-03-18T14:51:12.403 INFO:tasks.workunit.client.0.smithi012.stdout:1/947: link c1 d8/c12f 0 2024-03-18T14:51:12.403 INFO:tasks.workunit.client.0.smithi012.stdout:1/948: creat d8/d1e/d27/d44/dc2/f130 x:0 0 0 2024-03-18T14:51:12.403 INFO:tasks.workunit.client.0.smithi012.stdout:1/949: chown d8/db/d10/d31/d81/dc5/le5 254588 1 2024-03-18T14:51:12.406 INFO:tasks.workunit.client.1.smithi100.stdout:7/714: mknod d5/d14/d16/d2e/d62/dd1/d86/de6/c112 0 2024-03-18T14:51:12.406 INFO:tasks.workunit.client.1.smithi100.stdout:7/715: write d5/d14/d16/d2e/d62/d66/fbf [685443,87634] 0 2024-03-18T14:51:12.409 INFO:tasks.workunit.client.0.smithi012.stdout:1/950: mknod d8/db/c131 0 2024-03-18T14:51:12.411 INFO:tasks.workunit.client.1.smithi100.stdout:0/773: creat dd/d2b/f11a x:0 0 0 2024-03-18T14:51:12.412 INFO:tasks.workunit.client.1.smithi100.stdout:9/770: mkdir da/d46/dd6/df9 0 2024-03-18T14:51:12.413 INFO:tasks.workunit.client.0.smithi012.stdout:1/951: rmdir d8/db/d10/d31/d43/d5e/d65/d84 39 2024-03-18T14:51:12.415 INFO:tasks.workunit.client.1.smithi100.stdout:9/771: fdatasync da/dd/d29/d71/d72/f78 0 2024-03-18T14:51:12.416 INFO:tasks.workunit.client.1.smithi100.stdout:9/772: fsync da/dd/d3c/fc9 0 2024-03-18T14:51:12.416 INFO:tasks.workunit.client.1.smithi100.stdout:7/716: mkdir d5/d22/d3b/df5/d113 0 2024-03-18T14:51:12.418 INFO:tasks.workunit.client.0.smithi012.stdout:1/952: rename d8/db/d10/d31/d45/lb6 to d8/db/d10/d31/d43/d5e/d65/d70/dc7/l132 0 2024-03-18T14:51:12.421 INFO:tasks.workunit.client.1.smithi100.stdout:7/717: dread d5/fd [0,4194304] 0 2024-03-18T14:51:12.422 INFO:tasks.workunit.client.1.smithi100.stdout:9/773: creat da/dd/d3e/ffa x:0 0 0 2024-03-18T14:51:12.423 INFO:tasks.workunit.client.1.smithi100.stdout:7/718: dread d5/d14/d16/d2e/d62/ff2 [0,4194304] 0 2024-03-18T14:51:12.424 INFO:tasks.workunit.client.0.smithi012.stdout:1/953: link d8/db/d10/d31/d43/d5e/d65/d70/lc9 d8/db/d10/d31/l133 0 2024-03-18T14:51:12.425 INFO:tasks.workunit.client.0.smithi012.stdout:1/954: chown d8/db/d10/d4e/f53 3 1 2024-03-18T14:51:12.428 INFO:tasks.workunit.client.0.smithi012.stdout:1/955: mknod d8/db/d10/d31/d43/d5e/d65/d120/d12a/c134 0 2024-03-18T14:51:12.428 INFO:tasks.workunit.client.0.smithi012.stdout:1/956: write d8/d1e/d27/d44/dc2/f10b [695422,85901] 0 2024-03-18T14:51:12.431 INFO:tasks.workunit.client.0.smithi012.stdout:3/936: sync 2024-03-18T14:51:12.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/969: sync 2024-03-18T14:51:12.431 INFO:tasks.workunit.client.0.smithi012.stdout:5/970: stat d3/d15/d42/dc6/d73/dd3/dfb 0 2024-03-18T14:51:12.432 INFO:tasks.workunit.client.1.smithi100.stdout:4/731: dwrite d9/d13/d5a/f78 [0,4194304] 0 2024-03-18T14:51:12.433 INFO:tasks.workunit.client.1.smithi100.stdout:9/774: truncate da/d2c/d2f/f31 4128405 0 2024-03-18T14:51:12.435 INFO:tasks.workunit.client.1.smithi100.stdout:0/774: dwrite dd/f2e [4194304,4194304] 0 2024-03-18T14:51:12.436 INFO:tasks.workunit.client.0.smithi012.stdout:5/971: truncate d3/d15/d42/dc6/d73/d6b/df3/f7c 6379148 0 2024-03-18T14:51:12.436 INFO:tasks.workunit.client.0.smithi012.stdout:5/972: dread - d3/d84/dfa/f10c zero size 2024-03-18T14:51:12.439 INFO:tasks.workunit.client.1.smithi100.stdout:0/775: dread dd/d20/f5c [0,4194304] 0 2024-03-18T14:51:12.439 INFO:tasks.workunit.client.1.smithi100.stdout:7/719: rename d5/d14/d16/d2e/d62/dd1/f74 to d5/d22/d3b/d9f/da0/db1/dd3/f114 0 2024-03-18T14:51:12.442 INFO:tasks.workunit.client.0.smithi012.stdout:5/973: mkdir d3/d84/dfa/d145 0 2024-03-18T14:51:12.445 INFO:tasks.workunit.client.1.smithi100.stdout:4/732: creat d9/da6/fdc x:0 0 0 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:3/937: write dc/d3c/f74 [2714971,4840] 0 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:3/938: dread - dc/d3c/d4c/d57/d103/fdc zero size 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:3/939: fdatasync dc/d3c/d4c/d5f/f104 0 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:3/940: chown dc/d3c/d4c/d57/d103/d9a/d64/c8d 206 1 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:5/974: symlink d3/d15/d42/dc6/d73/d6b/df3/l146 0 2024-03-18T14:51:12.447 INFO:tasks.workunit.client.0.smithi012.stdout:5/975: truncate d3/d1e/f87 534901 0 2024-03-18T14:51:12.449 INFO:tasks.workunit.client.1.smithi100.stdout:9/775: rmdir da/d51/d55/d7e/dc1 0 2024-03-18T14:51:12.452 INFO:tasks.workunit.client.1.smithi100.stdout:4/733: dread d9/d3a/f3c [0,4194304] 0 2024-03-18T14:51:12.452 INFO:tasks.workunit.client.1.smithi100.stdout:4/734: write d9/fb [2907486,54408] 0 2024-03-18T14:51:12.453 INFO:tasks.workunit.client.1.smithi100.stdout:0/776: rename dd/d20/l70 to dd/l11b 0 2024-03-18T14:51:12.456 INFO:tasks.workunit.client.1.smithi100.stdout:7/720: unlink d5/d22/d24/d6e/cae 0 2024-03-18T14:51:12.456 INFO:tasks.workunit.client.1.smithi100.stdout:7/721: read - d5/d30/d4d/db4/fdb zero size 2024-03-18T14:51:12.458 INFO:tasks.workunit.client.0.smithi012.stdout:5/976: creat d3/d15/d42/dc6/d73/d6b/d8f/f147 x:0 0 0 2024-03-18T14:51:12.458 INFO:tasks.workunit.client.1.smithi100.stdout:9/776: symlink da/d46/dd6/de0/lfb 0 2024-03-18T14:51:12.459 INFO:tasks.workunit.client.0.smithi012.stdout:1/957: dwrite d8/fc [4194304,4194304] 0 2024-03-18T14:51:12.460 INFO:tasks.workunit.client.0.smithi012.stdout:3/941: getdents dc/d3c/d4c/d5f 0 2024-03-18T14:51:12.466 INFO:tasks.workunit.client.0.smithi012.stdout:1/958: link d8/db/d10/d31/d45/d108/f12d d8/db/d10/d31/d43/d5e/d65/d84/dc1/f135 0 2024-03-18T14:51:12.466 INFO:tasks.workunit.client.0.smithi012.stdout:1/959: creat d8/db/d10/d31/d45/d108/f136 x:0 0 0 2024-03-18T14:51:12.466 INFO:tasks.workunit.client.0.smithi012.stdout:1/960: chown d8/d1e/d27/d44/d109/d126 2 1 2024-03-18T14:51:12.467 INFO:tasks.workunit.client.0.smithi012.stdout:5/977: write d3/d29/d35/fd4 [1925008,47388] 0 2024-03-18T14:51:12.467 INFO:tasks.workunit.client.1.smithi100.stdout:0/777: mkdir dd/d31/d91/d7b/d11c 0 2024-03-18T14:51:12.469 INFO:tasks.workunit.client.0.smithi012.stdout:3/942: chown dc/d30/d3a/db7/ce1 5858799 1 2024-03-18T14:51:12.469 INFO:tasks.workunit.client.1.smithi100.stdout:7/722: getdents d5/d14/d16/d2e/d62/dd1/d86/de6 0 2024-03-18T14:51:12.474 INFO:tasks.workunit.client.0.smithi012.stdout:1/961: mkdir d8/db/d10/d16/da1/d2e/d9c/d9d/d137 0 2024-03-18T14:51:12.474 INFO:tasks.workunit.client.0.smithi012.stdout:1/962: readlink d8/db/d10/d31/d43/l76 0 2024-03-18T14:51:12.476 INFO:tasks.workunit.client.1.smithi100.stdout:4/735: getdents d9/d3a 0 2024-03-18T14:51:12.479 INFO:tasks.workunit.client.1.smithi100.stdout:0/778: getdents dd/d2b/d95/d79/d7d 0 2024-03-18T14:51:12.479 INFO:tasks.workunit.client.1.smithi100.stdout:7/723: write d5/d22/d24/f91 [3174347,26121] 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.1.smithi100.stdout:7/724: write d5/d22/d3b/d9f/da0/fa8 [949970,76533] 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.1.smithi100.stdout:7/725: getdents d5/d22/d3b/df5/d113 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.1.smithi100.stdout:7/726: write d5/d22/d24/fcd [1464082,25387] 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.1.smithi100.stdout:7/727: fdatasync d5/d14/d16/d44/f92 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.1.smithi100.stdout:7/728: truncate d5/d14/ffc 758600 0 2024-03-18T14:51:12.480 INFO:tasks.workunit.client.0.smithi012.stdout:5/978: write d3/d29/f7a [3998948,80379] 0 2024-03-18T14:51:12.482 INFO:tasks.workunit.client.0.smithi012.stdout:3/943: unlink dc/fcb 0 2024-03-18T14:51:12.484 INFO:tasks.workunit.client.0.smithi012.stdout:1/963: creat d8/d1e/d27/d44/dc2/de8/f138 x:0 0 0 2024-03-18T14:51:12.486 INFO:tasks.workunit.client.0.smithi012.stdout:3/944: dread dc/d3c/d72/df3/d109/d11a/f120 [0,4194304] 0 2024-03-18T14:51:12.487 INFO:tasks.workunit.client.0.smithi012.stdout:3/945: read dc/d3c/d72/f7b [986110,116289] 0 2024-03-18T14:51:12.487 INFO:tasks.workunit.client.1.smithi100.stdout:7/729: write d5/d22/d24/f8b [2359814,120173] 0 2024-03-18T14:51:12.488 INFO:tasks.workunit.client.0.smithi012.stdout:1/964: chown d8/db/d10/d56/c71 3 1 2024-03-18T14:51:12.490 INFO:tasks.workunit.client.1.smithi100.stdout:4/736: creat d9/d17/fdd x:0 0 0 2024-03-18T14:51:12.491 INFO:tasks.workunit.client.0.smithi012.stdout:3/946: mknod dc/d116/c12e 0 2024-03-18T14:51:12.491 INFO:tasks.workunit.client.0.smithi012.stdout:3/947: write dc/d30/dab/fff [389550,82165] 0 2024-03-18T14:51:12.492 INFO:tasks.workunit.client.1.smithi100.stdout:0/779: getdents dd/d2b/d95 0 2024-03-18T14:51:12.495 INFO:tasks.workunit.client.0.smithi012.stdout:1/965: mknod d8/d1e/d106/c139 0 2024-03-18T14:51:12.495 INFO:tasks.workunit.client.0.smithi012.stdout:1/966: stat d8/db/fd 0 2024-03-18T14:51:12.498 INFO:tasks.workunit.client.0.smithi012.stdout:5/979: dwrite d3/d15/d42/dc6/d73/f92 [0,4194304] 0 2024-03-18T14:51:12.499 INFO:tasks.workunit.client.0.smithi012.stdout:3/948: rename dc/d3c/d72/cf1 to dc/d3c/d72/df3/c12f 0 2024-03-18T14:51:12.500 INFO:tasks.workunit.client.1.smithi100.stdout:9/777: rename da/d46/d4d/d67 to da/dc3/dfc 0 2024-03-18T14:51:12.502 INFO:tasks.workunit.client.0.smithi012.stdout:1/967: creat d8/db/d10/f13a x:0 0 0 2024-03-18T14:51:12.509 INFO:tasks.workunit.client.1.smithi100.stdout:7/730: mknod d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6/c115 0 2024-03-18T14:51:12.513 INFO:tasks.workunit.client.0.smithi012.stdout:3/949: link dc/d91/f10d dc/d116/f130 0 2024-03-18T14:51:12.513 INFO:tasks.workunit.client.0.smithi012.stdout:3/950: readlink dc/d3c/d72/le5 0 2024-03-18T14:51:12.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/731: mkdir d5/d30/d4d/db4/d116 0 2024-03-18T14:51:12.517 INFO:tasks.workunit.client.1.smithi100.stdout:7/732: chown d5/d14/d16/d2e/d62/d66/d4f/ca4 11 1 2024-03-18T14:51:12.518 INFO:tasks.workunit.client.1.smithi100.stdout:7/733: creat d5/d14/d16/d2e/d62/dfe/f117 x:0 0 0 2024-03-18T14:51:12.527 INFO:tasks.workunit.client.0.smithi012.stdout:5/980: dwrite d3/d15/d42/dc6/f6e [0,4194304] 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.0.smithi012.stdout:1/968: dwrite d8/d1e/d27/ffa [0,4194304] 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.0.smithi012.stdout:1/969: chown d8/d1e/d27 12854138 1 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/778: dwrite da/dd/f42 [0,4194304] 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/779: truncate da/d2c/d2f/f31 4711749 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/780: creat da/db/ffd x:0 0 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/781: dread da/d46/d4d/f9f [0,4194304] 0 2024-03-18T14:51:12.528 INFO:tasks.workunit.client.1.smithi100.stdout:9/782: dread - da/d2c/d2f/fe5 zero size 2024-03-18T14:51:12.529 INFO:tasks.workunit.client.1.smithi100.stdout:9/783: stat da/d51/d55/l7c 0 2024-03-18T14:51:12.530 INFO:tasks.workunit.client.1.smithi100.stdout:9/784: dread da/dd/fa1 [0,4194304] 0 2024-03-18T14:51:12.530 INFO:tasks.workunit.client.0.smithi012.stdout:5/981: dread d3/d15/d42/dc6/d86/f94 [0,4194304] 0 2024-03-18T14:51:12.534 INFO:tasks.workunit.client.0.smithi012.stdout:1/970: mkdir d8/db/d10/d16/da1/d2e/d13b 0 2024-03-18T14:51:12.536 INFO:tasks.workunit.client.1.smithi100.stdout:7/734: dwrite d5/d22/d3b/d47/fd5 [0,4194304] 0 2024-03-18T14:51:12.536 INFO:tasks.workunit.client.1.smithi100.stdout:7/735: creat d5/d22/d3b/d9f/da0/db1/dd3/f118 x:0 0 0 2024-03-18T14:51:12.536 INFO:tasks.workunit.client.0.smithi012.stdout:5/982: rename d3/d15/lb8 to d3/dad/df2/l148 0 2024-03-18T14:51:12.538 INFO:tasks.workunit.client.0.smithi012.stdout:1/971: mknod d8/db/d10/d31/c13c 0 2024-03-18T14:51:12.538 INFO:tasks.workunit.client.0.smithi012.stdout:1/972: fsync d8/d1e/d27/d44/dc2/de8/f138 0 2024-03-18T14:51:12.539 INFO:tasks.workunit.client.1.smithi100.stdout:4/737: write d9/d13/d39/d4c/f5e [1117475,90259] 0 2024-03-18T14:51:12.543 INFO:tasks.workunit.client.0.smithi012.stdout:1/973: truncate d8/db/d10/f60 2589315 0 2024-03-18T14:51:12.543 INFO:tasks.workunit.client.0.smithi012.stdout:1/974: chown d8/db/d10/d16/da1/d2e/f100 28836 1 2024-03-18T14:51:12.543 INFO:tasks.workunit.client.0.smithi012.stdout:1/975: dread - d8/db/d10/f13a zero size 2024-03-18T14:51:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: pgmap v10: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 212 MiB/s rd, 241 MiB/s wr, 376 op/s 2024-03-18T14:51:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/712650625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:51:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:12 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:12.545 INFO:tasks.workunit.client.1.smithi100.stdout:2/738: sync 2024-03-18T14:51:12.545 INFO:tasks.workunit.client.1.smithi100.stdout:4/738: getdents d9/d13/d39 0 2024-03-18T14:51:12.547 INFO:tasks.workunit.client.1.smithi100.stdout:2/739: creat d3/dd4/f100 x:0 0 0 2024-03-18T14:51:12.548 INFO:tasks.workunit.client.1.smithi100.stdout:2/740: creat d3/dd4/f101 x:0 0 0 2024-03-18T14:51:12.548 INFO:tasks.workunit.client.0.smithi012.stdout:1/976: mknod d8/db/d10/d31/d43/d5e/d65/d84/dc1/c13d 0 2024-03-18T14:51:12.549 INFO:tasks.workunit.client.0.smithi012.stdout:1/977: chown d8/f117 1705626 1 2024-03-18T14:51:12.550 INFO:tasks.workunit.client.1.smithi100.stdout:4/739: symlink d9/d13/d39/d76/lde 0 2024-03-18T14:51:12.550 INFO:tasks.workunit.client.0.smithi012.stdout:3/951: write dc/d3c/d72/fc0 [1124900,5112] 0 2024-03-18T14:51:12.552 INFO:tasks.workunit.client.0.smithi012.stdout:1/978: symlink d8/d1e/d27/d44/l13e 0 2024-03-18T14:51:12.552 INFO:tasks.workunit.client.0.smithi012.stdout:1/979: write d8/db/d10/d4e/f53 [1757764,71214] 0 2024-03-18T14:51:12.552 INFO:tasks.workunit.client.0.smithi012.stdout:1/980: write d8/db/d10/d16/d54/fa3 [4491856,64087] 0 2024-03-18T14:51:12.552 INFO:tasks.workunit.client.0.smithi012.stdout:1/981: chown d8/cd5 62 1 2024-03-18T14:51:12.553 INFO:tasks.workunit.client.1.smithi100.stdout:7/736: dwrite d5/d14/d16/d44/fc9 [0,4194304] 0 2024-03-18T14:51:12.553 INFO:tasks.workunit.client.1.smithi100.stdout:7/737: chown d5/d22/db3/fef 1802314 1 2024-03-18T14:51:12.553 INFO:tasks.workunit.client.1.smithi100.stdout:4/740: creat d9/d13/d25/fdf x:0 0 0 2024-03-18T14:51:12.560 INFO:tasks.workunit.client.1.smithi100.stdout:4/741: unlink l5 0 2024-03-18T14:51:12.560 INFO:tasks.workunit.client.1.smithi100.stdout:7/738: dread d5/d22/d3b/d9f/da0/db1/dd3/f114 [0,4194304] 0 2024-03-18T14:51:12.561 INFO:tasks.workunit.client.0.smithi012.stdout:1/982: write d8/d1e/d27/d44/dc2/f113 [3475628,12198] 0 2024-03-18T14:51:12.561 INFO:tasks.workunit.client.0.smithi012.stdout:1/983: write d8/d1e/d27/d44/dc2/de8/f138 [82670,28898] 0 2024-03-18T14:51:12.562 INFO:tasks.workunit.client.0.smithi012.stdout:3/952: dread f6 [4194304,4194304] 0 2024-03-18T14:51:12.562 INFO:tasks.workunit.client.0.smithi012.stdout:5/983: write d3/d84/f9c [431878,54180] 0 2024-03-18T14:51:12.562 INFO:tasks.workunit.client.1.smithi100.stdout:4/742: creat d9/d13/d5a/da4/fe0 x:0 0 0 2024-03-18T14:51:12.562 INFO:tasks.workunit.client.1.smithi100.stdout:4/743: write d9/d27/d62/d7b/f93 [736150,3150] 0 2024-03-18T14:51:12.563 INFO:tasks.workunit.client.1.smithi100.stdout:7/739: mknod d5/d22/d3b/d9f/c119 0 2024-03-18T14:51:12.566 INFO:tasks.workunit.client.1.smithi100.stdout:4/744: rmdir d9/d13/d5a/d5b/da0 39 2024-03-18T14:51:12.566 INFO:tasks.workunit.client.1.smithi100.stdout:4/745: readlink d9/d17/l49 0 2024-03-18T14:51:12.568 INFO:tasks.workunit.client.0.smithi012.stdout:3/953: creat dc/d30/d3a/da0/f131 x:0 0 0 2024-03-18T14:51:12.573 INFO:tasks.workunit.client.1.smithi100.stdout:4/746: unlink d9/d13/d5a/d5b/da0/cb0 0 2024-03-18T14:51:12.574 INFO:tasks.workunit.client.0.smithi012.stdout:5/984: write d3/d15/d42/dc6/d73/f99 [646411,6195] 0 2024-03-18T14:51:12.574 INFO:tasks.workunit.client.0.smithi012.stdout:5/985: dread - d3/db6/fc9 zero size 2024-03-18T14:51:12.574 INFO:tasks.workunit.client.0.smithi012.stdout:5/986: chown d3/d29/d36/d9b/ca2 31 1 2024-03-18T14:51:12.575 INFO:tasks.workunit.client.0.smithi012.stdout:5/987: write d3/d15/d42/dc6/d86/f136 [507791,70150] 0 2024-03-18T14:51:12.575 INFO:tasks.workunit.client.0.smithi012.stdout:5/988: chown d3/f43 5 1 2024-03-18T14:51:12.575 INFO:tasks.workunit.client.0.smithi012.stdout:5/989: chown d3/d29/d9d/dba 13540 1 2024-03-18T14:51:12.577 INFO:tasks.workunit.client.0.smithi012.stdout:5/990: mkdir d3/dad/df2/d102/d13f/d149 0 2024-03-18T14:51:12.581 INFO:tasks.workunit.client.0.smithi012.stdout:1/984: dwrite d8/db/d10/d31/d43/d5e/d65/d70/dc7/fe9 [0,4194304] 0 2024-03-18T14:51:12.582 INFO:tasks.workunit.client.0.smithi012.stdout:1/985: write d8/db/d10/d31/d43/d5e/da5/fb8 [1276984,118975] 0 2024-03-18T14:51:12.585 INFO:tasks.workunit.client.0.smithi012.stdout:1/986: creat d8/db/d10/d16/d6b/d107/f13f x:0 0 0 2024-03-18T14:51:12.587 INFO:tasks.workunit.client.0.smithi012.stdout:1/987: read d8/db/d10/d31/d43/d5e/d65/fda [382901,86118] 0 2024-03-18T14:51:12.598 INFO:tasks.workunit.client.0.smithi012.stdout:1/988: read d8/dbf/ff0 [2412320,86676] 0 2024-03-18T14:51:12.598 INFO:tasks.workunit.client.0.smithi012.stdout:1/989: write d8/d1e/d27/d44/dc2/de8/f138 [289611,58299] 0 2024-03-18T14:51:12.598 INFO:tasks.workunit.client.0.smithi012.stdout:1/990: write d8/d1e/d27/f48 [4934163,19592] 0 2024-03-18T14:51:12.599 INFO:tasks.workunit.client.0.smithi012.stdout:1/991: readlink d8/db/d10/d31/d81/dc5/le5 0 2024-03-18T14:51:12.599 INFO:tasks.workunit.client.0.smithi012.stdout:1/992: chown d8/d1e/d27/d44/dc2/f113 548 1 2024-03-18T14:51:12.599 INFO:tasks.workunit.client.0.smithi012.stdout:1/993: read - d8/f117 zero size 2024-03-18T14:51:12.599 INFO:tasks.workunit.client.0.smithi012.stdout:1/994: truncate d8/d1e/d27/fb7 24748 0 2024-03-18T14:51:12.604 INFO:tasks.workunit.client.0.smithi012.stdout:3/954: dwrite dc/d30/dab/fd0 [0,4194304] 0 2024-03-18T14:51:12.604 INFO:tasks.workunit.client.0.smithi012.stdout:3/955: dread - dc/d30/d3a/da0/dc5/f115 zero size 2024-03-18T14:51:12.605 INFO:tasks.workunit.client.1.smithi100.stdout:4/747: dwrite d9/d27/d62/f6c [0,4194304] 0 2024-03-18T14:51:12.606 INFO:tasks.workunit.client.1.smithi100.stdout:7/740: dwrite d5/d14/ffc [0,4194304] 0 2024-03-18T14:51:12.606 INFO:tasks.workunit.client.0.smithi012.stdout:5/991: dwrite d3/d29/d35/fea [0,4194304] 0 2024-03-18T14:51:12.606 INFO:tasks.workunit.client.0.smithi012.stdout:5/992: write d3/d29/d9d/dba/f10b [551683,97855] 0 2024-03-18T14:51:12.606 INFO:tasks.workunit.client.0.smithi012.stdout:5/993: dread - d3/d15/d42/dc6/d73/d6b/d8f/fe9 zero size 2024-03-18T14:51:12.607 INFO:tasks.workunit.client.1.smithi100.stdout:4/748: dread d9/d13/d25/f48 [0,4194304] 0 2024-03-18T14:51:12.607 INFO:tasks.workunit.client.1.smithi100.stdout:4/749: chown d9/d27/l67 2061 1 2024-03-18T14:51:12.609 INFO:tasks.workunit.client.0.smithi012.stdout:1/995: dread d8/db/d10/d31/d81/f11a [0,4194304] 0 2024-03-18T14:51:12.614 INFO:tasks.workunit.client.0.smithi012.stdout:5/994: creat d3/d15/d42/dc6/d65/f14a x:0 0 0 2024-03-18T14:51:12.618 INFO:tasks.workunit.client.0.smithi012.stdout:1/996: mkdir d8/db/d10/d56/dcc/d140 0 2024-03-18T14:51:12.618 INFO:tasks.workunit.client.0.smithi012.stdout:1/997: chown d8/d1e/d27/f48 1660920 1 2024-03-18T14:51:12.618 INFO:tasks.workunit.client.0.smithi012.stdout:1/998: stat d8/l24 0 2024-03-18T14:51:12.623 INFO:tasks.workunit.client.1.smithi100.stdout:7/741: mknod d5/d22/d24/d6e/d7a/c11a 0 2024-03-18T14:51:12.625 INFO:tasks.workunit.client.0.smithi012.stdout:3/956: dwrite dc/d3c/f6c [0,4194304] 0 2024-03-18T14:51:12.628 INFO:tasks.workunit.client.1.smithi100.stdout:4/750: getdents d9/d13/d5a/da4 0 2024-03-18T14:51:12.630 INFO:tasks.workunit.client.0.smithi012.stdout:1/999: write d8/db/dbe/f112 [3473975,54158] 0 2024-03-18T14:51:12.636 INFO:tasks.workunit.client.0.smithi012.stdout:3/957: dread dc/d30/fc4 [0,4194304] 0 2024-03-18T14:51:12.640 INFO:tasks.workunit.client.1.smithi100.stdout:4/751: dread d9/d13/d25/f4d [0,4194304] 0 2024-03-18T14:51:12.641 INFO:tasks.workunit.client.0.smithi012.stdout:3/958: rename dc/d3c/d72/df3/d109/f10c to dc/d3c/d4c/f132 0 2024-03-18T14:51:12.643 INFO:tasks.workunit.client.1.smithi100.stdout:7/742: dwrite d5/d14/d16/d2e/d62/dfe/f117 [0,4194304] 0 2024-03-18T14:51:12.643 INFO:tasks.workunit.client.1.smithi100.stdout:7/743: write d5/d14/d16/d2e/d62/d66/fc1 [492385,37226] 0 2024-03-18T14:51:12.643 INFO:tasks.workunit.client.1.smithi100.stdout:4/752: mkdir d9/d17/de1 0 2024-03-18T14:51:12.644 INFO:tasks.workunit.client.0.smithi012.stdout:3/959: rmdir dc/d3c/d72/df3/d109/d11a 39 2024-03-18T14:51:12.644 INFO:tasks.workunit.client.0.smithi012.stdout:3/960: write dc/d30/d3a/fd9 [602761,48159] 0 2024-03-18T14:51:12.645 INFO:tasks.workunit.client.1.smithi100.stdout:7/744: mknod d5/d22/d24/d6e/d7a/dc2/c11b 0 2024-03-18T14:51:12.645 INFO:tasks.workunit.client.1.smithi100.stdout:7/745: fdatasync d5/d14/d16/d2e/d62/dd1/d86/f60 0 2024-03-18T14:51:12.646 INFO:tasks.workunit.client.0.smithi012.stdout:3/961: creat dc/d30/d3a/db7/dd3/f133 x:0 0 0 2024-03-18T14:51:12.647 INFO:tasks.workunit.client.1.smithi100.stdout:4/753: mknod d9/dc8/ce2 0 2024-03-18T14:51:12.649 INFO:tasks.workunit.client.1.smithi100.stdout:4/754: mkdir d9/d13/de3 0 2024-03-18T14:51:12.649 INFO:tasks.workunit.client.1.smithi100.stdout:4/755: fdatasync d9/d27/d42/f72 0 2024-03-18T14:51:12.649 INFO:tasks.workunit.client.1.smithi100.stdout:4/756: truncate d9/d13/d5a/d5b/f70 1351594 0 2024-03-18T14:51:12.649 INFO:tasks.workunit.client.1.smithi100.stdout:4/757: stat d9/f2b 0 2024-03-18T14:51:12.658 INFO:tasks.workunit.client.0.smithi012.stdout:3/962: dread dc/d30/d3a/db7/f11f [0,4194304] 0 2024-03-18T14:51:12.658 INFO:tasks.workunit.client.0.smithi012.stdout:3/963: write dc/d3c/fda [280223,40090] 0 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:5/697: sync 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:3/737: sync 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:6/678: sync 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:1/728: sync 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:8/783: sync 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:5/698: creat d29/d2d/d39/d65/d7d/fe5 x:0 0 0 2024-03-18T14:51:12.659 INFO:tasks.workunit.client.1.smithi100.stdout:5/699: write f7 [531045,18708] 0 2024-03-18T14:51:12.661 INFO:tasks.workunit.client.1.smithi100.stdout:1/729: link d8/d30/d61/da2/la5 d8/d30/d61/d8d/le2 0 2024-03-18T14:51:12.661 INFO:tasks.workunit.client.1.smithi100.stdout:6/679: symlink d6/d7/df/d26/d4e/db5/dba/ld6 0 2024-03-18T14:51:12.661 INFO:tasks.workunit.client.1.smithi100.stdout:1/730: write d8/d2a/d37/d50/d6f/fac [515128,67802] 0 2024-03-18T14:51:12.663 INFO:tasks.workunit.client.1.smithi100.stdout:1/731: stat d8/d1e/d11/d2d/l2c 0 2024-03-18T14:51:12.664 INFO:tasks.workunit.client.1.smithi100.stdout:1/732: fdatasync d8/d2a/d37/f66 0 2024-03-18T14:51:12.664 INFO:tasks.workunit.client.1.smithi100.stdout:1/733: chown d8/d1e/d11/d6d/fa0 5687709 1 2024-03-18T14:51:12.664 INFO:tasks.workunit.client.1.smithi100.stdout:1/734: write d8/d1e/d11/d27/d83/fa1 [620101,129553] 0 2024-03-18T14:51:12.669 INFO:tasks.workunit.client.1.smithi100.stdout:6/680: rename d6/d7/df/d26/d4e/d66/d36/dbb to d6/da1/dd7 0 2024-03-18T14:51:12.669 INFO:tasks.workunit.client.0.smithi012.stdout:3/964: dread dc/d3c/d4c/d57/d103/fa8 [0,4194304] 0 2024-03-18T14:51:12.669 INFO:tasks.workunit.client.0.smithi012.stdout:3/965: read dc/d3c/f4d [274754,34008] 0 2024-03-18T14:51:12.671 INFO:tasks.workunit.client.1.smithi100.stdout:6/681: creat d6/d7/df/d72/fd8 x:0 0 0 2024-03-18T14:51:12.674 INFO:tasks.workunit.client.1.smithi100.stdout:7/746: dwrite d5/d14/d16/d2e/d62/dd1/d86/f75 [0,4194304] 0 2024-03-18T14:51:12.676 INFO:tasks.workunit.client.1.smithi100.stdout:4/758: dwrite d9/d3a/fb3 [0,4194304] 0 2024-03-18T14:51:12.676 INFO:tasks.workunit.client.1.smithi100.stdout:4/759: truncate d9/d27/d62/d7b/fd4 218355 0 2024-03-18T14:51:12.681 INFO:tasks.workunit.client.1.smithi100.stdout:7/747: mknod d5/d30/d4d/db4/c11c 0 2024-03-18T14:51:12.682 INFO:tasks.workunit.client.1.smithi100.stdout:6/682: dread d6/d7/df/d72/f33 [0,4194304] 0 2024-03-18T14:51:12.684 INFO:tasks.workunit.client.1.smithi100.stdout:4/760: rename d9/d13/d39/d76/da3/fb1 to d9/d3a/fe4 0 2024-03-18T14:51:12.684 INFO:tasks.workunit.client.1.smithi100.stdout:4/761: truncate d9/f11 4374485 0 2024-03-18T14:51:12.685 INFO:tasks.workunit.client.1.smithi100.stdout:4/762: rename d9/d13/c20 to d9/d13/d5a/da4/ce5 0 2024-03-18T14:51:12.685 INFO:tasks.workunit.client.1.smithi100.stdout:4/763: write d9/d13/d5a/da4/f96 [60885,22271] 0 2024-03-18T14:51:12.686 INFO:tasks.workunit.client.1.smithi100.stdout:4/764: unlink d9/d17/l49 0 2024-03-18T14:51:12.688 INFO:tasks.workunit.client.1.smithi100.stdout:4/765: link d9/d13/d5a/fbc d9/d13/d25/fe6 0 2024-03-18T14:51:12.697 INFO:tasks.workunit.client.1.smithi100.stdout:7/748: dread d5/d14/d16/d2e/d62/f97 [0,4194304] 0 2024-03-18T14:51:12.697 INFO:tasks.workunit.client.1.smithi100.stdout:7/749: write d5/d30/d4d/d5a/f6a [4392486,31129] 0 2024-03-18T14:51:12.697 INFO:tasks.workunit.client.1.smithi100.stdout:7/750: write d5/d90/fd0 [3636342,51995] 0 2024-03-18T14:51:12.697 INFO:tasks.workunit.client.1.smithi100.stdout:7/751: fdatasync d5/f52 0 2024-03-18T14:51:12.697 INFO:tasks.workunit.client.1.smithi100.stdout:7/752: chown d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6/f73 235 1 2024-03-18T14:51:12.699 INFO:tasks.workunit.client.1.smithi100.stdout:7/753: mknod d5/d22/db3/c11d 0 2024-03-18T14:51:12.699 INFO:tasks.workunit.client.1.smithi100.stdout:7/754: write d5/d22/d24/d6e/d7a/fe3 [919626,93589] 0 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/755: chown d5/d14 9 1 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/756: truncate d5/d22/d3b/d9f/da0/fc6 234267 0 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/757: dread - d5/d14/d16/d2e/d62/d66/d4f/dcb/f10a zero size 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/758: chown d5/d14/d16/d44/l70 58528266 1 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/759: truncate d5/d22/d24/f79 1826142 0 2024-03-18T14:51:12.700 INFO:tasks.workunit.client.1.smithi100.stdout:7/760: dread - d5/d22/d3b/d9f/da0/db1/dd3/fe0 zero size 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:1/735: dwrite d8/d1e/d11/d6d/d59/fd7 [0,4194304] 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:1/736: read d8/f7e [692614,48353] 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:3/738: dwrite d3/fc1 [0,4194304] 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:5/700: dwrite d29/d2d/d39/d83/f85 [0,4194304] 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:3/739: creat d3/d16/d13/d17/d7d/f101 x:0 0 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:3/740: chown d3/d16/d13/d17/d7d/fad 86 1 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:3/741: write d3/d16/f14 [4410004,113808] 0 2024-03-18T14:51:12.702 INFO:tasks.workunit.client.1.smithi100.stdout:8/784: dwrite d4/dc/dd/de/d21/d23/d47/d54/fb9 [0,4194304] 0 2024-03-18T14:51:12.703 INFO:tasks.workunit.client.1.smithi100.stdout:4/766: write d9/d13/d5a/f78 [207038,49696] 0 2024-03-18T14:51:12.703 INFO:tasks.workunit.client.0.smithi012.stdout:3/966: dwrite dc/d3c/f6c [4194304,4194304] 0 2024-03-18T14:51:12.708 INFO:tasks.workunit.client.0.smithi012.stdout:3/967: mknod dc/d3c/d4c/d57/d103/d9a/d64/df4/d127/c134 0 2024-03-18T14:51:12.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/968: creat dc/d30/d3a/da0/dc5/dc7/dd8/f135 x:0 0 0 2024-03-18T14:51:12.712 INFO:tasks.workunit.client.0.smithi012.stdout:3/969: creat dc/d3c/d4c/d57/d103/d9a/dd7/de3/f136 x:0 0 0 2024-03-18T14:51:12.714 INFO:tasks.workunit.client.1.smithi100.stdout:4/767: dread d9/d13/d25/fe6 [0,4194304] 0 2024-03-18T14:51:12.718 INFO:tasks.workunit.client.1.smithi100.stdout:1/737: write d8/d1e/d11/fc3 [1043298,62783] 0 2024-03-18T14:51:12.718 INFO:tasks.workunit.client.1.smithi100.stdout:1/738: truncate d8/d2a/d37/fe0 674532 0 2024-03-18T14:51:12.723 INFO:tasks.workunit.client.0.smithi012.stdout:3/970: truncate dc/f43 3995672 0 2024-03-18T14:51:12.726 INFO:tasks.workunit.client.1.smithi100.stdout:3/742: read d3/dd/f38 [79200,126899] 0 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/971: dread dc/d30/f6d [0,4194304] 0 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/972: fdatasync dc/d30/d3a/db7/dd3/f133 0 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/973: chown dc/d3c/d4c/faa 13 1 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/974: chown dc/d30/dab/fd0 22618 1 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/975: chown dc/d91/fbb 481131006 1 2024-03-18T14:51:12.728 INFO:tasks.workunit.client.0.smithi012.stdout:3/976: chown dc/d3c/d72/df3 415 1 2024-03-18T14:51:12.730 INFO:tasks.workunit.client.0.smithi012.stdout:3/977: mkdir dc/d30/d3a/da0/dc5/dc7/dd8/d137 0 2024-03-18T14:51:12.730 INFO:tasks.workunit.client.0.smithi012.stdout:3/978: chown dc/d3c/d4c/d57/d103/d9a/dd7/de3/c11c 161 1 2024-03-18T14:51:12.730 INFO:tasks.workunit.client.0.smithi012.stdout:3/979: creat dc/d30/d3a/da0/f138 x:0 0 0 2024-03-18T14:51:12.731 INFO:tasks.workunit.client.1.smithi100.stdout:5/701: dwrite d29/d2d/d39/d71/fd3 [4194304,4194304] 0 2024-03-18T14:51:12.732 INFO:tasks.workunit.client.1.smithi100.stdout:7/761: dwrite d5/d14/d16/d2e/d62/dd1/d86/d85/d87/df6/f73 [0,4194304] 0 2024-03-18T14:51:12.732 INFO:tasks.workunit.client.1.smithi100.stdout:7/762: readlink d5/d14/d16/d2e/l3d 0 2024-03-18T14:51:12.735 INFO:tasks.workunit.client.1.smithi100.stdout:8/785: dwrite d4/dc/dd/de/f53 [0,4194304] 0 2024-03-18T14:51:12.736 INFO:tasks.workunit.client.1.smithi100.stdout:4/768: creat d9/d13/d5a/d5b/da0/dd8/fe7 x:0 0 0 2024-03-18T14:51:12.737 INFO:tasks.workunit.client.1.smithi100.stdout:1/739: creat d8/d1e/d11/d27/d3e/fe3 x:0 0 0 2024-03-18T14:51:12.738 INFO:tasks.workunit.client.1.smithi100.stdout:3/743: dread d3/d16/d13/d1f/d3f/d4f/de0/ffd [0,4194304] 0 2024-03-18T14:51:12.741 INFO:tasks.workunit.client.1.smithi100.stdout:7/763: unlink d5/d14/d16/l68 0 2024-03-18T14:51:12.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:12.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: pgmap v10: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 212 MiB/s rd, 241 MiB/s wr, 376 op/s 2024-03-18T14:51:12.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:12.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:12.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/712650625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:51:12.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:12 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:12.744 INFO:tasks.workunit.client.1.smithi100.stdout:4/769: creat d9/d13/fe8 x:0 0 0 2024-03-18T14:51:12.745 INFO:tasks.workunit.client.1.smithi100.stdout:1/740: getdents d8/d1e/d11/d27/d3f/daa/dd5 0 2024-03-18T14:51:12.747 INFO:tasks.workunit.client.1.smithi100.stdout:3/744: rename d3/d16/d13/d1f/d37/c98 to d3/d16/d13/d1f/d3f/d4f/d65/de9/c102 0 2024-03-18T14:51:12.748 INFO:tasks.workunit.client.1.smithi100.stdout:7/764: symlink d5/d22/d24/dac/l11e 0 2024-03-18T14:51:12.749 INFO:tasks.workunit.client.1.smithi100.stdout:4/770: creat d9/d13/d5a/da4/fe9 x:0 0 0 2024-03-18T14:51:12.749 INFO:tasks.workunit.client.1.smithi100.stdout:4/771: chown d9/d13/d39/d76/fcc 8 1 2024-03-18T14:51:12.750 INFO:tasks.workunit.client.1.smithi100.stdout:1/741: creat d8/d1e/d11/d8a/fe4 x:0 0 0 2024-03-18T14:51:12.751 INFO:tasks.workunit.client.1.smithi100.stdout:3/745: mkdir d3/dd/d21/dcd/d103 0 2024-03-18T14:51:12.752 INFO:tasks.workunit.client.1.smithi100.stdout:3/746: write d3/dd/f28 [2667818,83101] 0 2024-03-18T14:51:12.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/765: getdents d5/d14/d16/d2e/d62 0 2024-03-18T14:51:12.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/766: dread - d5/d22/d3b/d9f/da0/db1/dd3/f118 zero size 2024-03-18T14:51:12.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/767: dread - d5/d30/f5d zero size 2024-03-18T14:51:12.753 INFO:tasks.workunit.client.1.smithi100.stdout:4/772: mkdir d9/d13/d5a/d5b/da0/dcd/dea 0 2024-03-18T14:51:12.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/742: unlink d8/d2a/d37/d50/d6f/d71/db7/fcd 0 2024-03-18T14:51:12.758 INFO:tasks.workunit.client.1.smithi100.stdout:1/743: chown f7 311174 1 2024-03-18T14:51:12.758 INFO:tasks.workunit.client.1.smithi100.stdout:3/747: chown d3/dd/d21/c85 320 1 2024-03-18T14:51:12.758 INFO:tasks.workunit.client.1.smithi100.stdout:3/748: read d3/dd/d49/f4a [1297004,35042] 0 2024-03-18T14:51:12.758 INFO:tasks.workunit.client.1.smithi100.stdout:7/768: getdents d5/d14/d16/d2e/d62/dc3 0 2024-03-18T14:51:12.759 INFO:tasks.workunit.client.1.smithi100.stdout:7/769: chown d5/d90/da1/cf9 1341972 1 2024-03-18T14:51:12.759 INFO:tasks.workunit.client.1.smithi100.stdout:4/773: truncate d9/d27/d42/f4a 1917830 0 2024-03-18T14:51:12.760 INFO:tasks.workunit.client.0.smithi012.stdout:3/980: dwrite dc/d3c/d4c/f132 [0,4194304] 0 2024-03-18T14:51:12.760 INFO:tasks.workunit.client.0.smithi012.stdout:3/981: chown dc/d3c/d4c/d57/d103/d9a/l87 116 1 2024-03-18T14:51:12.761 INFO:tasks.workunit.client.1.smithi100.stdout:4/774: truncate d9/d13/d5a/d5b/f9e 1536734 0 2024-03-18T14:51:12.761 INFO:tasks.workunit.client.1.smithi100.stdout:4/775: write d9/d3a/f69 [749514,54920] 0 2024-03-18T14:51:12.761 INFO:tasks.workunit.client.1.smithi100.stdout:4/776: dread - d9/d17/fdd zero size 2024-03-18T14:51:12.761 INFO:tasks.workunit.client.1.smithi100.stdout:5/702: dwrite d29/f34 [0,4194304] 0 2024-03-18T14:51:12.761 INFO:tasks.workunit.client.1.smithi100.stdout:8/786: dwrite d4/dc/dd/de/d19/d37/fd7 [0,4194304] 0 2024-03-18T14:51:12.762 INFO:tasks.workunit.client.1.smithi100.stdout:1/744: link d8/l5a d8/d1e/d11/d27/d3f/daa/dd1/le5 0 2024-03-18T14:51:12.762 INFO:tasks.workunit.client.1.smithi100.stdout:1/745: fsync d8/d1e/d11/d27/fa7 0 2024-03-18T14:51:12.765 INFO:tasks.workunit.client.1.smithi100.stdout:4/777: write d9/d13/d25/fa5 [299072,49764] 0 2024-03-18T14:51:12.771 INFO:tasks.workunit.client.0.smithi012.stdout:3/982: unlink dc/d3c/d4c/f10f 0 2024-03-18T14:51:12.771 INFO:tasks.workunit.client.0.smithi012.stdout:3/983: write dc/d3c/d4c/d57/d103/d9a/dd7/de3/f136 [1037092,10817] 0 2024-03-18T14:51:12.772 INFO:tasks.workunit.client.1.smithi100.stdout:4/778: write d9/d27/d62/f9a [819729,35202] 0 2024-03-18T14:51:12.772 INFO:tasks.workunit.client.1.smithi100.stdout:4/779: stat d9/d13/d5a/fbc 0 2024-03-18T14:51:12.773 INFO:tasks.workunit.client.1.smithi100.stdout:5/703: creat d29/d2d/d39/d83/dc1/fe6 x:0 0 0 2024-03-18T14:51:12.775 INFO:tasks.workunit.client.1.smithi100.stdout:0/780: sync 2024-03-18T14:51:12.775 INFO:tasks.workunit.client.1.smithi100.stdout:2/741: sync 2024-03-18T14:51:12.775 INFO:tasks.workunit.client.1.smithi100.stdout:9/785: sync 2024-03-18T14:51:12.775 INFO:tasks.workunit.client.1.smithi100.stdout:2/742: fdatasync d3/d6/d29/d36/d49/f51 0 2024-03-18T14:51:12.776 INFO:tasks.workunit.client.0.smithi012.stdout:3/984: unlink dc/f12 0 2024-03-18T14:51:12.776 INFO:tasks.workunit.client.0.smithi012.stdout:3/985: fdatasync dc/d3c/d59/f101 0 2024-03-18T14:51:12.776 INFO:tasks.workunit.client.0.smithi012.stdout:3/986: fdatasync dc/d30/fac 0 2024-03-18T14:51:12.779 INFO:tasks.workunit.client.1.smithi100.stdout:3/749: dwrite d3/d16/d13/d17/d7d/f82 [0,4194304] 0 2024-03-18T14:51:12.779 INFO:tasks.workunit.client.1.smithi100.stdout:3/750: truncate d3/f57 793319 0 2024-03-18T14:51:12.779 INFO:tasks.workunit.client.1.smithi100.stdout:3/751: readlink d3/dd/d21/lff 0 2024-03-18T14:51:12.780 INFO:tasks.workunit.client.1.smithi100.stdout:8/787: mkdir d4/dc/dd/de/d21/d23/d119 0 2024-03-18T14:51:12.781 INFO:tasks.workunit.client.0.smithi012.stdout:3/987: mknod dc/d3c/d4c/d57/d103/d9a/d64/df4/c139 0 2024-03-18T14:51:12.783 INFO:tasks.workunit.client.1.smithi100.stdout:5/704: dread d29/f34 [0,4194304] 0 2024-03-18T14:51:12.783 INFO:tasks.workunit.client.1.smithi100.stdout:5/705: creat d29/d2d/dbe/dc2/fe7 x:0 0 0 2024-03-18T14:51:12.788 INFO:tasks.workunit.client.1.smithi100.stdout:2/743: symlink d3/da/de/d20/l102 0 2024-03-18T14:51:12.788 INFO:tasks.workunit.client.1.smithi100.stdout:2/744: write d3/d6/d5f/fe5 [2566941,74659] 0 2024-03-18T14:51:12.790 INFO:tasks.workunit.client.1.smithi100.stdout:7/770: dwrite d5/d14/d16/d2e/d62/f8c [0,4194304] 0 2024-03-18T14:51:12.795 INFO:tasks.workunit.client.1.smithi100.stdout:4/780: symlink d9/d13/d25/leb 0 2024-03-18T14:51:12.795 INFO:tasks.workunit.client.1.smithi100.stdout:4/781: chown d9/d3a/f64 471981 1 2024-03-18T14:51:12.795 INFO:tasks.workunit.client.1.smithi100.stdout:4/782: chown d9/d13/d5a 970618619 1 2024-03-18T14:51:12.796 INFO:tasks.workunit.client.1.smithi100.stdout:7/771: dread d5/d14/d16/d2e/d62/f97 [0,4194304] 0 2024-03-18T14:51:12.799 INFO:tasks.workunit.client.1.smithi100.stdout:8/788: rmdir d4/dc/d31/d66 39 2024-03-18T14:51:12.800 INFO:tasks.workunit.client.1.smithi100.stdout:2/745: dread d3/da/de/f3c [0,4194304] 0 2024-03-18T14:51:12.800 INFO:tasks.workunit.client.1.smithi100.stdout:2/746: getdents d3/d8c 0 2024-03-18T14:51:12.801 INFO:tasks.workunit.client.1.smithi100.stdout:2/747: getdents d3/da/de/d20/d82/dd3 0 2024-03-18T14:51:12.801 INFO:tasks.workunit.client.1.smithi100.stdout:1/746: rename d8/d1e/d11/d27/d3e to d8/d30/d61/d8d/dde/de6 0 2024-03-18T14:51:12.802 INFO:tasks.workunit.client.1.smithi100.stdout:4/783: symlink d9/d27/lec 0 2024-03-18T14:51:12.804 INFO:tasks.workunit.client.1.smithi100.stdout:8/789: unlink d4/dc/d31/lfb 0 2024-03-18T14:51:12.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/747: creat d8/d30/d61/d8d/dde/de6/fe7 x:0 0 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/748: dread d3/d1a/d76/f9f [0,4194304] 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/749: readlink d3/da/de/d20/le9 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/750: fsync d3/d1a/f2d 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/751: write d3/da/de/d20/dea/ff1 [954930,78228] 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/752: chown d3/da/de/d20/dea/cbc 12141 1 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/753: chown d3/da/l11 3767 1 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/754: truncate d3/da/de/d20/ffd 495205 0 2024-03-18T14:51:12.807 INFO:tasks.workunit.client.1.smithi100.stdout:2/755: dread - d3/da/de/d20/d82/dad/de2/ffa zero size 2024-03-18T14:51:12.808 INFO:tasks.workunit.client.1.smithi100.stdout:4/784: rmdir d9/d13/de3 0 2024-03-18T14:51:12.808 INFO:tasks.workunit.client.1.smithi100.stdout:4/785: creat d9/d13/d5a/d5b/da0/dd8/fed x:0 0 0 2024-03-18T14:51:12.810 INFO:tasks.workunit.client.1.smithi100.stdout:7/772: mknod d5/d22/d3b/d9b/c11f 0 2024-03-18T14:51:12.811 INFO:tasks.workunit.client.1.smithi100.stdout:4/786: dread d9/d13/d39/d4c/f5e [0,4194304] 0 2024-03-18T14:51:12.811 INFO:tasks.workunit.client.1.smithi100.stdout:4/787: creat d9/d13/d5a/da4/fee x:0 0 0 2024-03-18T14:51:12.812 INFO:tasks.workunit.client.1.smithi100.stdout:8/790: rename d4/dc/d97/cdf to d4/dc/d31/d85/c11a 0 2024-03-18T14:51:12.812 INFO:tasks.workunit.client.1.smithi100.stdout:8/791: creat d4/dc/d1f/d59/d6c/f11b x:0 0 0 2024-03-18T14:51:12.813 INFO:tasks.workunit.client.1.smithi100.stdout:1/748: creat d8/d1e/d11/d27/d3f/fe8 x:0 0 0 2024-03-18T14:51:12.818 INFO:tasks.workunit.client.1.smithi100.stdout:7/773: link d5/d14/d16/d2e/d62/d66/d4f/fd8 d5/d22/d24/f120 0 2024-03-18T14:51:12.821 INFO:tasks.workunit.client.1.smithi100.stdout:2/756: rename d3/da/de/d48/c81 to d3/da/de/c103 0 2024-03-18T14:51:12.822 INFO:tasks.workunit.client.1.smithi100.stdout:8/792: dread d4/dc/d31/d85/fb2 [0,4194304] 0 2024-03-18T14:51:12.823 INFO:tasks.workunit.client.1.smithi100.stdout:1/749: unlink d8/d2a/l4f 0 2024-03-18T14:51:12.824 INFO:tasks.workunit.client.1.smithi100.stdout:7/774: getdents d5/d90 0 2024-03-18T14:51:12.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/793: rmdir d4/dc/dd/d48/dd8 39 2024-03-18T14:51:12.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/794: fsync d4/dc/dd/de/d19/f2f 0 2024-03-18T14:51:12.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/795: write d4/dc/dd/de/d19/db0/dd5/f104 [393715,67850] 0 2024-03-18T14:51:12.826 INFO:tasks.workunit.client.1.smithi100.stdout:8/796: chown d4/dc/dd/de/d21/d23/d25/c81 536553791 1 2024-03-18T14:51:12.829 INFO:tasks.workunit.client.1.smithi100.stdout:1/750: rename d8/d30/d61/d8d/dde/de6/lbb to d8/d1e/d11/d6d/d59/le9 0 2024-03-18T14:51:12.829 INFO:tasks.workunit.client.1.smithi100.stdout:3/752: dwrite d3/dd/d21/d3e/fba [0,4194304] 0 2024-03-18T14:51:12.830 INFO:tasks.workunit.client.1.smithi100.stdout:3/753: read d3/d16/d13/d1f/f59 [3943595,2513] 0 2024-03-18T14:51:12.830 INFO:tasks.workunit.client.1.smithi100.stdout:3/754: write d3/d16/d13/d17/d7d/fad [1011818,34410] 0 2024-03-18T14:51:12.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/781: dwrite dd/d20/fa0 [0,4194304] 0 2024-03-18T14:51:12.830 INFO:tasks.workunit.client.1.smithi100.stdout:0/782: chown dd/d20/db5 28365088 1 2024-03-18T14:51:12.831 INFO:tasks.workunit.client.1.smithi100.stdout:8/797: symlink d4/dc/dd/de/d21/d23/d47/l11c 0 2024-03-18T14:51:12.831 INFO:tasks.workunit.client.0.smithi012.stdout:3/988: dwrite dc/d30/d35/f2e [4194304,4194304] 0 2024-03-18T14:51:12.832 INFO:tasks.workunit.client.1.smithi100.stdout:4/788: dwrite d9/d13/f33 [0,4194304] 0 2024-03-18T14:51:12.832 INFO:tasks.workunit.client.1.smithi100.stdout:8/798: dread d4/dc/dd/de/d21/f72 [0,4194304] 0 2024-03-18T14:51:12.832 INFO:tasks.workunit.client.1.smithi100.stdout:8/799: fsync d4/dc/d1f/d59/d6c/fdd 0 2024-03-18T14:51:12.832 INFO:tasks.workunit.client.1.smithi100.stdout:4/789: write d9/d13/d5a/d5b/da0/dcd/fd3 [648214,76081] 0 2024-03-18T14:51:12.834 INFO:tasks.workunit.client.1.smithi100.stdout:2/757: write d3/da/de/d20/dea/d40/d53/f65 [821154,35183] 0 2024-03-18T14:51:12.839 INFO:tasks.workunit.client.1.smithi100.stdout:1/751: write d8/d2a/f6b [6963114,2698] 0 2024-03-18T14:51:12.841 INFO:tasks.workunit.client.1.smithi100.stdout:0/783: dread dd/f55 [0,4194304] 0 2024-03-18T14:51:12.841 INFO:tasks.workunit.client.1.smithi100.stdout:0/784: fdatasync dd/d20/f92 0 2024-03-18T14:51:12.843 INFO:tasks.workunit.client.1.smithi100.stdout:7/775: rename d5/l2f to d5/d90/da1/l121 0 2024-03-18T14:51:12.844 INFO:tasks.workunit.client.1.smithi100.stdout:3/755: unlink d3/d16/d13/d17/f33 0 2024-03-18T14:51:12.845 INFO:tasks.workunit.client.0.smithi012.stdout:3/989: read dc/d3c/d4c/faa [2230639,12096] 0 2024-03-18T14:51:12.846 INFO:tasks.workunit.client.1.smithi100.stdout:7/776: write d5/d22/d24/d6e/fd9 [2122222,51924] 0 2024-03-18T14:51:12.847 INFO:tasks.workunit.client.1.smithi100.stdout:0/785: dread dd/d2b/f89 [0,4194304] 0 2024-03-18T14:51:12.847 INFO:tasks.workunit.client.1.smithi100.stdout:0/786: read - dd/d31/d91/fe1 zero size 2024-03-18T14:51:12.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/777: write d5/d22/d3b/d9f/da0/fa8 [1691721,74261] 0 2024-03-18T14:51:12.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/778: chown d5/d14/d16/d2e/l31 1131744236 1 2024-03-18T14:51:12.850 INFO:tasks.workunit.client.1.smithi100.stdout:8/800: mknod d4/dc/d1f/d59/d6c/d75/dea/c11d 0 2024-03-18T14:51:12.853 INFO:tasks.workunit.client.1.smithi100.stdout:4/790: link d9/d27/d62/f6c d9/d27/d62/d7b/d82/fef 0 2024-03-18T14:51:12.857 INFO:tasks.workunit.client.1.smithi100.stdout:2/758: link d3/d6/d29/d36/d49/f54 d3/da/de/d20/dea/d40/f104 0 2024-03-18T14:51:12.857 INFO:tasks.workunit.client.1.smithi100.stdout:2/759: write d3/d6/d5f/d6a/fcf [294114,104975] 0 2024-03-18T14:51:12.867 INFO:tasks.workunit.client.1.smithi100.stdout:1/752: link d8/d1e/d11/d27/d3f/daa/dd5/ddc/c99 d8/d30/d61/da2/cea 0 2024-03-18T14:51:12.867 INFO:tasks.workunit.client.1.smithi100.stdout:1/753: chown d8/d30/d61/d8d/cbf 452171 1 2024-03-18T14:51:12.867 INFO:tasks.workunit.client.1.smithi100.stdout:3/756: mknod d3/d16/d13/d1f/d3f/d4f/d60/da9/c104 0 2024-03-18T14:51:12.867 INFO:tasks.workunit.client.1.smithi100.stdout:0/787: creat dd/d31/d91/d56/d59/f11d x:0 0 0 2024-03-18T14:51:12.867 INFO:tasks.workunit.client.1.smithi100.stdout:8/801: mknod d4/dc/dd/d48/c11e 0 2024-03-18T14:51:12.868 INFO:tasks.workunit.client.1.smithi100.stdout:7/779: dwrite d5/fd [0,4194304] 0 2024-03-18T14:51:12.868 INFO:tasks.workunit.client.1.smithi100.stdout:4/791: mkdir d9/d13/d5a/d5b/da0/dd8/df0 0 2024-03-18T14:51:12.868 INFO:tasks.workunit.client.0.smithi012.stdout:3/990: dwrite dc/f2b [4194304,4194304] 0 2024-03-18T14:51:12.868 INFO:tasks.workunit.client.0.smithi012.stdout:3/991: dread - dc/d3c/d72/df3/f114 zero size 2024-03-18T14:51:12.869 INFO:tasks.workunit.client.1.smithi100.stdout:3/757: write d3/d16/d13/d1f/d3f/d4f/de0/ffd [2303900,49206] 0 2024-03-18T14:51:12.869 INFO:tasks.workunit.client.1.smithi100.stdout:1/754: rename d8/d1e/d11/d6d/d59/le9 to d8/d1e/d11/d6d/leb 0 2024-03-18T14:51:12.869 INFO:tasks.workunit.client.1.smithi100.stdout:8/802: rmdir d4/dc/dd/de/d19/d37 39 2024-03-18T14:51:12.869 INFO:tasks.workunit.client.1.smithi100.stdout:8/803: readlink d4/dc/dd/de/d21/d23/d25/lee 0 2024-03-18T14:51:12.870 INFO:tasks.workunit.client.0.smithi012.stdout:3/992: readlink dc/d30/d3a/l42 0 2024-03-18T14:51:12.874 INFO:tasks.workunit.client.1.smithi100.stdout:2/760: dwrite d3/da/de/d20/dea/d4f/fb5 [0,4194304] 0 2024-03-18T14:51:12.875 INFO:tasks.workunit.client.1.smithi100.stdout:2/761: creat d3/d6/d5f/d63/f105 x:0 0 0 2024-03-18T14:51:12.878 INFO:tasks.workunit.client.1.smithi100.stdout:2/762: dread d3/d6/d5f/d6a/f99 [0,4194304] 0 2024-03-18T14:51:12.878 INFO:tasks.workunit.client.1.smithi100.stdout:2/763: fdatasync d3/f10 0 2024-03-18T14:51:12.878 INFO:tasks.workunit.client.1.smithi100.stdout:2/764: fdatasync d3/da/de/d20/dea/fa6 0 2024-03-18T14:51:12.878 INFO:tasks.workunit.client.1.smithi100.stdout:2/765: chown d3/da 900389 1 2024-03-18T14:51:12.879 INFO:tasks.workunit.client.1.smithi100.stdout:7/780: mkdir d5/d14/d16/d2e/d62/dd1/d86/de6/d122 0 2024-03-18T14:51:12.880 INFO:tasks.workunit.client.1.smithi100.stdout:4/792: creat d9/d13/d39/d79/ff1 x:0 0 0 2024-03-18T14:51:12.883 INFO:tasks.workunit.client.1.smithi100.stdout:3/758: symlink d3/dd/d21/d3e/df8/db9/df0/l105 0 2024-03-18T14:51:12.883 INFO:tasks.workunit.client.1.smithi100.stdout:3/759: creat d3/da5/f106 x:0 0 0 2024-03-18T14:51:12.883 INFO:tasks.workunit.client.1.smithi100.stdout:3/760: read d3/d16/d13/d1f/d3f/d4f/de0/ffd [903643,84876] 0 2024-03-18T14:51:12.883 INFO:tasks.workunit.client.1.smithi100.stdout:3/761: dread - d3/dd/d49/f72 zero size 2024-03-18T14:51:12.884 INFO:tasks.workunit.client.0.smithi012.stdout:3/993: write dc/d30/f99 [3489140,24946] 0 2024-03-18T14:51:12.885 INFO:tasks.workunit.client.1.smithi100.stdout:8/804: mknod d4/dc/dd/d48/dd8/df0/c11f 0 2024-03-18T14:51:12.885 INFO:tasks.workunit.client.1.smithi100.stdout:4/793: dread d9/d13/d5a/da4/f96 [0,4194304] 0 2024-03-18T14:51:12.885 INFO:tasks.workunit.client.1.smithi100.stdout:4/794: fdatasync d9/d13/d5a/da4/fe0 0 2024-03-18T14:51:12.885 INFO:tasks.workunit.client.1.smithi100.stdout:4/795: readlink d9/d13/l44 0 2024-03-18T14:51:12.885 INFO:tasks.workunit.client.1.smithi100.stdout:8/805: write d4/dc/d31/f69 [1345477,102830] 0 2024-03-18T14:51:12.890 INFO:tasks.workunit.client.1.smithi100.stdout:7/781: symlink d5/d90/l123 0 2024-03-18T14:51:12.890 INFO:tasks.workunit.client.1.smithi100.stdout:7/782: readlink d5/l8 0 2024-03-18T14:51:12.890 INFO:tasks.workunit.client.1.smithi100.stdout:7/783: read d5/d14/d16/d2e/d62/dd1/d86/f7f [412494,108613] 0 2024-03-18T14:51:12.893 INFO:tasks.workunit.client.1.smithi100.stdout:4/796: truncate d9/d17/f23 546650 0 2024-03-18T14:51:12.893 INFO:tasks.workunit.client.1.smithi100.stdout:0/788: dwrite dd/d20/f21 [0,4194304] 0 2024-03-18T14:51:12.893 INFO:tasks.workunit.client.1.smithi100.stdout:0/789: fdatasync dd/d20/f27 0 2024-03-18T14:51:12.893 INFO:tasks.workunit.client.1.smithi100.stdout:0/790: write dd/d31/d91/d56/d62/fed [961891,39278] 0 2024-03-18T14:51:12.894 INFO:tasks.workunit.client.1.smithi100.stdout:1/755: dwrite d8/d30/d61/d8d/dde/de6/f94 [0,4194304] 0 2024-03-18T14:51:12.899 INFO:tasks.workunit.client.1.smithi100.stdout:2/766: dread d3/da/de/d20/dea/f38 [0,4194304] 0 2024-03-18T14:51:12.899 INFO:tasks.workunit.client.1.smithi100.stdout:6/683: sync 2024-03-18T14:51:12.901 INFO:tasks.workunit.client.1.smithi100.stdout:3/762: dread d3/d16/d13/d17/d7d/f82 [0,4194304] 0 2024-03-18T14:51:12.905 INFO:tasks.workunit.client.1.smithi100.stdout:1/756: symlink d8/d30/d61/d8d/lec 0 2024-03-18T14:51:12.905 INFO:tasks.workunit.client.1.smithi100.stdout:7/784: truncate d5/d22/d24/d39/f45 2912505 0 2024-03-18T14:51:12.905 INFO:tasks.workunit.client.1.smithi100.stdout:0/791: write dd/d31/d91/d56/d59/f7c [4120484,93320] 0 2024-03-18T14:51:12.905 INFO:tasks.workunit.client.1.smithi100.stdout:0/792: fsync dd/d31/d91/d56/d71/dcc/dcf/fdf 0 2024-03-18T14:51:12.906 INFO:tasks.workunit.client.1.smithi100.stdout:4/797: dread d9/f11 [0,4194304] 0 2024-03-18T14:51:12.906 INFO:tasks.workunit.client.1.smithi100.stdout:2/767: rename d3/ld5 to d3/da/de/d20/d82/dad/de2/l106 0 2024-03-18T14:51:12.909 INFO:tasks.workunit.client.1.smithi100.stdout:1/757: link d8/d30/c32 d8/d1e/d11/d8a/ced 0 2024-03-18T14:51:12.910 INFO:tasks.workunit.client.0.smithi012.stdout:3/994: dwrite dc/d3c/d59/fc8 [0,4194304] 0 2024-03-18T14:51:12.910 INFO:tasks.workunit.client.0.smithi012.stdout:3/995: read - dc/d3c/d4c/d57/d103/d9a/f9b zero size 2024-03-18T14:51:12.911 INFO:tasks.workunit.client.1.smithi100.stdout:0/793: rmdir dd/d2b/d95/d79/d114 0 2024-03-18T14:51:12.911 INFO:tasks.workunit.client.1.smithi100.stdout:6/684: dread d6/d7/df/d26/d4e/d66/f68 [0,4194304] 0 2024-03-18T14:51:12.912 INFO:tasks.workunit.client.1.smithi100.stdout:8/806: dwrite d4/dc/dd/de/f57 [0,4194304] 0 2024-03-18T14:51:12.912 INFO:tasks.workunit.client.1.smithi100.stdout:4/798: unlink d9/d3a/f9c 0 2024-03-18T14:51:12.913 INFO:tasks.workunit.client.1.smithi100.stdout:4/799: stat d9/d17/f23 0 2024-03-18T14:51:12.913 INFO:tasks.workunit.client.0.smithi012.stdout:3/996: rmdir dc/d3c/d4c/d57/d103/d9a/dd7/d124 39 2024-03-18T14:51:12.915 INFO:tasks.workunit.client.1.smithi100.stdout:2/768: mkdir d3/da/de/d20/d107 0 2024-03-18T14:51:12.915 INFO:tasks.workunit.client.1.smithi100.stdout:2/769: truncate d3/d6/ff7 788155 0 2024-03-18T14:51:12.917 INFO:tasks.workunit.client.1.smithi100.stdout:3/763: mknod d3/c107 0 2024-03-18T14:51:12.917 INFO:tasks.workunit.client.1.smithi100.stdout:3/764: fdatasync d3/f10 0 2024-03-18T14:51:12.919 INFO:tasks.workunit.client.1.smithi100.stdout:1/758: creat d8/fee x:0 0 0 2024-03-18T14:51:12.919 INFO:tasks.workunit.client.1.smithi100.stdout:1/759: truncate d8/d2a/f41 2044055 0 2024-03-18T14:51:12.920 INFO:tasks.workunit.client.0.smithi012.stdout:3/997: symlink dc/d30/d35/l13a 0 2024-03-18T14:51:12.923 INFO:tasks.workunit.client.1.smithi100.stdout:6/685: getdents d6/d7/df 0 2024-03-18T14:51:12.923 INFO:tasks.workunit.client.1.smithi100.stdout:6/686: truncate d6/d7/f5f 4341361 0 2024-03-18T14:51:12.923 INFO:tasks.workunit.client.1.smithi100.stdout:6/687: creat d6/d7/df/d26/d4e/d66/fd9 x:0 0 0 2024-03-18T14:51:12.923 INFO:tasks.workunit.client.1.smithi100.stdout:0/794: dread dd/f16 [0,4194304] 0 2024-03-18T14:51:12.924 INFO:tasks.workunit.client.1.smithi100.stdout:8/807: creat d4/dc/d1f/d59/f120 x:0 0 0 2024-03-18T14:51:12.924 INFO:tasks.workunit.client.1.smithi100.stdout:2/770: dread d3/d6/d5f/d63/f72 [0,4194304] 0 2024-03-18T14:51:12.924 INFO:tasks.workunit.client.1.smithi100.stdout:2/771: chown d3/d6/d29/d36/d49/lfb 1898625491 1 2024-03-18T14:51:12.931 INFO:tasks.workunit.client.1.smithi100.stdout:6/688: read d6/d12/f43 [4167442,35393] 0 2024-03-18T14:51:12.931 INFO:tasks.workunit.client.1.smithi100.stdout:6/689: write d6/f8f [102535,124924] 0 2024-03-18T14:51:12.932 INFO:tasks.workunit.client.1.smithi100.stdout:8/808: symlink d4/dc/d31/d85/d8e/l121 0 2024-03-18T14:51:12.934 INFO:tasks.workunit.client.1.smithi100.stdout:2/772: symlink d3/d6/d5f/d6a/l108 0 2024-03-18T14:51:12.934 INFO:tasks.workunit.client.1.smithi100.stdout:1/760: dread f5 [0,4194304] 0 2024-03-18T14:51:12.939 INFO:tasks.workunit.client.1.smithi100.stdout:7/785: write d5/d22/d24/d39/f45 [1188256,4252] 0 2024-03-18T14:51:12.939 INFO:tasks.workunit.client.1.smithi100.stdout:7/786: chown d5/d14/d16/d2e/d62/dd1/d86/d85/d87/de5/deb/lf0 2126 1 2024-03-18T14:51:12.939 INFO:tasks.workunit.client.1.smithi100.stdout:7/787: chown d5/l8 0 1 2024-03-18T14:51:12.939 INFO:tasks.workunit.client.1.smithi100.stdout:2/773: mknod d3/d6/c109 0 2024-03-18T14:51:12.939 INFO:tasks.workunit.client.1.smithi100.stdout:2/774: chown d3/d6/d5f/d6a 214014 1 2024-03-18T14:51:12.940 INFO:tasks.workunit.client.1.smithi100.stdout:4/800: dwrite d9/d3a/f64 [0,4194304] 0 2024-03-18T14:51:12.941 INFO:tasks.workunit.client.1.smithi100.stdout:3/765: dwrite d3/d16/d13/d1f/f6c [0,4194304] 0 2024-03-18T14:51:12.941 INFO:tasks.workunit.client.1.smithi100.stdout:3/766: fsync d3/d16/d13/f47 0 2024-03-18T14:51:12.943 INFO:tasks.workunit.client.1.smithi100.stdout:0/795: rename dd/d2b/d95/d79/d7d/da9/db0 to dd/d2b/d11e 0 2024-03-18T14:51:12.943 INFO:tasks.workunit.client.1.smithi100.stdout:6/690: dread d6/d7/df/d26/f45 [0,4194304] 0 2024-03-18T14:51:12.947 INFO:tasks.workunit.client.1.smithi100.stdout:1/761: readlink d8/dd/l65 0 2024-03-18T14:51:12.947 INFO:tasks.workunit.client.1.smithi100.stdout:1/762: fdatasync d8/d1e/d11/f1b 0 2024-03-18T14:51:12.948 INFO:tasks.workunit.client.1.smithi100.stdout:0/796: symlink dd/d31/d91/d7b/l11f 0 2024-03-18T14:51:12.951 INFO:tasks.workunit.client.1.smithi100.stdout:7/788: rename d5/d14/d16/d2e/d62 to d5/d14/d16/d63/d124 0 2024-03-18T14:51:12.952 INFO:tasks.workunit.client.1.smithi100.stdout:0/797: mkdir dd/d31/d91/d56/d71/dcc/dcf/d120 0 2024-03-18T14:51:12.953 INFO:tasks.workunit.client.1.smithi100.stdout:2/775: dread d3/d6/d5f/f6d [0,4194304] 0 2024-03-18T14:51:12.955 INFO:tasks.workunit.client.1.smithi100.stdout:7/789: rename d5/d22/d24/d6e/lb2 to d5/d14/d16/d63/d124/dc3/l125 0 2024-03-18T14:51:12.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/776: write d3/d6/d29/f35 [1984547,48125] 0 2024-03-18T14:51:12.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/777: stat d3/d1a/d6c/f8b 0 2024-03-18T14:51:12.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/778: dread - d3/da/de/d20/d82/fee zero size 2024-03-18T14:51:12.955 INFO:tasks.workunit.client.1.smithi100.stdout:2/779: dread - d3/d6/d5f/d63/f105 zero size 2024-03-18T14:51:12.956 INFO:tasks.workunit.client.1.smithi100.stdout:0/798: mkdir dd/d31/d91/d56/d121 0 2024-03-18T14:51:12.958 INFO:tasks.workunit.client.1.smithi100.stdout:4/801: read d9/d27/d42/f6a [299754,100807] 0 2024-03-18T14:51:12.958 INFO:tasks.workunit.client.1.smithi100.stdout:4/802: creat d9/d13/ff2 x:0 0 0 2024-03-18T14:51:12.958 INFO:tasks.workunit.client.1.smithi100.stdout:6/691: dread d6/d7/df/d26/d4e/f82 [0,4194304] 0 2024-03-18T14:51:12.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/780: rmdir d3/da/de 39 2024-03-18T14:51:12.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/781: write d3/da/de/d20/d82/d95/faa [545851,61642] 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:1/763: read d8/d1e/d11/d27/d73/d89/fa4 [2775297,105956] 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:0/799: mknod dd/d31/d91/d56/d71/dcc/dcf/c122 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/809: dwrite d4/dc/d31/f46 [4194304,4194304] 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:3/767: dwrite d3/fd5 [0,4194304] 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/810: stat d4/dc/d1f/d59/d6c/f11b 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:3/768: truncate d3/d16/d13/d17/d7d/fad 1495340 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/811: stat d4/dc/d97/caa 0 2024-03-18T14:51:12.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/812: fsync d4/dc/dd/de/d19/db0/f94 0 2024-03-18T14:51:12.964 INFO:tasks.workunit.client.0.smithi012.stdout:3/998: dwrite dc/d30/d3a/d112/f122 [0,4194304] 0 2024-03-18T14:51:12.964 INFO:tasks.workunit.client.0.smithi012.stdout:3/999: fdatasync dc/d30/fd6 0 2024-03-18T14:51:12.964 INFO:tasks.workunit.client.1.smithi100.stdout:2/782: dread d3/d6/d29/d36/d4a/fb9 [0,4194304] 0 2024-03-18T14:51:12.965 INFO:tasks.workunit.client.1.smithi100.stdout:6/692: rename d6/d7/df/d26/d4e/d66/caa to d6/d7/df/d72/dc9/cda 0 2024-03-18T14:51:12.967 INFO:tasks.workunit.client.1.smithi100.stdout:1/764: symlink d8/d1e/d11/d27/d3f/daa/dd5/lef 0 2024-03-18T14:51:12.970 INFO:tasks.workunit.client.1.smithi100.stdout:0/800: creat dd/d2b/d95/d79/d7d/da9/f123 x:0 0 0 2024-03-18T14:51:12.972 INFO:tasks.workunit.client.1.smithi100.stdout:2/783: write d3/d6/d5f/fc6 [4106855,7402] 0 2024-03-18T14:51:12.972 INFO:tasks.workunit.client.1.smithi100.stdout:2/784: stat d3/d6/d29/d36/d49/c61 0 2024-03-18T14:51:12.972 INFO:tasks.workunit.client.1.smithi100.stdout:2/785: write d3/d6/d5f/fe5 [3601873,52887] 0 2024-03-18T14:51:12.976 INFO:tasks.workunit.client.1.smithi100.stdout:8/813: creat d4/dc/d31/f122 x:0 0 0 2024-03-18T14:51:12.982 INFO:tasks.workunit.client.1.smithi100.stdout:6/693: creat d6/d12/fdb x:0 0 0 2024-03-18T14:51:12.982 INFO:tasks.workunit.client.1.smithi100.stdout:6/694: stat d6/d7/df/d72/d65/d7a/da5 0 2024-03-18T14:51:12.985 INFO:tasks.workunit.client.1.smithi100.stdout:1/765: symlink d8/d1e/d11/d27/d73/d89/lf0 0 2024-03-18T14:51:12.985 INFO:tasks.workunit.client.1.smithi100.stdout:1/766: write d8/f52 [570468,111565] 0 2024-03-18T14:51:12.985 INFO:tasks.workunit.client.1.smithi100.stdout:1/767: write d8/d2a/d37/d50/d6f/d71/fad [1896968,11818] 0 2024-03-18T14:51:12.985 INFO:tasks.workunit.client.1.smithi100.stdout:1/768: fsync d8/d2a/d37/fe0 0 2024-03-18T14:51:12.985 INFO:tasks.workunit.client.1.smithi100.stdout:1/769: stat d8/d1e/f45 0 2024-03-18T14:51:12.987 INFO:tasks.workunit.client.1.smithi100.stdout:0/801: rename dd/d31/d91/f66 to dd/d31/d91/d56/d71/dcc/f124 0 2024-03-18T14:51:12.989 INFO:tasks.workunit.client.1.smithi100.stdout:5/706: sync 2024-03-18T14:51:12.989 INFO:tasks.workunit.client.1.smithi100.stdout:9/786: sync 2024-03-18T14:51:12.989 INFO:tasks.workunit.client.1.smithi100.stdout:5/707: truncate d29/d2d/d32/fd6 208660 0 2024-03-18T14:51:12.990 INFO:tasks.workunit.client.1.smithi100.stdout:3/769: rmdir d3/d16/d13/d17 39 2024-03-18T14:51:12.990 INFO:tasks.workunit.client.1.smithi100.stdout:3/770: chown d3/dd/l26 450 1 2024-03-18T14:51:12.990 INFO:tasks.workunit.client.1.smithi100.stdout:5/708: write d29/d2d/d39/d71/f77 [816388,13491] 0 2024-03-18T14:51:12.996 INFO:tasks.workunit.client.1.smithi100.stdout:1/770: symlink d8/d30/d61/da2/lf1 0 2024-03-18T14:51:12.998 INFO:tasks.workunit.client.1.smithi100.stdout:3/771: mknod d3/d16/c108 0 2024-03-18T14:51:13.000 INFO:tasks.workunit.client.1.smithi100.stdout:2/786: dwrite d3/d1a/d6c/fa4 [0,4194304] 0 2024-03-18T14:51:13.001 INFO:tasks.workunit.client.1.smithi100.stdout:8/814: rmdir d4/dc/dd/de/d19/db0/dd5 39 2024-03-18T14:51:13.004 INFO:tasks.workunit.client.1.smithi100.stdout:8/815: creat d4/dc/dd/d48/f123 x:0 0 0 2024-03-18T14:51:13.007 INFO:tasks.workunit.client.1.smithi100.stdout:8/816: mkdir d4/dc/dd/de/d21/d23/d47/d54/d124 0 2024-03-18T14:51:13.007 INFO:tasks.workunit.client.1.smithi100.stdout:8/817: readlink d4/d3c/l35 0 2024-03-18T14:51:13.007 INFO:tasks.workunit.client.1.smithi100.stdout:5/709: write d29/d2d/d39/d83/dc1/dbb/f5d [4091735,33194] 0 2024-03-18T14:51:13.008 INFO:tasks.workunit.client.1.smithi100.stdout:8/818: mkdir d4/dc/dd/de/d21/d23/d119/d125 0 2024-03-18T14:51:13.009 INFO:tasks.workunit.client.1.smithi100.stdout:3/772: write d3/f10 [7904174,81659] 0 2024-03-18T14:51:13.011 INFO:tasks.workunit.client.1.smithi100.stdout:8/819: truncate d4/d96/fed 796546 0 2024-03-18T14:51:13.011 INFO:tasks.workunit.client.1.smithi100.stdout:8/820: write d4/dc/d31/d66/f9b [5682562,641] 0 2024-03-18T14:51:13.012 INFO:tasks.workunit.client.1.smithi100.stdout:2/787: dread d3/d6/d29/f92 [0,4194304] 0 2024-03-18T14:51:13.012 INFO:tasks.workunit.client.1.smithi100.stdout:2/788: write d3/d6/d5f/d63/f72 [3987150,125084] 0 2024-03-18T14:51:13.014 INFO:tasks.workunit.client.1.smithi100.stdout:3/773: link d3/dd/d49/f70 d3/d16/d13/d1f/d3f/d4f/d65/f109 0 2024-03-18T14:51:13.017 INFO:tasks.workunit.client.1.smithi100.stdout:3/774: symlink d3/dd/l10a 0 2024-03-18T14:51:13.022 INFO:tasks.workunit.client.1.smithi100.stdout:3/775: dread d3/d16/d13/d17/dab/fdb [0,4194304] 0 2024-03-18T14:51:13.022 INFO:tasks.workunit.client.1.smithi100.stdout:3/776: write d3/d16/d13/f5f [470245,62429] 0 2024-03-18T14:51:13.023 INFO:tasks.workunit.client.1.smithi100.stdout:3/777: chown d3/d16/d13/d1f/d3f/d4f/d60/l8e 1818121889 1 2024-03-18T14:51:13.025 INFO:tasks.workunit.client.1.smithi100.stdout:8/821: write d4/dc/d1f/d59/d6c/d75/dea/feb [2674084,72574] 0 2024-03-18T14:51:13.025 INFO:tasks.workunit.client.1.smithi100.stdout:6/695: dwrite d6/d12/da2/faf [0,4194304] 0 2024-03-18T14:51:13.025 INFO:tasks.workunit.client.1.smithi100.stdout:6/696: chown d6/d12/da2/faf 21 1 2024-03-18T14:51:13.027 INFO:tasks.workunit.client.1.smithi100.stdout:0/802: dwrite fa [0,4194304] 0 2024-03-18T14:51:13.028 INFO:tasks.workunit.client.1.smithi100.stdout:0/803: creat dd/d20/d44/f125 x:0 0 0 2024-03-18T14:51:13.028 INFO:tasks.workunit.client.1.smithi100.stdout:3/778: dread d3/dd/d21/dcd/fd9 [0,4194304] 0 2024-03-18T14:51:13.028 INFO:tasks.workunit.client.1.smithi100.stdout:3/779: stat d3/d16/d13/d17/d7d/fee 0 2024-03-18T14:51:13.028 INFO:tasks.workunit.client.1.smithi100.stdout:3/780: read d3/d16/fc [1952939,80510] 0 2024-03-18T14:51:13.029 INFO:tasks.workunit.client.1.smithi100.stdout:1/771: dwrite d8/d1e/f86 [4194304,4194304] 0 2024-03-18T14:51:13.029 INFO:tasks.workunit.client.1.smithi100.stdout:9/787: dwrite da/db/f4c [0,4194304] 0 2024-03-18T14:51:13.029 INFO:tasks.workunit.client.1.smithi100.stdout:9/788: stat da/d2c/d2f/f4a 0 2024-03-18T14:51:13.029 INFO:tasks.workunit.client.1.smithi100.stdout:5/710: dwrite d29/d4a/f9c [0,4194304] 0 2024-03-18T14:51:13.037 INFO:tasks.workunit.client.1.smithi100.stdout:6/697: write d6/d12/da2/faf [2734652,118284] 0 2024-03-18T14:51:13.041 INFO:tasks.workunit.client.1.smithi100.stdout:9/789: dread da/d46/d4d/fad [0,4194304] 0 2024-03-18T14:51:13.041 INFO:tasks.workunit.client.1.smithi100.stdout:9/790: dread - da/dd/d29/d71/d72/fb6 zero size 2024-03-18T14:51:13.042 INFO:tasks.workunit.client.1.smithi100.stdout:8/822: dwrite d4/dc/dd/de/d21/d23/f27 [8388608,4194304] 0 2024-03-18T14:51:13.043 INFO:tasks.workunit.client.1.smithi100.stdout:8/823: creat d4/dc/d31/d85/dd6/df9/f126 x:0 0 0 2024-03-18T14:51:13.043 INFO:tasks.workunit.client.1.smithi100.stdout:8/824: fdatasync d4/f60 0 2024-03-18T14:51:13.043 INFO:tasks.workunit.client.1.smithi100.stdout:0/804: truncate dd/d2b/f40 1283370 0 2024-03-18T14:51:13.043 INFO:tasks.workunit.client.1.smithi100.stdout:0/805: readlink dd/d31/d91/d56/d59/lbd 0 2024-03-18T14:51:13.046 INFO:tasks.workunit.client.1.smithi100.stdout:3/781: creat d3/d16/d13/f10b x:0 0 0 2024-03-18T14:51:13.048 INFO:tasks.workunit.client.1.smithi100.stdout:1/772: unlink d8/d2a/d9b/l9f 0 2024-03-18T14:51:13.050 INFO:tasks.workunit.client.1.smithi100.stdout:5/711: link d29/f7e d29/d2d/d39/d71/d96/fe8 0 2024-03-18T14:51:13.056 INFO:tasks.workunit.client.1.smithi100.stdout:3/782: symlink d3/d16/d13/d1f/d3f/d4f/d65/de9/l10c 0 2024-03-18T14:51:13.057 INFO:tasks.workunit.client.1.smithi100.stdout:1/773: creat d8/d1e/d11/d8a/dda/ff2 x:0 0 0 2024-03-18T14:51:13.057 INFO:tasks.workunit.client.1.smithi100.stdout:1/774: truncate d8/d1e/d11/d2d/fe1 882197 0 2024-03-18T14:51:13.057 INFO:tasks.workunit.client.1.smithi100.stdout:0/806: write dd/d31/f52 [1322929,118430] 0 2024-03-18T14:51:13.058 INFO:tasks.workunit.client.1.smithi100.stdout:5/712: symlink d29/d2d/le9 0 2024-03-18T14:51:13.059 INFO:tasks.workunit.client.1.smithi100.stdout:6/698: dwrite d6/d12/fdb [0,4194304] 0 2024-03-18T14:51:13.061 INFO:tasks.workunit.client.1.smithi100.stdout:9/791: rmdir da/d2c 39 2024-03-18T14:51:13.062 INFO:tasks.workunit.client.1.smithi100.stdout:1/775: getdents d8/d2a/d37/d50/d6f/d71/dc9 0 2024-03-18T14:51:13.065 INFO:tasks.workunit.client.1.smithi100.stdout:3/783: write d3/fa [3585537,80127] 0 2024-03-18T14:51:13.065 INFO:tasks.workunit.client.1.smithi100.stdout:3/784: readlink d3/d16/d13/d1f/d3f/l6f 0 2024-03-18T14:51:13.065 INFO:tasks.workunit.client.1.smithi100.stdout:0/807: link dd/d2b/d95/dc1/cf7 dd/d31/d91/d56/de0/d113/c126 0 2024-03-18T14:51:13.065 INFO:tasks.workunit.client.1.smithi100.stdout:0/808: readlink dd/d31/d91/d56/d71/dcc/dcf/l115 0 2024-03-18T14:51:13.066 INFO:tasks.workunit.client.1.smithi100.stdout:5/713: mkdir d29/d2d/d39/d60/dbf/dea 0 2024-03-18T14:51:13.068 INFO:tasks.workunit.client.1.smithi100.stdout:8/825: dwrite d4/dc/dd/de/d21/d23/d47/f86 [0,4194304] 0 2024-03-18T14:51:13.070 INFO:tasks.workunit.client.1.smithi100.stdout:6/699: unlink d6/d7/df/d72/d65/d7a/cac 0 2024-03-18T14:51:13.071 INFO:tasks.workunit.client.1.smithi100.stdout:9/792: mknod da/dc3/cfe 0 2024-03-18T14:51:13.072 INFO:tasks.workunit.client.1.smithi100.stdout:1/776: mkdir d8/d2a/d37/d50/d6f/df3 0 2024-03-18T14:51:13.072 INFO:tasks.workunit.client.1.smithi100.stdout:1/777: truncate d8/d1e/d11/d27/d3f/fe8 682711 0 2024-03-18T14:51:13.072 INFO:tasks.workunit.client.1.smithi100.stdout:1/778: fdatasync d8/d1e/d11/d27/fa7 0 2024-03-18T14:51:13.074 INFO:tasks.workunit.client.1.smithi100.stdout:3/785: mkdir d3/dd/d10d 0 2024-03-18T14:51:13.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/714: mknod d29/d2d/d39/d83/dc1/dbb/d9e/ceb 0 2024-03-18T14:51:13.076 INFO:tasks.workunit.client.1.smithi100.stdout:8/826: symlink d4/d3c/l127 0 2024-03-18T14:51:13.083 INFO:tasks.workunit.client.1.smithi100.stdout:6/700: creat d6/d7/df/d26/d4e/d53/fdc x:0 0 0 2024-03-18T14:51:13.083 INFO:tasks.workunit.client.1.smithi100.stdout:6/701: read d6/d7/df/d26/d4e/f82 [136074,18902] 0 2024-03-18T14:51:13.083 INFO:tasks.workunit.client.1.smithi100.stdout:6/702: readlink d6/d7/df/d26/d4e/d53/la8 0 2024-03-18T14:51:13.084 INFO:tasks.workunit.client.1.smithi100.stdout:9/793: creat da/d88/fff x:0 0 0 2024-03-18T14:51:13.086 INFO:tasks.workunit.client.1.smithi100.stdout:3/786: truncate d3/d16/d13/f47 3770864 0 2024-03-18T14:51:13.088 INFO:tasks.workunit.client.1.smithi100.stdout:5/715: mknod d29/d2d/d39/d61/ddd/cec 0 2024-03-18T14:51:13.088 INFO:tasks.workunit.client.1.smithi100.stdout:5/716: write d29/d2d/dbe/dc2/fe7 [844557,23042] 0 2024-03-18T14:51:13.088 INFO:tasks.workunit.client.1.smithi100.stdout:5/717: creat d29/d2d/d32/d92/fed x:0 0 0 2024-03-18T14:51:13.090 INFO:tasks.workunit.client.1.smithi100.stdout:6/703: getdents d6/d7/df/d26/d4e/d66/d36/d4f 0 2024-03-18T14:51:13.092 INFO:tasks.workunit.client.1.smithi100.stdout:9/794: unlink da/d2c/d2f/f7d 0 2024-03-18T14:51:13.093 INFO:tasks.workunit.client.1.smithi100.stdout:1/779: symlink d8/dd/lf4 0 2024-03-18T14:51:13.096 INFO:tasks.workunit.client.1.smithi100.stdout:3/787: dread d3/d16/d13/d1f/d3f/d4f/d60/f6b [0,4194304] 0 2024-03-18T14:51:13.097 INFO:tasks.workunit.client.1.smithi100.stdout:6/704: rename d6/d7/df/c37 to d6/d7/df/d72/d48/cdd 0 2024-03-18T14:51:13.098 INFO:tasks.workunit.client.1.smithi100.stdout:5/718: read d29/d2d/d39/d83/f85 [1730032,129476] 0 2024-03-18T14:51:13.098 INFO:tasks.workunit.client.1.smithi100.stdout:8/827: dwrite d4/dc/dd/de/f63 [0,4194304] 0 2024-03-18T14:51:13.100 INFO:tasks.workunit.client.1.smithi100.stdout:9/795: creat da/d46/f100 x:0 0 0 2024-03-18T14:51:13.100 INFO:tasks.workunit.client.1.smithi100.stdout:9/796: write da/d46/dd6/fe8 [345300,59047] 0 2024-03-18T14:51:13.100 INFO:tasks.workunit.client.1.smithi100.stdout:9/797: readlink da/l22 0 2024-03-18T14:51:13.102 INFO:tasks.workunit.client.1.smithi100.stdout:3/788: truncate d3/dd/d49/fb7 2412455 0 2024-03-18T14:51:13.102 INFO:tasks.workunit.client.1.smithi100.stdout:3/789: readlink d3/l34 0 2024-03-18T14:51:13.110 INFO:tasks.workunit.client.1.smithi100.stdout:9/798: creat da/dd/d29/d71/d72/db2/f101 x:0 0 0 2024-03-18T14:51:13.112 INFO:tasks.workunit.client.1.smithi100.stdout:1/780: mkdir d8/df5 0 2024-03-18T14:51:13.113 INFO:tasks.workunit.client.1.smithi100.stdout:6/705: getdents d6 0 2024-03-18T14:51:13.114 INFO:tasks.workunit.client.1.smithi100.stdout:9/799: link da/dd/d3b/cd1 da/dc3/c102 0 2024-03-18T14:51:13.115 INFO:tasks.workunit.client.0.smithi012.stdout:5/995: sync 2024-03-18T14:51:13.115 INFO:tasks.workunit.client.0.smithi012.stdout:5/996: chown d3/d1e/f87 505 1 2024-03-18T14:51:13.115 INFO:tasks.workunit.client.0.smithi012.stdout:5/997: fdatasync d3/d29/d35/fd4 0 2024-03-18T14:51:13.116 INFO:tasks.workunit.client.1.smithi100.stdout:1/781: creat d8/d1e/d11/d27/d83/ff6 x:0 0 0 2024-03-18T14:51:13.118 INFO:tasks.workunit.client.1.smithi100.stdout:6/706: creat d6/d7/df/d26/d4e/d66/d36/fde x:0 0 0 2024-03-18T14:51:13.119 INFO:tasks.workunit.client.1.smithi100.stdout:9/800: dread da/dd/d3b/fc6 [4194304,4194304] 0 2024-03-18T14:51:13.119 INFO:tasks.workunit.client.1.smithi100.stdout:9/801: dread - da/dcb/fee zero size 2024-03-18T14:51:13.119 INFO:tasks.workunit.client.0.smithi012.stdout:5/998: truncate d3/d29/d35/fd4 2609612 0 2024-03-18T14:51:13.120 INFO:tasks.workunit.client.0.smithi012.stdout:5/999: creat d3/d15/d42/dc6/d73/d6b/d8f/f14b x:0 0 0 2024-03-18T14:51:13.125 INFO:tasks.workunit.client.1.smithi100.stdout:9/802: rename da/db to da/dd/d3b/d103 0 2024-03-18T14:51:13.125 INFO:tasks.workunit.client.1.smithi100.stdout:9/803: truncate da/dc3/dfc/fab 676803 0 2024-03-18T14:51:13.126 INFO:tasks.workunit.client.1.smithi100.stdout:9/804: mkdir da/dc3/d104 0 2024-03-18T14:51:13.127 INFO:tasks.workunit.client.1.smithi100.stdout:9/805: dread - da/dd/da0/fa6 zero size 2024-03-18T14:51:13.127 INFO:tasks.workunit.client.1.smithi100.stdout:9/806: chown da/d51/cf2 30869146 1 2024-03-18T14:51:13.127 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf -- ./tmp.lGbcUYkaOr 2024-03-18T14:51:13.128 INFO:tasks.workunit.client.1.smithi100.stdout:9/807: truncate da/dd/d3c/d6b/f84 2693527 0 2024-03-18T14:51:13.136 INFO:tasks.workunit.client.1.smithi100.stdout:5/719: dwrite d29/d2d/d39/f46 [0,4194304] 0 2024-03-18T14:51:13.145 INFO:tasks.workunit.client.1.smithi100.stdout:3/790: dwrite d3/d16/d13/d1f/d3f/d4f/d65/feb [0,4194304] 0 2024-03-18T14:51:13.145 INFO:tasks.workunit.client.1.smithi100.stdout:1/782: dwrite d8/d1e/d11/d27/d83/ff6 [0,4194304] 0 2024-03-18T14:51:13.147 INFO:tasks.workunit.client.1.smithi100.stdout:7/790: sync 2024-03-18T14:51:13.148 INFO:tasks.workunit.client.1.smithi100.stdout:7/791: fdatasync d5/d14/d16/d63/d124/fdf 0 2024-03-18T14:51:13.150 INFO:tasks.workunit.client.1.smithi100.stdout:3/791: mkdir d3/d16/dca/d10e 0 2024-03-18T14:51:13.150 INFO:tasks.workunit.client.1.smithi100.stdout:3/792: chown d3/dd/d21/d3e/df8/db9/cdc 23467 1 2024-03-18T14:51:13.150 INFO:tasks.workunit.client.1.smithi100.stdout:3/793: creat d3/d16/d13/d1f/d3f/d4f/d60/da9/f10f x:0 0 0 2024-03-18T14:51:13.152 INFO:tasks.workunit.client.1.smithi100.stdout:6/707: dwrite d6/d12/f5c [4194304,4194304] 0 2024-03-18T14:51:13.156 INFO:tasks.workunit.client.1.smithi100.stdout:1/783: symlink d8/d1e/lf7 0 2024-03-18T14:51:13.156 INFO:tasks.workunit.client.1.smithi100.stdout:9/808: dwrite da/d46/d8b/f35 [0,4194304] 0 2024-03-18T14:51:13.157 INFO:tasks.workunit.client.1.smithi100.stdout:9/809: creat da/dcb/f105 x:0 0 0 2024-03-18T14:51:13.157 INFO:tasks.workunit.client.1.smithi100.stdout:5/720: dwrite d29/d2d/d32/d92/daa/fca [0,4194304] 0 2024-03-18T14:51:13.161 INFO:tasks.workunit.client.1.smithi100.stdout:3/794: getdents d3/dd/d21/d3e/df8/db9/df0 0 2024-03-18T14:51:13.161 INFO:tasks.workunit.client.1.smithi100.stdout:3/795: write d3/dd/d21/dae/fd1 [762457,93250] 0 2024-03-18T14:51:13.161 INFO:tasks.workunit.client.1.smithi100.stdout:3/796: chown d3/dd/d49/fb7 47184675 1 2024-03-18T14:51:13.166 INFO:tasks.workunit.client.1.smithi100.stdout:6/708: mknod d6/d7/df/d26/d4e/db5/cdf 0 2024-03-18T14:51:13.166 INFO:tasks.workunit.client.1.smithi100.stdout:6/709: write d6/d7/df/f7c [1221335,18639] 0 2024-03-18T14:51:13.166 INFO:tasks.workunit.client.1.smithi100.stdout:6/710: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f5d [4893394,29359] 0 2024-03-18T14:51:13.167 INFO:tasks.workunit.client.1.smithi100.stdout:7/792: dwrite d5/fd [0,4194304] 0 2024-03-18T14:51:13.177 INFO:tasks.workunit.client.1.smithi100.stdout:1/784: rename d8/d1e/d11/d2d/fe1 to d8/d1e/d11/d27/d3f/daa/dd5/ff8 0 2024-03-18T14:51:13.177 INFO:tasks.workunit.client.1.smithi100.stdout:1/785: stat d8/d1e/lf7 0 2024-03-18T14:51:13.177 INFO:tasks.workunit.client.1.smithi100.stdout:1/786: truncate d8/d2a/fb1 111101 0 2024-03-18T14:51:13.184 INFO:tasks.workunit.client.1.smithi100.stdout:5/721: mkdir d29/d2d/d39/d83/dc1/dbb/d9e/dd7/dee 0 2024-03-18T14:51:13.186 INFO:tasks.workunit.client.1.smithi100.stdout:3/797: symlink d3/d16/d13/d17/d7d/l110 0 2024-03-18T14:51:13.188 INFO:tasks.workunit.client.1.smithi100.stdout:6/711: creat d6/d7/df/d26/d4e/d66/d36/d4f/db8/fe0 x:0 0 0 2024-03-18T14:51:13.188 INFO:tasks.workunit.client.1.smithi100.stdout:6/712: dread - d6/f91 zero size 2024-03-18T14:51:13.193 INFO:tasks.workunit.client.1.smithi100.stdout:7/793: mkdir d5/d14/d16/d2e/d126 0 2024-03-18T14:51:13.196 INFO:tasks.workunit.client.1.smithi100.stdout:6/713: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [0,4194304] 0 2024-03-18T14:51:13.197 INFO:tasks.workunit.client.1.smithi100.stdout:1/787: creat d8/d1e/d11/d27/d73/ff9 x:0 0 0 2024-03-18T14:51:13.197 INFO:tasks.workunit.client.1.smithi100.stdout:1/788: stat d8/d1e/d11/d2d/l79 0 2024-03-18T14:51:13.197 INFO:tasks.workunit.client.1.smithi100.stdout:1/789: fdatasync d8/d1e/f86 0 2024-03-18T14:51:13.197 INFO:tasks.workunit.client.1.smithi100.stdout:1/790: creat d8/d1e/d11/d2d/ffa x:0 0 0 2024-03-18T14:51:13.204 INFO:tasks.workunit.client.1.smithi100.stdout:5/722: creat d29/d2d/d39/d61/ddd/fef x:0 0 0 2024-03-18T14:51:13.206 INFO:tasks.workunit.client.1.smithi100.stdout:6/714: dread d6/d7/df/d26/f45 [0,4194304] 0 2024-03-18T14:51:13.209 INFO:tasks.workunit.client.1.smithi100.stdout:3/798: creat d3/dd/d21/dcd/d103/f111 x:0 0 0 2024-03-18T14:51:13.210 INFO:tasks.workunit.client.1.smithi100.stdout:3/799: dread - d3/dd/d21/d3e/df8/d74/fb5 zero size 2024-03-18T14:51:13.218 INFO:tasks.workunit.client.1.smithi100.stdout:1/791: dwrite d8/d2a/d37/d50/d6f/d71/fd0 [0,4194304] 0 2024-03-18T14:51:13.218 INFO:tasks.workunit.client.1.smithi100.stdout:1/792: readlink d8/d1e/d11/d8a/l98 0 2024-03-18T14:51:13.218 INFO:tasks.workunit.client.1.smithi100.stdout:7/794: dwrite d5/f72 [0,4194304] 0 2024-03-18T14:51:13.218 INFO:tasks.workunit.client.1.smithi100.stdout:7/795: write d5/d22/d3b/d47/f49 [2047092,13780] 0 2024-03-18T14:51:13.230 INFO:tasks.workunit.client.1.smithi100.stdout:5/723: symlink d29/d2d/d39/d74/lf0 0 2024-03-18T14:51:13.235 INFO:tasks.workunit.client.1.smithi100.stdout:1/793: dwrite d8/d1e/f45 [0,4194304] 0 2024-03-18T14:51:13.237 INFO:tasks.workunit.client.1.smithi100.stdout:3/800: mkdir d3/d16/d13/d1f/d3f/d4f/de0/de6/d112 0 2024-03-18T14:51:13.246 INFO:tasks.workunit.client.1.smithi100.stdout:7/796: creat d5/d14/d16/d63/d124/dc3/f127 x:0 0 0 2024-03-18T14:51:13.249 INFO:tasks.workunit.client.1.smithi100.stdout:9/810: rename da/dd/da0 to da/dc3/ddb/d106 0 2024-03-18T14:51:13.251 INFO:tasks.workunit.client.1.smithi100.stdout:3/801: creat d3/d16/d13/d1f/d3f/d4f/de0/de6/f113 x:0 0 0 2024-03-18T14:51:13.252 INFO:tasks.workunit.client.1.smithi100.stdout:3/802: write d3/da5/fe8 [605675,26028] 0 2024-03-18T14:51:13.253 INFO:tasks.workunit.client.1.smithi100.stdout:7/797: mkdir d5/d90/da1/d128 0 2024-03-18T14:51:13.258 INFO:tasks.workunit.client.1.smithi100.stdout:1/794: rename d8/d30/cd2 to d8/d2a/d37/d50/dc5/cfb 0 2024-03-18T14:51:13.258 INFO:tasks.workunit.client.1.smithi100.stdout:1/795: truncate d8/d2a/d37/fe0 676104 0 2024-03-18T14:51:13.259 INFO:tasks.workunit.client.1.smithi100.stdout:9/811: unlink da/d46/d8b/f66 0 2024-03-18T14:51:13.262 INFO:tasks.workunit.client.1.smithi100.stdout:7/798: unlink d5/d14/d16/d63/d124/dd1/d86/l99 0 2024-03-18T14:51:13.263 INFO:tasks.workunit.client.1.smithi100.stdout:1/796: mknod d8/d2a/d37/d50/d6f/d71/cfc 0 2024-03-18T14:51:13.265 INFO:tasks.workunit.client.1.smithi100.stdout:9/812: getdents da/dd/d29 0 2024-03-18T14:51:13.265 INFO:tasks.workunit.client.1.smithi100.stdout:9/813: write da/d88/fff [547157,29169] 0 2024-03-18T14:51:13.266 INFO:tasks.workunit.client.1.smithi100.stdout:3/803: mkdir d3/d114 0 2024-03-18T14:51:13.266 INFO:tasks.workunit.client.1.smithi100.stdout:3/804: stat d3/dd/c6d 0 2024-03-18T14:51:13.267 INFO:tasks.workunit.client.1.smithi100.stdout:7/799: mkdir d5/d22/d24/d6e/d7a/dc2/d111/d129 0 2024-03-18T14:51:13.270 INFO:tasks.workunit.client.1.smithi100.stdout:9/814: rmdir da/d2c/d70/ddd 0 2024-03-18T14:51:13.273 INFO:tasks.workunit.client.1.smithi100.stdout:1/797: dread d8/d1e/d11/d27/d3f/f74 [0,4194304] 0 2024-03-18T14:51:13.273 INFO:tasks.workunit.client.1.smithi100.stdout:1/798: fdatasync d8/d1e/d11/d27/d3f/f49 0 2024-03-18T14:51:13.274 INFO:tasks.workunit.client.1.smithi100.stdout:9/815: rename da/dd/d3e/cd7 to da/dc3/ddb/d106/c107 0 2024-03-18T14:51:13.274 INFO:tasks.workunit.client.1.smithi100.stdout:9/816: truncate da/d2c/f77 187195 0 2024-03-18T14:51:13.277 INFO:tasks.workunit.client.1.smithi100.stdout:1/799: unlink d8/d1e/d11/d27/d3f/f51 0 2024-03-18T14:51:13.277 INFO:tasks.workunit.client.1.smithi100.stdout:1/800: write d8/d1e/d11/d8a/fe4 [963124,82361] 0 2024-03-18T14:51:13.277 INFO:tasks.workunit.client.1.smithi100.stdout:1/801: rename d8 to d8/d1e/d11/d27/d73/d89/dfd 22 2024-03-18T14:51:13.278 INFO:tasks.workunit.client.1.smithi100.stdout:9/817: rmdir da/d46/d4d 39 2024-03-18T14:51:13.280 INFO:tasks.workunit.client.1.smithi100.stdout:9/818: creat da/dd/d29/d71/df5/f108 x:0 0 0 2024-03-18T14:51:13.284 INFO:tasks.workunit.client.1.smithi100.stdout:9/819: dread da/d51/f69 [0,4194304] 0 2024-03-18T14:51:13.288 INFO:tasks.workunit.client.1.smithi100.stdout:9/820: rename da/dd/d29/d71/d72/cd8 to da/dc3/d104/c109 0 2024-03-18T14:51:13.289 INFO:tasks.workunit.client.1.smithi100.stdout:9/821: dread - da/d46/d8b/f37 zero size 2024-03-18T14:51:13.289 INFO:tasks.workunit.client.1.smithi100.stdout:9/822: creat da/dd/d29/d9e/f10a x:0 0 0 2024-03-18T14:51:13.299 INFO:tasks.workunit.client.1.smithi100.stdout:9/823: truncate da/dd/d3c/d6b/f84 1391191 0 2024-03-18T14:51:13.299 INFO:tasks.workunit.client.1.smithi100.stdout:9/824: write da/dd/d29/d71/df5/f93 [42238,67571] 0 2024-03-18T14:51:13.302 INFO:tasks.workunit.client.1.smithi100.stdout:9/825: symlink da/d51/d55/l10b 0 2024-03-18T14:51:13.310 INFO:tasks.workunit.client.1.smithi100.stdout:2/789: sync 2024-03-18T14:51:13.310 INFO:tasks.workunit.client.1.smithi100.stdout:4/803: sync 2024-03-18T14:51:13.310 INFO:tasks.workunit.client.1.smithi100.stdout:2/790: dread - d3/d1a/fbe zero size 2024-03-18T14:51:13.310 INFO:tasks.workunit.client.1.smithi100.stdout:2/791: chown d3/da/de/d20/d82/dad/dc5/le1 30023191 1 2024-03-18T14:51:13.311 INFO:tasks.workunit.client.1.smithi100.stdout:8/828: sync 2024-03-18T14:51:13.311 INFO:tasks.workunit.client.1.smithi100.stdout:0/809: sync 2024-03-18T14:51:13.311 INFO:tasks.workunit.client.1.smithi100.stdout:1/802: write d8/d2a/d37/d50/d6f/fac [248404,128968] 0 2024-03-18T14:51:13.311 INFO:tasks.workunit.client.1.smithi100.stdout:4/804: mknod d9/d27/cf3 0 2024-03-18T14:51:13.311 INFO:tasks.workunit.client.1.smithi100.stdout:4/805: write d9/fe [708271,35095] 0 2024-03-18T14:51:13.312 INFO:tasks.workunit.client.1.smithi100.stdout:4/806: creat d9/d13/d5a/d5b/da0/dcd/ff4 x:0 0 0 2024-03-18T14:51:13.316 INFO:tasks.workunit.client.1.smithi100.stdout:8/829: dread d4/dc/d1f/f32 [0,4194304] 0 2024-03-18T14:51:13.316 INFO:tasks.workunit.client.1.smithi100.stdout:2/792: symlink d3/d6/d5f/l10a 0 2024-03-18T14:51:13.316 INFO:tasks.workunit.client.1.smithi100.stdout:2/793: fdatasync d3/d1a/f2d 0 2024-03-18T14:51:13.316 INFO:tasks.workunit.client.1.smithi100.stdout:2/794: chown d3/da/de/d20/dea/d4f/fc2 21385625 1 2024-03-18T14:51:13.316 INFO:tasks.workunit.client.1.smithi100.stdout:2/795: creat d3/dc3/f10b x:0 0 0 2024-03-18T14:51:13.318 INFO:tasks.workunit.client.1.smithi100.stdout:1/803: rename d8/d1e/d11/d27/d73/ff9 to d8/d30/d61/d8d/dde/de6/ffe 0 2024-03-18T14:51:13.318 INFO:tasks.workunit.client.1.smithi100.stdout:9/826: dread da/d88/fff [0,4194304] 0 2024-03-18T14:51:13.319 INFO:tasks.workunit.client.1.smithi100.stdout:5/724: dwrite d29/d2d/d39/d61/f89 [0,4194304] 0 2024-03-18T14:51:13.321 INFO:tasks.workunit.client.1.smithi100.stdout:3/805: dwrite d3/ff [0,4194304] 0 2024-03-18T14:51:13.321 INFO:tasks.workunit.client.1.smithi100.stdout:3/806: write d3/d16/d13/d1f/d3f/d4f/d60/f63 [233129,108081] 0 2024-03-18T14:51:13.321 INFO:tasks.workunit.client.1.smithi100.stdout:3/807: fsync d3/d16/d13/d17/d7d/fa3 0 2024-03-18T14:51:13.323 INFO:tasks.workunit.client.1.smithi100.stdout:9/827: dread da/dc3/dfc/f75 [0,4194304] 0 2024-03-18T14:51:13.323 INFO:tasks.workunit.client.1.smithi100.stdout:9/828: chown da/dd/d3b/d103/db0 4538751 1 2024-03-18T14:51:13.332 INFO:tasks.workunit.client.1.smithi100.stdout:5/725: read d29/d2d/d32/d92/daa/fa1 [8080600,102537] 0 2024-03-18T14:51:13.332 INFO:tasks.workunit.client.1.smithi100.stdout:5/726: chown d29/d2d/d8f/caf 491517561 1 2024-03-18T14:51:13.332 INFO:tasks.workunit.client.1.smithi100.stdout:1/804: dread d8/d1e/f45 [0,4194304] 0 2024-03-18T14:51:13.338 INFO:tasks.workunit.client.1.smithi100.stdout:0/810: dwrite dd/f16 [4194304,4194304] 0 2024-03-18T14:51:13.338 INFO:tasks.workunit.client.1.smithi100.stdout:0/811: getdents dd/d31/d91/d56/d71/dcc/dcf/d120 0 2024-03-18T14:51:13.338 INFO:tasks.workunit.client.1.smithi100.stdout:0/812: fsync dd/d61/fe8 0 2024-03-18T14:51:13.338 INFO:tasks.workunit.client.1.smithi100.stdout:4/807: rmdir d9/d13/d5a/da4 39 2024-03-18T14:51:13.341 INFO:tasks.workunit.client.1.smithi100.stdout:3/808: mknod d3/d16/d13/d1f/d7c/c115 0 2024-03-18T14:51:13.341 INFO:tasks.workunit.client.1.smithi100.stdout:3/809: write d3/d16/d13/d1f/d3f/d4f/d60/f63 [959681,39200] 0 2024-03-18T14:51:13.341 INFO:tasks.workunit.client.1.smithi100.stdout:3/810: write d3/d16/d13/d17/dab/fb0 [366578,65747] 0 2024-03-18T14:51:13.341 INFO:tasks.workunit.client.1.smithi100.stdout:3/811: fdatasync d3/d16/d13/d17/d7d/fad 0 2024-03-18T14:51:13.346 INFO:tasks.workunit.client.1.smithi100.stdout:2/796: dwrite d3/d6/f9 [0,4194304] 0 2024-03-18T14:51:13.347 INFO:tasks.workunit.client.1.smithi100.stdout:2/797: write d3/da/de/d20/f24 [246763,96464] 0 2024-03-18T14:51:13.347 INFO:tasks.workunit.client.1.smithi100.stdout:2/798: creat d3/d1a/d76/f10c x:0 0 0 2024-03-18T14:51:13.347 INFO:tasks.workunit.client.1.smithi100.stdout:1/805: dwrite d8/d2a/d37/f5f [0,4194304] 0 2024-03-18T14:51:13.353 INFO:tasks.workunit.client.1.smithi100.stdout:1/806: dread d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:13.357 INFO:tasks.workunit.client.1.smithi100.stdout:9/829: unlink da/f1f 0 2024-03-18T14:51:13.361 INFO:tasks.workunit.client.1.smithi100.stdout:8/830: rename d4/dc/dd/de to d4/dc/d1f/d33/d128 0 2024-03-18T14:51:13.362 INFO:tasks.workunit.client.1.smithi100.stdout:5/727: truncate d29/d2d/d39/f56 62965 0 2024-03-18T14:51:13.365 INFO:tasks.workunit.client.1.smithi100.stdout:0/813: mkdir dd/d31/d127 0 2024-03-18T14:51:13.366 INFO:tasks.workunit.client.1.smithi100.stdout:4/808: write d9/d13/d25/fe6 [2012865,98520] 0 2024-03-18T14:51:13.369 INFO:tasks.workunit.client.1.smithi100.stdout:3/812: link d3/d16/d13/d1f/d3f/d4f/d65/lc3 d3/da5/l116 0 2024-03-18T14:51:13.370 INFO:tasks.workunit.client.1.smithi100.stdout:8/831: dread d4/dc/d1f/d59/d6c/f98 [0,4194304] 0 2024-03-18T14:51:13.370 INFO:tasks.workunit.client.1.smithi100.stdout:8/832: readlink d4/dc/d1f/d33/d128/d21/d23/d47/le7 0 2024-03-18T14:51:13.372 INFO:tasks.workunit.client.1.smithi100.stdout:2/799: truncate d3/d1a/d76/fc7 108281 0 2024-03-18T14:51:13.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/814: dread dd/d2b/f4c [0,4194304] 0 2024-03-18T14:51:13.376 INFO:tasks.workunit.client.1.smithi100.stdout:3/813: dread d3/d16/d13/f47 [0,4194304] 0 2024-03-18T14:51:13.378 INFO:tasks.workunit.client.1.smithi100.stdout:9/830: mknod da/d46/dd6/de0/c10c 0 2024-03-18T14:51:13.379 INFO:tasks.workunit.client.1.smithi100.stdout:5/728: rmdir d29/d2d/dbe 39 2024-03-18T14:51:13.380 INFO:tasks.workunit.client.1.smithi100.stdout:8/833: truncate d4/dc/dd/d48/f117 3223485 0 2024-03-18T14:51:13.382 INFO:tasks.workunit.client.1.smithi100.stdout:2/800: mknod d3/d6/c10d 0 2024-03-18T14:51:13.384 INFO:tasks.workunit.client.1.smithi100.stdout:0/815: mknod dd/d31/d91/d56/d71/dcc/c128 0 2024-03-18T14:51:13.386 INFO:tasks.workunit.client.1.smithi100.stdout:3/814: link d3/dd/l19 d3/dd/d21/d3e/df8/d74/l117 0 2024-03-18T14:51:13.388 INFO:tasks.workunit.client.1.smithi100.stdout:9/831: mknod da/dd/d3b/d103/c10d 0 2024-03-18T14:51:13.393 INFO:tasks.workunit.client.1.smithi100.stdout:8/834: creat d4/dc/dd/d48/dd8/df0/f129 x:0 0 0 2024-03-18T14:51:13.394 INFO:tasks.workunit.client.1.smithi100.stdout:2/801: mknod d3/da/de/d20/dea/c10e 0 2024-03-18T14:51:13.394 INFO:tasks.workunit.client.1.smithi100.stdout:2/802: write d3/d6/d29/d36/d4a/ff5 [759718,16432] 0 2024-03-18T14:51:13.394 INFO:tasks.workunit.client.1.smithi100.stdout:2/803: creat d3/f10f x:0 0 0 2024-03-18T14:51:13.395 INFO:tasks.workunit.client.1.smithi100.stdout:2/804: write d3/d1a/d76/f10c [350547,75467] 0 2024-03-18T14:51:13.398 INFO:tasks.workunit.client.1.smithi100.stdout:0/816: creat dd/d61/dcd/dff/f129 x:0 0 0 2024-03-18T14:51:13.399 INFO:tasks.workunit.client.1.smithi100.stdout:0/817: fdatasync dd/d31/d91/d56/d62/fd8 0 2024-03-18T14:51:13.400 INFO:tasks.workunit.client.1.smithi100.stdout:3/815: getdents d3/d16/d13/d17/dab 0 2024-03-18T14:51:13.402 INFO:tasks.workunit.client.1.smithi100.stdout:9/832: truncate da/d46/d4d/f9f 545887 0 2024-03-18T14:51:13.402 INFO:tasks.workunit.client.1.smithi100.stdout:9/833: write da/d2c/d70/fc4 [586531,110210] 0 2024-03-18T14:51:13.409 INFO:tasks.workunit.client.1.smithi100.stdout:5/729: getdents d29/d4a 0 2024-03-18T14:51:13.409 INFO:tasks.workunit.client.1.smithi100.stdout:5/730: read - d29/d2d/d32/d92/fed zero size 2024-03-18T14:51:13.411 INFO:tasks.workunit.client.1.smithi100.stdout:8/835: creat d4/dc/d1f/d33/d128/d19/dc0/d113/f12a x:0 0 0 2024-03-18T14:51:13.411 INFO:tasks.workunit.client.1.smithi100.stdout:8/836: truncate d4/dc/d1f/d59/d6c/fcf 297001 0 2024-03-18T14:51:13.415 INFO:tasks.workunit.client.1.smithi100.stdout:2/805: link d3/d6/d5f/d63/f72 d3/da/de/d20/d82/f110 0 2024-03-18T14:51:13.418 INFO:tasks.workunit.client.1.smithi100.stdout:9/834: mknod da/dd/d29/d9e/dcd/c10e 0 2024-03-18T14:51:13.418 INFO:tasks.workunit.client.1.smithi100.stdout:9/835: write da/d51/d55/d7e/f9d [5201076,74210] 0 2024-03-18T14:51:13.419 INFO:tasks.workunit.client.1.smithi100.stdout:9/836: write da/dd/d3b/fbc [798639,53089] 0 2024-03-18T14:51:13.419 INFO:tasks.workunit.client.1.smithi100.stdout:9/837: creat da/d88/f10f x:0 0 0 2024-03-18T14:51:13.419 INFO:tasks.workunit.client.1.smithi100.stdout:0/818: dwrite dd/d20/d44/f5e [0,4194304] 0 2024-03-18T14:51:13.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/731: rename d29/d2d/d39/fae to d29/d2d/dbe/dc2/ff1 0 2024-03-18T14:51:13.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/732: fsync d29/d2d/d39/d71/d96/fe0 0 2024-03-18T14:51:13.422 INFO:tasks.workunit.client.1.smithi100.stdout:8/837: creat d4/dc/d1f/d33/d128/d19/d2c/da2/f12b x:0 0 0 2024-03-18T14:51:13.426 INFO:tasks.workunit.client.1.smithi100.stdout:3/816: rmdir d3/dd/d49 39 2024-03-18T14:51:13.426 INFO:tasks.workunit.client.1.smithi100.stdout:3/817: write d3/f29 [6188159,27281] 0 2024-03-18T14:51:13.426 INFO:tasks.workunit.client.1.smithi100.stdout:3/818: write d3/ff7 [121936,47126] 0 2024-03-18T14:51:13.426 INFO:tasks.workunit.client.1.smithi100.stdout:3/819: stat d3/d16/d13/d1f/d3f/d4f/d65 0 2024-03-18T14:51:13.431 INFO:tasks.workunit.client.1.smithi100.stdout:9/838: mknod da/dd/d3b/c110 0 2024-03-18T14:51:13.431 INFO:tasks.workunit.client.1.smithi100.stdout:9/839: chown da/d51/d55/d7e/c85 96254 1 2024-03-18T14:51:13.431 INFO:tasks.workunit.client.1.smithi100.stdout:9/840: fdatasync da/d46/d8b/f3f 0 2024-03-18T14:51:13.443 INFO:tasks.workunit.client.1.smithi100.stdout:0/819: unlink dd/f2e 0 2024-03-18T14:51:13.447 INFO:tasks.workunit.client.1.smithi100.stdout:0/820: write dd/d31/d91/d56/d71/dcc/dcf/fdf [3882325,21080] 0 2024-03-18T14:51:13.447 INFO:tasks.workunit.client.1.smithi100.stdout:0/821: creat dd/d31/d91/d56/f12a x:0 0 0 2024-03-18T14:51:13.447 INFO:tasks.workunit.client.1.smithi100.stdout:0/822: creat dd/d31/d91/d56/d62/d6e/f12b x:0 0 0 2024-03-18T14:51:13.447 INFO:tasks.workunit.client.1.smithi100.stdout:0/823: write dd/f55 [2919893,14342] 0 2024-03-18T14:51:13.449 INFO:tasks.workunit.client.1.smithi100.stdout:5/733: rmdir d29/d2d/d39/d83/dc1/dbb/d9e/dd7 39 2024-03-18T14:51:13.450 INFO:tasks.workunit.client.1.smithi100.stdout:8/838: mkdir d4/d5a/dbb/d12c 0 2024-03-18T14:51:13.450 INFO:tasks.workunit.client.1.smithi100.stdout:5/734: fsync d29/d2d/d39/d83/dc1/dbb/f76 0 2024-03-18T14:51:13.450 INFO:tasks.workunit.client.1.smithi100.stdout:3/820: mknod d3/d16/d13/d17/d7d/c118 0 2024-03-18T14:51:13.450 INFO:tasks.workunit.client.1.smithi100.stdout:9/841: dwrite da/d46/d8b/f3f [0,4194304] 0 2024-03-18T14:51:13.450 INFO:tasks.workunit.client.1.smithi100.stdout:9/842: write da/d46/d4d/fad [543479,84841] 0 2024-03-18T14:51:13.456 INFO:tasks.workunit.client.1.smithi100.stdout:0/824: creat dd/d31/d91/d56/de0/f12c x:0 0 0 2024-03-18T14:51:13.460 INFO:tasks.workunit.client.1.smithi100.stdout:0/825: dread dd/d2b/d95/d79/d7d/fd0 [0,4194304] 0 2024-03-18T14:51:13.460 INFO:tasks.workunit.client.1.smithi100.stdout:5/735: write d29/d2d/d39/d65/f6a [3209328,63766] 0 2024-03-18T14:51:13.465 INFO:tasks.workunit.client.1.smithi100.stdout:3/821: mknod d3/d16/c119 0 2024-03-18T14:51:13.469 INFO:tasks.workunit.client.1.smithi100.stdout:8/839: dread d4/dc/f8a [0,4194304] 0 2024-03-18T14:51:13.469 INFO:tasks.workunit.client.1.smithi100.stdout:9/843: creat da/dd/d3b/d103/f111 x:0 0 0 2024-03-18T14:51:13.469 INFO:tasks.workunit.client.1.smithi100.stdout:9/844: write da/dd/d3e/ffa [34131,114498] 0 2024-03-18T14:51:13.469 INFO:tasks.workunit.client.1.smithi100.stdout:9/845: chown da/d2c/f77 7814770 1 2024-03-18T14:51:13.470 INFO:tasks.workunit.client.1.smithi100.stdout:0/826: creat dd/d20/f12d x:0 0 0 2024-03-18T14:51:13.470 INFO:tasks.workunit.client.1.smithi100.stdout:0/827: fdatasync dd/d20/f32 0 2024-03-18T14:51:13.470 INFO:tasks.workunit.client.1.smithi100.stdout:5/736: symlink d29/dd8/lf2 0 2024-03-18T14:51:13.474 INFO:tasks.workunit.client.1.smithi100.stdout:3/822: symlink d3/d16/d13/d1f/d3f/d4f/d65/de9/l11a 0 2024-03-18T14:51:13.475 INFO:tasks.workunit.client.1.smithi100.stdout:9/846: mknod da/d46/d8b/c112 0 2024-03-18T14:51:13.477 INFO:tasks.workunit.client.1.smithi100.stdout:0/828: mknod dd/d31/d91/d65/c12e 0 2024-03-18T14:51:13.478 INFO:tasks.workunit.client.1.smithi100.stdout:3/823: rename d3/d16/d13/d1f/d7c/f83 to d3/d16/d13/d1f/d3f/d4f/d60/da9/f11b 0 2024-03-18T14:51:13.485 INFO:tasks.workunit.client.1.smithi100.stdout:8/840: dwrite d4/dc/d1f/d33/d128/d21/d23/d47/f5b [4194304,4194304] 0 2024-03-18T14:51:13.485 INFO:tasks.workunit.client.1.smithi100.stdout:8/841: chown d4/dc/d1f/d33/d128/d19/d37/f7f 0 1 2024-03-18T14:51:13.485 INFO:tasks.workunit.client.1.smithi100.stdout:8/842: fsync d4/f15 0 2024-03-18T14:51:13.494 INFO:tasks.workunit.client.1.smithi100.stdout:8/843: write d4/dc/d1f/d33/d128/d21/d23/d25/f71 [814722,32044] 0 2024-03-18T14:51:13.494 INFO:tasks.workunit.client.1.smithi100.stdout:8/844: dread - d4/dc/d97/f106 zero size 2024-03-18T14:51:13.508 INFO:tasks.workunit.client.1.smithi100.stdout:8/845: dread d4/f6 [0,4194304] 0 2024-03-18T14:51:13.508 INFO:tasks.workunit.client.1.smithi100.stdout:8/846: fdatasync d4/dc/d31/d85/d8e/ff2 0 2024-03-18T14:51:13.509 INFO:tasks.workunit.client.1.smithi100.stdout:8/847: rename d4/f6 to d4/dc/d1f/d33/d128/d21/d23/d47/dff/f12d 0 2024-03-18T14:51:13.516 INFO:tasks.workunit.client.1.smithi100.stdout:9/847: dwrite da/dcb/f105 [0,4194304] 0 2024-03-18T14:51:13.517 INFO:tasks.workunit.client.1.smithi100.stdout:3/824: dwrite d3/d16/d13/d1f/d7c/f89 [0,4194304] 0 2024-03-18T14:51:13.517 INFO:tasks.workunit.client.1.smithi100.stdout:3/825: write d3/d16/d13/d17/d7d/f101 [581943,125838] 0 2024-03-18T14:51:13.517 INFO:tasks.workunit.client.1.smithi100.stdout:0/829: dwrite dd/d20/f97 [0,4194304] 0 2024-03-18T14:51:13.520 INFO:tasks.workunit.client.1.smithi100.stdout:9/848: rename da/d46/d4d/f9f to da/d51/f113 0 2024-03-18T14:51:13.520 INFO:tasks.workunit.client.1.smithi100.stdout:9/849: chown da/dc3/ddb/d106 5369 1 2024-03-18T14:51:13.520 INFO:tasks.workunit.client.1.smithi100.stdout:9/850: fdatasync da/d51/d55/f5c 0 2024-03-18T14:51:13.520 INFO:tasks.workunit.client.1.smithi100.stdout:9/851: fdatasync da/d46/d8b/f3f 0 2024-03-18T14:51:13.523 INFO:tasks.workunit.client.1.smithi100.stdout:3/826: symlink d3/dd/d21/d3e/l11c 0 2024-03-18T14:51:13.523 INFO:tasks.workunit.client.1.smithi100.stdout:3/827: chown d3/d16/d13/d1f/fda 13 1 2024-03-18T14:51:13.523 INFO:tasks.workunit.client.1.smithi100.stdout:3/828: chown d3/d16/d13/d1f/d37/c66 1 1 2024-03-18T14:51:13.530 INFO:tasks.workunit.client.1.smithi100.stdout:9/852: dread da/dd/d3b/d103/f13 [0,4194304] 0 2024-03-18T14:51:13.532 INFO:tasks.workunit.client.1.smithi100.stdout:9/853: dread da/dd/d3e/ffa [0,4194304] 0 2024-03-18T14:51:13.539 INFO:tasks.workunit.client.1.smithi100.stdout:0/830: dwrite dd/d31/d91/d56/d62/d6e/f12b [0,4194304] 0 2024-03-18T14:51:13.542 INFO:tasks.workunit.client.1.smithi100.stdout:9/854: dread da/d46/d8b/f3a [0,4194304] 0 2024-03-18T14:51:13.551 INFO:tasks.workunit.client.1.smithi100.stdout:8/848: dwrite d4/dc/d1f/d33/d128/d21/f72 [0,4194304] 0 2024-03-18T14:51:13.553 INFO:tasks.workunit.client.1.smithi100.stdout:0/831: mkdir dd/d31/d91/d56/d71/dcc/dcf/d120/d12f 0 2024-03-18T14:51:13.554 INFO:tasks.workunit.client.1.smithi100.stdout:6/715: sync 2024-03-18T14:51:13.554 INFO:tasks.workunit.client.1.smithi100.stdout:7/800: sync 2024-03-18T14:51:13.554 INFO:tasks.workunit.client.1.smithi100.stdout:7/801: chown d5/d30/d4d/d5a/f6f 4771276 1 2024-03-18T14:51:13.555 INFO:tasks.workunit.client.1.smithi100.stdout:9/855: mkdir da/d2c/d2f/d114 0 2024-03-18T14:51:13.555 INFO:tasks.workunit.client.1.smithi100.stdout:9/856: fsync da/d51/d55/d7e/ff8 0 2024-03-18T14:51:13.560 INFO:tasks.workunit.client.1.smithi100.stdout:9/857: stat da/dc3/c102 0 2024-03-18T14:51:13.560 INFO:tasks.workunit.client.1.smithi100.stdout:9/858: read da/d46/dd6/fe8 [350695,99438] 0 2024-03-18T14:51:13.561 INFO:tasks.workunit.client.1.smithi100.stdout:9/859: chown da/dd/d3c/d4b/l6c 51229 1 2024-03-18T14:51:13.561 INFO:tasks.workunit.client.1.smithi100.stdout:9/860: chown da/d46/lb3 27930 1 2024-03-18T14:51:13.561 INFO:tasks.workunit.client.1.smithi100.stdout:9/861: write da/dc3/dfc/f6e [1049347,81376] 0 2024-03-18T14:51:13.561 INFO:tasks.workunit.client.1.smithi100.stdout:0/832: dread dd/d31/d91/d56/d71/dcc/f124 [0,4194304] 0 2024-03-18T14:51:13.563 INFO:tasks.workunit.client.1.smithi100.stdout:7/802: write d5/d14/d16/d44/fc9 [3007172,10677] 0 2024-03-18T14:51:13.564 INFO:tasks.workunit.client.1.smithi100.stdout:6/716: dread d6/d7/df/d72/d48/d76/fad [0,4194304] 0 2024-03-18T14:51:13.564 INFO:tasks.workunit.client.1.smithi100.stdout:6/717: fsync d6/d7/df/d26/d4e/d66/d36/fa0 0 2024-03-18T14:51:13.565 INFO:tasks.workunit.client.1.smithi100.stdout:9/862: read da/d88/fff [508650,57738] 0 2024-03-18T14:51:13.565 INFO:tasks.workunit.client.1.smithi100.stdout:0/833: symlink dd/d31/d91/d56/de0/l130 0 2024-03-18T14:51:13.566 INFO:tasks.workunit.client.1.smithi100.stdout:0/834: write dd/fa6 [600960,57878] 0 2024-03-18T14:51:13.574 INFO:tasks.workunit.client.1.smithi100.stdout:7/803: creat d5/d22/db3/f12a x:0 0 0 2024-03-18T14:51:13.575 INFO:tasks.workunit.client.1.smithi100.stdout:6/718: mkdir d6/d7/df/d26/d4e/db5/de1 0 2024-03-18T14:51:13.579 INFO:tasks.workunit.client.1.smithi100.stdout:7/804: symlink d5/d30/d4d/db4/l12b 0 2024-03-18T14:51:13.579 INFO:tasks.workunit.client.1.smithi100.stdout:7/805: dread - d5/d22/d24/d6e/d105/f109 zero size 2024-03-18T14:51:13.579 INFO:tasks.workunit.client.1.smithi100.stdout:6/719: symlink d6/d12/d2c/le2 0 2024-03-18T14:51:13.582 INFO:tasks.workunit.client.1.smithi100.stdout:7/806: getdents d5/d22/d3b/d9b 0 2024-03-18T14:51:13.582 INFO:tasks.workunit.client.1.smithi100.stdout:7/807: write d5/d22/d3b/d9f/da0/db1/dd3/ff1 [544124,81179] 0 2024-03-18T14:51:13.582 INFO:tasks.workunit.client.1.smithi100.stdout:7/808: stat d5/d14/d16/d63/d124/d66/d4f/dcb 0 2024-03-18T14:51:13.582 INFO:tasks.workunit.client.1.smithi100.stdout:7/809: creat d5/d14/d16/d63/d124/dc3/f12c x:0 0 0 2024-03-18T14:51:13.588 INFO:tasks.workunit.client.1.smithi100.stdout:9/863: dwrite da/dd/d3b/d103/fe6 [0,4194304] 0 2024-03-18T14:51:13.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/835: dwrite dd/f42 [0,4194304] 0 2024-03-18T14:51:13.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/836: fsync dd/d20/d44/f125 0 2024-03-18T14:51:13.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/837: fsync dd/f1b 0 2024-03-18T14:51:13.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/838: chown dd/d2b/d95/d79/d7d/da9/dac/c116 11 1 2024-03-18T14:51:13.597 INFO:tasks.workunit.client.1.smithi100.stdout:7/810: rmdir d5/d30 39 2024-03-18T14:51:13.597 INFO:tasks.workunit.client.1.smithi100.stdout:7/811: chown d5/d14/d16/d63/d124 16911891 1 2024-03-18T14:51:13.601 INFO:tasks.workunit.client.1.smithi100.stdout:6/720: dwrite d6/d7/df/d72/f85 [0,4194304] 0 2024-03-18T14:51:13.601 INFO:tasks.workunit.client.1.smithi100.stdout:9/864: dwrite da/d51/f113 [0,4194304] 0 2024-03-18T14:51:13.605 INFO:tasks.workunit.client.1.smithi100.stdout:0/839: mkdir dd/d31/d127/d131 0 2024-03-18T14:51:13.608 INFO:tasks.workunit.client.1.smithi100.stdout:6/721: mknod d6/d7/df/d72/d48/d76/ce3 0 2024-03-18T14:51:13.612 INFO:tasks.workunit.client.1.smithi100.stdout:9/865: rmdir da/d46/d4d 39 2024-03-18T14:51:13.612 INFO:tasks.workunit.client.1.smithi100.stdout:9/866: fdatasync da/dd/d29/d71/d72/faa 0 2024-03-18T14:51:13.614 INFO:tasks.workunit.client.1.smithi100.stdout:7/812: dwrite d5/d30/f5d [0,4194304] 0 2024-03-18T14:51:13.615 INFO:tasks.workunit.client.1.smithi100.stdout:7/813: chown d5/d14/l6b 11 1 2024-03-18T14:51:13.615 INFO:tasks.workunit.client.1.smithi100.stdout:7/814: stat d5/f9c 0 2024-03-18T14:51:13.615 INFO:tasks.workunit.client.1.smithi100.stdout:0/840: creat dd/d31/d91/d56/d71/dcc/dcf/d120/d12f/f132 x:0 0 0 2024-03-18T14:51:13.618 INFO:tasks.workunit.client.1.smithi100.stdout:7/815: dread d5/d22/d3b/d47/f49 [0,4194304] 0 2024-03-18T14:51:13.618 INFO:tasks.workunit.client.1.smithi100.stdout:7/816: chown d5/d22/d3b/l101 1 1 2024-03-18T14:51:13.619 INFO:tasks.workunit.client.1.smithi100.stdout:6/722: mkdir d6/d7/df/d26/d4e/d66/d36/d4f/de4 0 2024-03-18T14:51:13.624 INFO:tasks.workunit.client.1.smithi100.stdout:5/737: sync 2024-03-18T14:51:13.624 INFO:tasks.workunit.client.1.smithi100.stdout:2/806: sync 2024-03-18T14:51:13.625 INFO:tasks.workunit.client.1.smithi100.stdout:2/807: truncate d3/da/de/d20/d82/dad/fca 680957 0 2024-03-18T14:51:13.625 INFO:tasks.workunit.client.1.smithi100.stdout:5/738: write d29/d4a/f54 [613930,69761] 0 2024-03-18T14:51:13.625 INFO:tasks.workunit.client.1.smithi100.stdout:5/739: readlink d29/d2d/d39/d83/dc1/dbb/l59 0 2024-03-18T14:51:13.625 INFO:tasks.workunit.client.1.smithi100.stdout:4/809: sync 2024-03-18T14:51:13.625 INFO:tasks.workunit.client.1.smithi100.stdout:1/807: sync 2024-03-18T14:51:13.626 INFO:tasks.workunit.client.1.smithi100.stdout:2/808: read d3/da/de/d6e/fcd [209019,52511] 0 2024-03-18T14:51:13.628 INFO:tasks.workunit.client.1.smithi100.stdout:6/723: read d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa6 [3149828,106634] 0 2024-03-18T14:51:13.628 INFO:tasks.workunit.client.1.smithi100.stdout:6/724: write d6/d7/df/d26/f45 [2248388,22631] 0 2024-03-18T14:51:13.629 INFO:tasks.workunit.client.1.smithi100.stdout:0/841: creat dd/d31/d91/d56/de0/d113/f133 x:0 0 0 2024-03-18T14:51:13.629 INFO:tasks.workunit.client.1.smithi100.stdout:0/842: write dd/d31/d91/d56/d59/f11d [1012474,29752] 0 2024-03-18T14:51:13.632 INFO:tasks.workunit.client.1.smithi100.stdout:1/808: dread d8/d1e/d11/d8a/fe4 [0,4194304] 0 2024-03-18T14:51:13.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/867: rmdir da/dcb 39 2024-03-18T14:51:13.633 INFO:tasks.workunit.client.1.smithi100.stdout:9/868: dread - da/d51/d55/d7e/ff3 zero size 2024-03-18T14:51:13.635 INFO:tasks.workunit.client.1.smithi100.stdout:7/817: dwrite d5/d30/d4d/d5a/f80 [0,4194304] 0 2024-03-18T14:51:13.635 INFO:tasks.workunit.client.1.smithi100.stdout:6/725: creat d6/fe5 x:0 0 0 2024-03-18T14:51:13.636 INFO:tasks.workunit.client.1.smithi100.stdout:2/809: dread d3/d6/d5f/fc6 [0,4194304] 0 2024-03-18T14:51:13.636 INFO:tasks.workunit.client.1.smithi100.stdout:0/843: symlink dd/d31/d91/d56/d62/l134 0 2024-03-18T14:51:13.638 INFO:tasks.workunit.client.1.smithi100.stdout:1/809: mknod d8/d1e/d11/d6d/d59/cff 0 2024-03-18T14:51:13.642 INFO:tasks.workunit.client.1.smithi100.stdout:5/740: write d29/d2d/d39/d83/f8b [1427219,12368] 0 2024-03-18T14:51:13.642 INFO:tasks.workunit.client.1.smithi100.stdout:5/741: write d29/d2d/d8f/fb8 [1437956,28207] 0 2024-03-18T14:51:13.642 INFO:tasks.workunit.client.1.smithi100.stdout:5/742: chown d29/d2d/d39/d71 574 1 2024-03-18T14:51:13.646 INFO:tasks.workunit.client.1.smithi100.stdout:7/818: creat d5/d14/d16/d2e/d126/f12d x:0 0 0 2024-03-18T14:51:13.648 INFO:tasks.workunit.client.1.smithi100.stdout:2/810: mkdir d3/da/db8/d111 0 2024-03-18T14:51:13.649 INFO:tasks.workunit.client.1.smithi100.stdout:1/810: creat d8/d2a/d37/d50/f100 x:0 0 0 2024-03-18T14:51:13.649 INFO:tasks.workunit.client.1.smithi100.stdout:1/811: fdatasync d8/d2a/f41 0 2024-03-18T14:51:13.651 INFO:tasks.workunit.client.1.smithi100.stdout:9/869: getdents da/d2c/d2f 0 2024-03-18T14:51:13.655 INFO:tasks.workunit.client.1.smithi100.stdout:2/811: unlink d3/dc3/fd1 0 2024-03-18T14:51:13.656 INFO:tasks.workunit.client.1.smithi100.stdout:1/812: stat d8/d2a/d37/d50/f5e 0 2024-03-18T14:51:13.656 INFO:tasks.workunit.client.1.smithi100.stdout:5/743: dread d29/d2d/fd4 [0,4194304] 0 2024-03-18T14:51:13.657 INFO:tasks.workunit.client.1.smithi100.stdout:9/870: rmdir da/d46/dd6/df9 0 2024-03-18T14:51:13.658 INFO:tasks.workunit.client.1.smithi100.stdout:2/812: truncate d3/fc 3024661 0 2024-03-18T14:51:13.659 INFO:tasks.workunit.client.1.smithi100.stdout:1/813: creat d8/d2a/d37/d50/d6f/df3/f101 x:0 0 0 2024-03-18T14:51:13.659 INFO:tasks.workunit.client.1.smithi100.stdout:1/814: readlink d8/d1e/l57 0 2024-03-18T14:51:13.661 INFO:tasks.workunit.client.1.smithi100.stdout:5/744: creat d29/d2d/d39/d61/ff3 x:0 0 0 2024-03-18T14:51:13.662 INFO:tasks.workunit.client.1.smithi100.stdout:6/726: dwrite d6/d7/df/f7c [0,4194304] 0 2024-03-18T14:51:13.663 INFO:tasks.workunit.client.1.smithi100.stdout:9/871: link da/dd/d29/f34 da/dd/d3b/f115 0 2024-03-18T14:51:13.664 INFO:tasks.workunit.client.1.smithi100.stdout:0/844: dwrite dd/d20/f27 [0,4194304] 0 2024-03-18T14:51:13.664 INFO:tasks.workunit.client.1.smithi100.stdout:0/845: chown dd/c17 1877 1 2024-03-18T14:51:13.664 INFO:tasks.workunit.client.1.smithi100.stdout:0/846: read - dd/d20/f92 zero size 2024-03-18T14:51:13.664 INFO:tasks.workunit.client.1.smithi100.stdout:0/847: creat dd/d2b/d95/d79/d7d/da9/dac/f135 x:0 0 0 2024-03-18T14:51:13.664 INFO:tasks.workunit.client.1.smithi100.stdout:4/810: dwrite d9/d13/d39/f8f [0,4194304] 0 2024-03-18T14:51:13.666 INFO:tasks.workunit.client.1.smithi100.stdout:7/819: dwrite d5/d22/d24/dac/fb6 [0,4194304] 0 2024-03-18T14:51:13.667 INFO:tasks.workunit.client.1.smithi100.stdout:1/815: symlink d8/d30/d61/l102 0 2024-03-18T14:51:13.667 INFO:tasks.workunit.client.1.smithi100.stdout:1/816: truncate d8/d1e/d11/d8a/fa8 261634 0 2024-03-18T14:51:13.667 INFO:tasks.workunit.client.1.smithi100.stdout:1/817: stat d8/d2a/f47 0 2024-03-18T14:51:13.668 INFO:tasks.workunit.client.1.smithi100.stdout:5/745: mkdir d29/d2d/d8f/df4 0 2024-03-18T14:51:13.673 INFO:tasks.workunit.client.1.smithi100.stdout:6/727: getdents d6/d7/df/d26/d4e/d66/d36/d4f/d71/d7f 0 2024-03-18T14:51:13.675 INFO:tasks.workunit.client.1.smithi100.stdout:9/872: creat da/d51/f116 x:0 0 0 2024-03-18T14:51:13.682 INFO:tasks.workunit.client.1.smithi100.stdout:0/848: dwrite dd/d20/f2c [0,4194304] 0 2024-03-18T14:51:13.682 INFO:tasks.workunit.client.1.smithi100.stdout:2/813: dwrite d3/d6/fa9 [0,4194304] 0 2024-03-18T14:51:13.683 INFO:tasks.workunit.client.1.smithi100.stdout:4/811: unlink d9/d13/d5a/d5b/da0/dd8/fe7 0 2024-03-18T14:51:13.684 INFO:tasks.workunit.client.1.smithi100.stdout:5/746: write d29/d2d/d32/f7c [1563834,120518] 0 2024-03-18T14:51:13.691 INFO:tasks.workunit.client.1.smithi100.stdout:7/820: symlink d5/d22/d24/l12e 0 2024-03-18T14:51:13.692 INFO:tasks.workunit.client.1.smithi100.stdout:0/849: dread dd/d31/d91/d65/fc8 [0,4194304] 0 2024-03-18T14:51:13.693 INFO:tasks.workunit.client.1.smithi100.stdout:0/850: creat dd/d20/d44/f136 x:0 0 0 2024-03-18T14:51:13.694 INFO:tasks.workunit.client.1.smithi100.stdout:1/818: creat d8/d1e/d11/d8a/f103 x:0 0 0 2024-03-18T14:51:13.696 INFO:tasks.workunit.client.1.smithi100.stdout:6/728: unlink d6/d7/df/d26/d4e/d66/d36/fbd 0 2024-03-18T14:51:13.696 INFO:tasks.workunit.client.1.smithi100.stdout:6/729: write d6/d7/df/d26/f45 [1279195,98418] 0 2024-03-18T14:51:13.698 INFO:tasks.workunit.client.1.smithi100.stdout:4/812: dwrite d9/fd7 [0,4194304] 0 2024-03-18T14:51:13.703 INFO:tasks.workunit.client.1.smithi100.stdout:2/814: creat d3/da/de/d20/d82/dd3/f112 x:0 0 0 2024-03-18T14:51:13.703 INFO:tasks.workunit.client.1.smithi100.stdout:2/815: write d3/d6/d29/f35 [2000566,100070] 0 2024-03-18T14:51:13.706 INFO:tasks.workunit.client.1.smithi100.stdout:5/747: link d29/d2d/d39/d71/cb7 d29/d2d/d8f/df4/cf5 0 2024-03-18T14:51:13.712 INFO:tasks.workunit.client.1.smithi100.stdout:7/821: unlink d5/d30/d4d/d5a/f80 0 2024-03-18T14:51:13.717 INFO:tasks.workunit.client.1.smithi100.stdout:0/851: mknod dd/d20/c137 0 2024-03-18T14:51:13.718 INFO:tasks.workunit.client.1.smithi100.stdout:0/852: read - dd/d61/fea zero size 2024-03-18T14:51:13.718 INFO:tasks.workunit.client.1.smithi100.stdout:1/819: unlink d8/l5a 0 2024-03-18T14:51:13.719 INFO:tasks.workunit.client.1.smithi100.stdout:6/730: rmdir d6/d12/d2c 39 2024-03-18T14:51:13.719 INFO:tasks.workunit.client.1.smithi100.stdout:6/731: truncate d6/d7/df/d26/d4e/d66/d36/fde 942912 0 2024-03-18T14:51:13.719 INFO:tasks.workunit.client.1.smithi100.stdout:6/732: truncate d6/d7/df/d72/fd8 358516 0 2024-03-18T14:51:13.719 INFO:tasks.workunit.client.1.smithi100.stdout:6/733: write d6/fe [4397462,53785] 0 2024-03-18T14:51:13.721 INFO:tasks.workunit.client.1.smithi100.stdout:9/873: rmdir da/d46/dd6 39 2024-03-18T14:51:13.724 INFO:tasks.workunit.client.1.smithi100.stdout:9/874: dread da/dd/d29/f5a [0,4194304] 0 2024-03-18T14:51:13.731 INFO:tasks.workunit.client.1.smithi100.stdout:4/813: mkdir d9/d13/d39/d4c/df5 0 2024-03-18T14:51:13.734 INFO:tasks.workunit.client.1.smithi100.stdout:2/816: unlink d3/da/de/d20/dea/d40/d53/ld0 0 2024-03-18T14:51:13.734 INFO:tasks.workunit.client.1.smithi100.stdout:2/817: chown d3/d6/ca2 1836 1 2024-03-18T14:51:13.737 INFO:tasks.workunit.client.1.smithi100.stdout:5/748: creat d29/dd0/ff6 x:0 0 0 2024-03-18T14:51:13.744 INFO:tasks.workunit.client.1.smithi100.stdout:2/818: dread d3/da/f3a [0,4194304] 0 2024-03-18T14:51:13.744 INFO:tasks.workunit.client.1.smithi100.stdout:2/819: chown d3/d6/d29/fda 12534096 1 2024-03-18T14:51:13.744 INFO:tasks.workunit.client.1.smithi100.stdout:2/820: truncate d3/d6/d5f/d6a/fc8 552520 0 2024-03-18T14:51:13.745 INFO:tasks.workunit.client.1.smithi100.stdout:0/853: mknod dd/d61/dcd/dff/c138 0 2024-03-18T14:51:13.745 INFO:tasks.workunit.client.1.smithi100.stdout:1/820: getdents d8/d1e/d11/d27/d3f/daa/dd1 0 2024-03-18T14:51:13.745 INFO:tasks.workunit.client.1.smithi100.stdout:1/821: fsync d8/d2a/f47 0 2024-03-18T14:51:13.747 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:13 smithi100 ceph-mon[37807]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:13.752 INFO:tasks.workunit.client.1.smithi100.stdout:6/734: mknod d6/d12/d78/ce6 0 2024-03-18T14:51:13.753 INFO:tasks.workunit.client.1.smithi100.stdout:9/875: getdents da/dd/d3c 0 2024-03-18T14:51:13.755 INFO:tasks.workunit.client.1.smithi100.stdout:6/735: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [3392866,55717] 0 2024-03-18T14:51:13.755 INFO:tasks.workunit.client.1.smithi100.stdout:5/749: dwrite d29/d2d/f41 [4194304,4194304] 0 2024-03-18T14:51:13.755 INFO:tasks.workunit.client.1.smithi100.stdout:5/750: creat d29/ff7 x:0 0 0 2024-03-18T14:51:13.756 INFO:tasks.workunit.client.1.smithi100.stdout:9/876: dread da/dd/f60 [0,4194304] 0 2024-03-18T14:51:13.762 INFO:tasks.workunit.client.1.smithi100.stdout:7/822: rmdir d5/d22/d24/d39/d10c 0 2024-03-18T14:51:13.762 INFO:tasks.workunit.client.1.smithi100.stdout:7/823: chown d5/d14/d16/d63/d124/dd1/cbe 37611631 1 2024-03-18T14:51:13.763 INFO:tasks.workunit.client.1.smithi100.stdout:2/821: creat d3/da/de/d20/d82/f113 x:0 0 0 2024-03-18T14:51:13.770 INFO:tasks.workunit.client.1.smithi100.stdout:1/822: mkdir d8/d1e/d11/d27/d73/d89/d104 0 2024-03-18T14:51:13.770 INFO:tasks.workunit.client.1.smithi100.stdout:1/823: chown d8/d1e/d11/d8a/ldd 2362300 1 2024-03-18T14:51:13.770 INFO:tasks.workunit.client.1.smithi100.stdout:1/824: write d8/d2a/d37/d50/d6f/d81/d91/fca [75718,22910] 0 2024-03-18T14:51:13.775 INFO:tasks.workunit.client.1.smithi100.stdout:3/829: sync 2024-03-18T14:51:13.777 INFO:tasks.workunit.client.1.smithi100.stdout:8/849: sync 2024-03-18T14:51:13.777 INFO:tasks.workunit.client.1.smithi100.stdout:6/736: mkdir d6/d7/df/d72/d65/d7a/da5/de7 0 2024-03-18T14:51:13.781 INFO:tasks.workunit.client.1.smithi100.stdout:6/737: dread d6/f8f [0,4194304] 0 2024-03-18T14:51:13.782 INFO:tasks.workunit.client.1.smithi100.stdout:5/751: rename d29/d2d/dbe/lcc to d29/d2d/d39/lf8 0 2024-03-18T14:51:13.782 INFO:tasks.workunit.client.1.smithi100.stdout:7/824: dwrite d5/fa2 [0,4194304] 0 2024-03-18T14:51:13.782 INFO:tasks.workunit.client.1.smithi100.stdout:2/822: dwrite d3/da/de/d20/ffd [0,4194304] 0 2024-03-18T14:51:13.782 INFO:tasks.workunit.client.1.smithi100.stdout:2/823: stat d3/da/db8/ceb 0 2024-03-18T14:51:13.784 INFO:tasks.workunit.client.1.smithi100.stdout:9/877: mknod da/dd/d29/d71/df5/c117 0 2024-03-18T14:51:13.784 INFO:tasks.workunit.client.1.smithi100.stdout:9/878: write da/d46/f100 [503159,61839] 0 2024-03-18T14:51:13.784 INFO:tasks.workunit.client.1.smithi100.stdout:9/879: chown da/dd/d29/d9e/f10a 2989733 1 2024-03-18T14:51:13.787 INFO:tasks.workunit.client.1.smithi100.stdout:4/814: rmdir d9/d17 39 2024-03-18T14:51:13.789 INFO:tasks.workunit.client.1.smithi100.stdout:2/824: dread d3/d1a/d6c/f8b [0,4194304] 0 2024-03-18T14:51:13.797 INFO:tasks.workunit.client.1.smithi100.stdout:0/854: rmdir dd/d31/d91/d56/d71 39 2024-03-18T14:51:13.797 INFO:tasks.workunit.client.1.smithi100.stdout:0/855: chown dd/f18 243458978 1 2024-03-18T14:51:13.797 INFO:tasks.workunit.client.1.smithi100.stdout:0/856: read dd/d31/d91/f3f [4755203,107424] 0 2024-03-18T14:51:13.798 INFO:tasks.workunit.client.1.smithi100.stdout:8/850: dwrite d4/dc/dd/d48/f123 [0,4194304] 0 2024-03-18T14:51:13.804 INFO:tasks.workunit.client.1.smithi100.stdout:1/825: link d8/fee d8/d2a/d37/d50/d6f/d71/db7/f105 0 2024-03-18T14:51:13.804 INFO:tasks.workunit.client.1.smithi100.stdout:1/826: write d8/d1e/d11/d2d/ffa [957190,86135] 0 2024-03-18T14:51:13.805 INFO:tasks.workunit.client.1.smithi100.stdout:3/830: symlink d3/dd/d21/d3e/df8/db9/l11d 0 2024-03-18T14:51:13.807 INFO:tasks.workunit.client.1.smithi100.stdout:6/738: mkdir d6/d7/df/d26/d4e/d66/d36/d4f/d8a/de8 0 2024-03-18T14:51:13.809 INFO:tasks.workunit.client.1.smithi100.stdout:5/752: mkdir d29/d2d/d39/d83/dc1/dbb/d9e/dd7/df9 0 2024-03-18T14:51:13.811 INFO:tasks.workunit.client.1.smithi100.stdout:9/880: dwrite da/d51/f116 [0,4194304] 0 2024-03-18T14:51:13.818 INFO:tasks.workunit.client.1.smithi100.stdout:7/825: creat d5/d22/d3b/d9f/da0/db1/f12f x:0 0 0 2024-03-18T14:51:13.821 INFO:tasks.workunit.client.1.smithi100.stdout:5/753: dwrite d29/d2d/d32/d92/fed [0,4194304] 0 2024-03-18T14:51:13.823 INFO:tasks.workunit.client.1.smithi100.stdout:0/857: mknod dd/d31/d91/d56/d62/d6e/c139 0 2024-03-18T14:51:13.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/851: symlink d4/dc/d1f/d33/d128/d21/d23/d25/l12e 0 2024-03-18T14:51:13.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/852: fsync d4/dc/d1f/d33/d128/d21/f72 0 2024-03-18T14:51:13.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/853: fsync d4/dc/d1f/d33/d128/d19/db0/dd5/f104 0 2024-03-18T14:51:13.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/854: dread - d4/dc/d1f/d33/d128/d19/d37/fb6 zero size 2024-03-18T14:51:13.830 INFO:tasks.workunit.client.1.smithi100.stdout:8/855: write f1 [4699128,30207] 0 2024-03-18T14:51:13.831 INFO:tasks.workunit.client.1.smithi100.stdout:8/856: dread - d4/dc/d1f/d59/d6c/f115 zero size 2024-03-18T14:51:13.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/827: unlink d8/d1e/d11/d27/fa7 0 2024-03-18T14:51:13.834 INFO:tasks.workunit.client.1.smithi100.stdout:1/828: creat d8/d30/d61/d8d/f106 x:0 0 0 2024-03-18T14:51:13.841 INFO:tasks.workunit.client.1.smithi100.stdout:6/739: rmdir d6/d12/d2c 39 2024-03-18T14:51:13.841 INFO:tasks.workunit.client.1.smithi100.stdout:6/740: write d6/d7/df/d72/d65/fa9 [883359,3501] 0 2024-03-18T14:51:13.841 INFO:tasks.workunit.client.1.smithi100.stdout:6/741: readlink d6/d7/df/d26/l81 0 2024-03-18T14:51:13.841 INFO:tasks.workunit.client.1.smithi100.stdout:6/742: fdatasync d6/d7/df/d26/d4e/d66/d36/d4f/db8/fe0 0 2024-03-18T14:51:13.846 INFO:tasks.workunit.client.1.smithi100.stdout:9/881: creat da/dd/d29/d71/d72/f118 x:0 0 0 2024-03-18T14:51:13.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/826: creat d5/d22/d24/d6e/f130 x:0 0 0 2024-03-18T14:51:13.847 INFO:tasks.workunit.client.1.smithi100.stdout:8/857: dwrite d4/dc/d1f/d33/d128/d19/d37/f65 [0,4194304] 0 2024-03-18T14:51:13.849 INFO:tasks.workunit.client.1.smithi100.stdout:2/825: getdents d3/d6 0 2024-03-18T14:51:13.854 INFO:tasks.workunit.client.1.smithi100.stdout:0/858: unlink dd/d2b/d95/d79/d7d/fa8 0 2024-03-18T14:51:13.855 INFO:tasks.workunit.client.1.smithi100.stdout:6/743: write d6/d7/f8 [866734,36128] 0 2024-03-18T14:51:13.855 INFO:tasks.workunit.client.1.smithi100.stdout:6/744: write d6/d12/d78/fb4 [592941,37206] 0 2024-03-18T14:51:13.856 INFO:tasks.workunit.client.1.smithi100.stdout:8/858: dread d4/dc/d1f/d33/d128/d21/d23/d47/f5b [4194304,4194304] 0 2024-03-18T14:51:13.861 INFO:tasks.workunit.client.1.smithi100.stdout:1/829: mknod d8/d30/d61/d8d/c107 0 2024-03-18T14:51:13.863 INFO:tasks.workunit.client.1.smithi100.stdout:3/831: rename d3/dd/c6d to d3/dd/d21/c11e 0 2024-03-18T14:51:13.864 INFO:tasks.workunit.client.1.smithi100.stdout:6/745: read d6/d7/df/d26/f56 [1504181,105238] 0 2024-03-18T14:51:13.864 INFO:tasks.workunit.client.1.smithi100.stdout:2/826: dwrite d3/da/de/d20/ffd [0,4194304] 0 2024-03-18T14:51:13.864 INFO:tasks.workunit.client.1.smithi100.stdout:9/882: mkdir da/dd/d29/d71/d72/db2/d119 0 2024-03-18T14:51:13.867 INFO:tasks.workunit.client.1.smithi100.stdout:7/827: creat d5/d14/d16/d63/d124/dd1/d86/de6/d122/f131 x:0 0 0 2024-03-18T14:51:13.869 INFO:tasks.workunit.client.1.smithi100.stdout:0/859: unlink dd/d31/d91/d65/c12e 0 2024-03-18T14:51:13.874 INFO:tasks.workunit.client.1.smithi100.stdout:3/832: write d3/d16/d13/d17/f7f [2467229,66701] 0 2024-03-18T14:51:13.874 INFO:tasks.workunit.client.1.smithi100.stdout:8/859: link d4/dc/dd/l13 d4/dc/d97/l12f 0 2024-03-18T14:51:13.879 INFO:tasks.workunit.client.1.smithi100.stdout:3/833: write d3/d16/d13/d17/d7d/f82 [2673562,103659] 0 2024-03-18T14:51:13.894 INFO:tasks.workunit.client.1.smithi100.stdout:5/754: rename d29/d2d/d39/d83/f85 to d29/d2d/d39/d83/dc1/dbb/d9e/dd7/dee/ffa 0 2024-03-18T14:51:13.896 INFO:tasks.workunit.client.1.smithi100.stdout:9/883: dwrite da/dd/d3b/d103/fd5 [0,4194304] 0 2024-03-18T14:51:13.896 INFO:tasks.workunit.client.1.smithi100.stdout:7/828: dwrite d5/d14/d16/d44/fa7 [0,4194304] 0 2024-03-18T14:51:13.896 INFO:tasks.workunit.client.1.smithi100.stdout:0/860: dwrite dd/d20/d44/f125 [0,4194304] 0 2024-03-18T14:51:13.900 INFO:tasks.workunit.client.1.smithi100.stdout:8/860: symlink d4/dc/d31/l130 0 2024-03-18T14:51:13.903 INFO:tasks.workunit.client.1.smithi100.stdout:3/834: creat d3/dd/d21/d3e/df8/df2/f11f x:0 0 0 2024-03-18T14:51:13.905 INFO:tasks.workunit.client.1.smithi100.stdout:0/861: dread dd/d20/d44/f125 [0,4194304] 0 2024-03-18T14:51:13.909 INFO:tasks.workunit.client.1.smithi100.stdout:1/830: rename d8/d2a/d37/la6 to d8/d30/d61/db4/l108 0 2024-03-18T14:51:13.918 INFO:tasks.workunit.client.1.smithi100.stdout:9/884: mknod da/dd/d3e/c11a 0 2024-03-18T14:51:13.919 INFO:tasks.workunit.client.1.smithi100.stdout:9/885: write da/dd/d29/f2e [1525661,19897] 0 2024-03-18T14:51:13.926 INFO:tasks.workunit.client.1.smithi100.stdout:7/829: mkdir d5/d22/d24/d6e/d7a/dc2/d111/d132 0 2024-03-18T14:51:13.934 INFO:tasks.workunit.client.1.smithi100.stdout:2/827: getdents d3/da/de/d20/d82/dd3 0 2024-03-18T14:51:13.936 INFO:tasks.workunit.client.1.smithi100.stdout:8/861: unlink d4/dc/dd/d48/fa6 0 2024-03-18T14:51:13.937 INFO:tasks.workunit.client.1.smithi100.stdout:3/835: creat d3/d16/f120 x:0 0 0 2024-03-18T14:51:13.939 INFO:tasks.workunit.client.1.smithi100.stdout:0/862: mkdir dd/d31/d91/d56/d71/dcc/dcf/d120/d13a 0 2024-03-18T14:51:13.940 INFO:tasks.workunit.client.1.smithi100.stdout:1/831: mkdir d8/d30/d109 0 2024-03-18T14:51:13.940 INFO:tasks.workunit.client.1.smithi100.stdout:1/832: fdatasync d8/d2a/fa9 0 2024-03-18T14:51:13.940 INFO:tasks.workunit.client.1.smithi100.stdout:1/833: truncate d8/d1e/d11/d27/d3f/fe8 1503812 0 2024-03-18T14:51:13.940 INFO:tasks.workunit.client.1.smithi100.stdout:1/834: dread - d8/d30/d61/d8d/f92 zero size 2024-03-18T14:51:13.941 INFO:tasks.workunit.client.1.smithi100.stdout:8/862: dread f1 [4194304,4194304] 0 2024-03-18T14:51:13.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/830: creat d5/d30/d4d/db4/d116/f133 x:0 0 0 2024-03-18T14:51:13.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/831: readlink d5/d30/d4d/db4/le9 0 2024-03-18T14:51:13.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/832: creat d5/d22/d24/dac/f134 x:0 0 0 2024-03-18T14:51:13.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/833: stat d5/d22/d3b/d9f 0 2024-03-18T14:51:13.942 INFO:tasks.workunit.client.1.smithi100.stdout:7/834: chown d5/d14/d16/d63/cb8 28 1 2024-03-18T14:51:13.943 INFO:tasks.workunit.client.1.smithi100.stdout:7/835: fdatasync d5/d22/d24/fcd 0 2024-03-18T14:51:13.943 INFO:tasks.workunit.client.1.smithi100.stdout:2/828: creat d3/d6/d29/d36/ddf/f114 x:0 0 0 2024-03-18T14:51:13.943 INFO:tasks.workunit.client.1.smithi100.stdout:3/836: symlink d3/dd/l121 0 2024-03-18T14:51:13.943 INFO:tasks.workunit.client.1.smithi100.stdout:3/837: write d3/d16/d13/f39 [161301,68064] 0 2024-03-18T14:51:13.943 INFO:tasks.workunit.client.1.smithi100.stdout:3/838: write d3/dd/d49/f5d [1064486,61345] 0 2024-03-18T14:51:13.945 INFO:tasks.workunit.client.1.smithi100.stdout:1/835: getdents d8/d1e/d11/d27/d3f/daa/dd5 0 2024-03-18T14:51:13.945 INFO:tasks.workunit.client.1.smithi100.stdout:1/836: truncate d8/d30/d61/d8d/f92 696646 0 2024-03-18T14:51:13.945 INFO:tasks.workunit.client.1.smithi100.stdout:8/863: chown d4/dc/d1f/d33/d128/d19/c45 1908637139 1 2024-03-18T14:51:13.945 INFO:tasks.workunit.client.1.smithi100.stdout:8/864: write d4/dc/d31/d85/d8e/f100 [987504,72644] 0 2024-03-18T14:51:13.950 INFO:tasks.workunit.client.1.smithi100.stdout:7/836: getdents d5/d14/d16/d63 0 2024-03-18T14:51:13.950 INFO:tasks.workunit.client.1.smithi100.stdout:7/837: truncate d5/d22/d24/d6e/d7a/dc2/fee 4781036 0 2024-03-18T14:51:13.950 INFO:tasks.workunit.client.1.smithi100.stdout:7/838: dread - d5/d14/d16/d63/d124/ffd zero size 2024-03-18T14:51:13.951 INFO:tasks.workunit.client.1.smithi100.stdout:9/886: dwrite da/dd/d3b/f115 [0,4194304] 0 2024-03-18T14:51:13.951 INFO:tasks.workunit.client.1.smithi100.stdout:9/887: creat da/d46/f11b x:0 0 0 2024-03-18T14:51:13.951 INFO:tasks.workunit.client.1.smithi100.stdout:9/888: stat da/d2c/d70/l86 0 2024-03-18T14:51:13.951 INFO:tasks.workunit.client.1.smithi100.stdout:9/889: chown da/dd/d3b/d103/cbf 13 1 2024-03-18T14:51:13.951 INFO:tasks.workunit.client.1.smithi100.stdout:2/829: truncate d3/fc 278493 0 2024-03-18T14:51:13.952 INFO:tasks.workunit.client.1.smithi100.stdout:0/863: rename dd/d2b/d95/d79/d7d to dd/d31/d91/d13b 0 2024-03-18T14:51:13.954 INFO:tasks.workunit.client.1.smithi100.stdout:8/865: rmdir d4/d3c 39 2024-03-18T14:51:13.956 INFO:tasks.workunit.client.1.smithi100.stdout:7/839: truncate d5/d22/d3b/d9f/da0/fc6 254270 0 2024-03-18T14:51:13.956 INFO:tasks.workunit.client.1.smithi100.stdout:7/840: readlink d5/d22/d24/l5f 0 2024-03-18T14:51:13.957 INFO:tasks.workunit.client.1.smithi100.stdout:3/839: dread d3/dd/f38 [0,4194304] 0 2024-03-18T14:51:13.957 INFO:tasks.workunit.client.1.smithi100.stdout:5/755: dwrite d29/d2d/d39/f56 [0,4194304] 0 2024-03-18T14:51:13.957 INFO:tasks.workunit.client.1.smithi100.stdout:3/840: creat d3/da5/f122 x:0 0 0 2024-03-18T14:51:13.957 INFO:tasks.workunit.client.1.smithi100.stdout:9/890: mknod da/dc3/d104/c11c 0 2024-03-18T14:51:13.957 INFO:tasks.workunit.client.1.smithi100.stdout:9/891: creat da/dd/d29/d71/d72/db2/f11d x:0 0 0 2024-03-18T14:51:13.958 INFO:tasks.workunit.client.1.smithi100.stdout:0/864: dread dd/d20/d44/f9e [0,4194304] 0 2024-03-18T14:51:13.958 INFO:tasks.workunit.client.1.smithi100.stdout:0/865: chown dd/le9 660 1 2024-03-18T14:51:13.959 INFO:tasks.workunit.client.1.smithi100.stdout:2/830: truncate d3/d6/d29/d36/f9e 5544589 0 2024-03-18T14:51:13.960 INFO:tasks.workunit.client.1.smithi100.stdout:8/866: creat d4/dc/d1f/d33/d128/d21/f131 x:0 0 0 2024-03-18T14:51:13.960 INFO:tasks.workunit.client.1.smithi100.stdout:1/837: write d8/d1e/d11/d27/d73/d89/fa4 [7398135,121461] 0 2024-03-18T14:51:13.960 INFO:tasks.workunit.client.1.smithi100.stdout:5/756: dread d29/d2d/d39/d83/dc1/dbb/f76 [0,4194304] 0 2024-03-18T14:51:13.960 INFO:tasks.workunit.client.1.smithi100.stdout:5/757: chown d29/dd0 654322289 1 2024-03-18T14:51:13.960 INFO:tasks.workunit.client.1.smithi100.stdout:5/758: read - d29/d2d/d39/d83/dc1/dbb/f6e zero size 2024-03-18T14:51:13.961 INFO:tasks.workunit.client.1.smithi100.stdout:8/867: dread d4/dc/d31/d85/fb7 [0,4194304] 0 2024-03-18T14:51:13.962 INFO:tasks.workunit.client.1.smithi100.stdout:7/841: link d5/f52 d5/d22/d24/d6e/d7a/dc2/d111/f135 0 2024-03-18T14:51:13.962 INFO:tasks.workunit.client.1.smithi100.stdout:7/842: readlink d5/d14/d16/d2e/l3d 0 2024-03-18T14:51:13.962 INFO:tasks.workunit.client.1.smithi100.stdout:7/843: stat d5/d14/d16/f1d 0 2024-03-18T14:51:13.986 INFO:tasks.workunit.client.1.smithi100.stdout:3/841: truncate d3/d16/d13/d1f/f97 3418097 0 2024-03-18T14:51:13.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/842: write d3/dd/d21/d3e/df8/d74/fb5 [520670,24047] 0 2024-03-18T14:51:13.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/843: truncate d3/dd/d49/f9f 814384 0 2024-03-18T14:51:13.987 INFO:tasks.workunit.client.1.smithi100.stdout:3/844: fdatasync d3/fd5 0 2024-03-18T14:51:13.990 INFO:tasks.workunit.client.1.smithi100.stdout:0/866: creat dd/d2b/d11e/f13c x:0 0 0 2024-03-18T14:51:13.990 INFO:tasks.workunit.client.1.smithi100.stdout:0/867: truncate dd/d31/d91/d56/d59/ff2 835798 0 2024-03-18T14:51:13.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:13 smithi012 ceph-mon[29182]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:13.992 INFO:tasks.workunit.client.1.smithi100.stdout:5/759: symlink d29/d2d/d39/d83/dc1/dbb/lfb 0 2024-03-18T14:51:13.993 INFO:tasks.workunit.client.1.smithi100.stdout:8/868: mknod d4/dc/d1f/d59/c132 0 2024-03-18T14:51:13.995 INFO:tasks.workunit.client.1.smithi100.stdout:4/815: sync 2024-03-18T14:51:13.995 INFO:tasks.workunit.client.1.smithi100.stdout:6/746: sync 2024-03-18T14:51:13.995 INFO:tasks.workunit.client.1.smithi100.stdout:6/747: chown d6/d7/df/d26/d4e/db5/dba/ld6 29295 1 2024-03-18T14:51:13.998 INFO:tasks.workunit.client.1.smithi100.stdout:8/869: getdents d4/dc 0 2024-03-18T14:51:14.000 INFO:tasks.workunit.client.1.smithi100.stdout:2/831: dwrite d3/da/de/d20/f24 [0,4194304] 0 2024-03-18T14:51:14.000 INFO:tasks.workunit.client.1.smithi100.stdout:3/845: symlink d3/l123 0 2024-03-18T14:51:14.000 INFO:tasks.workunit.client.1.smithi100.stdout:7/844: dwrite d5/d22/d3b/d9f/da0/db1/fda [0,4194304] 0 2024-03-18T14:51:14.001 INFO:tasks.workunit.client.1.smithi100.stdout:7/845: write d5/d22/d24/d6e/f130 [401620,127573] 0 2024-03-18T14:51:14.001 INFO:tasks.workunit.client.1.smithi100.stdout:7/846: chown d5/d30/d4d/d5a 18971 1 2024-03-18T14:51:14.002 INFO:tasks.workunit.client.1.smithi100.stdout:4/816: rmdir d9/d13/d5a/d5b 39 2024-03-18T14:51:14.002 INFO:tasks.workunit.client.1.smithi100.stdout:4/817: dread - d9/d13/d5a/d5b/da0/dd8/fed zero size 2024-03-18T14:51:14.003 INFO:tasks.workunit.client.1.smithi100.stdout:4/818: dread d9/d13/d5a/da4/f96 [0,4194304] 0 2024-03-18T14:51:14.005 INFO:tasks.workunit.client.1.smithi100.stdout:6/748: creat d6/d9f/fe9 x:0 0 0 2024-03-18T14:51:14.007 INFO:tasks.workunit.client.1.smithi100.stdout:9/892: rename da/d46 to da/dd/d3e/d11e 0 2024-03-18T14:51:14.009 INFO:tasks.workunit.client.1.smithi100.stdout:8/870: creat d4/dc/d1f/d33/d128/d19/f133 x:0 0 0 2024-03-18T14:51:14.009 INFO:tasks.workunit.client.1.smithi100.stdout:8/871: write d4/f60 [550076,39125] 0 2024-03-18T14:51:14.011 INFO:tasks.workunit.client.1.smithi100.stdout:2/832: unlink d3/lb0 0 2024-03-18T14:51:14.014 INFO:tasks.workunit.client.1.smithi100.stdout:3/846: dwrite d3/d16/d13/d1f/fda [0,4194304] 0 2024-03-18T14:51:14.015 INFO:tasks.workunit.client.1.smithi100.stdout:3/847: write d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb [242024,75706] 0 2024-03-18T14:51:14.015 INFO:tasks.workunit.client.1.smithi100.stdout:7/847: creat d5/d90/da1/d128/f136 x:0 0 0 2024-03-18T14:51:14.015 INFO:tasks.workunit.client.1.smithi100.stdout:7/848: chown d5/d14/d16 958207 1 2024-03-18T14:51:14.017 INFO:tasks.workunit.client.1.smithi100.stdout:6/749: creat d6/d7/df/d72/d8b/fea x:0 0 0 2024-03-18T14:51:14.019 INFO:tasks.workunit.client.1.smithi100.stdout:0/868: rename dd/d2b/f89 to dd/d31/d91/d13b/da9/dac/f13d 0 2024-03-18T14:51:14.020 INFO:tasks.workunit.client.1.smithi100.stdout:9/893: symlink da/dd/d3e/d11e/dd6/de0/l11f 0 2024-03-18T14:51:14.022 INFO:tasks.workunit.client.1.smithi100.stdout:8/872: creat d4/dc/d1f/d33/d128/d21/d23/d25/f134 x:0 0 0 2024-03-18T14:51:14.023 INFO:tasks.workunit.client.1.smithi100.stdout:2/833: creat d3/da/de/d6e/f115 x:0 0 0 2024-03-18T14:51:14.030 INFO:tasks.workunit.client.1.smithi100.stdout:9/894: write da/dd/d3e/d11e/d8b/f35 [3191032,2073] 0 2024-03-18T14:51:14.030 INFO:tasks.workunit.client.1.smithi100.stdout:3/848: mknod d3/dd/d21/dae/c124 0 2024-03-18T14:51:14.030 INFO:tasks.workunit.client.1.smithi100.stdout:3/849: getdents d3/dd/d10d 0 2024-03-18T14:51:14.031 INFO:tasks.workunit.client.1.smithi100.stdout:6/750: mknod d6/d7/df/d72/d65/d7a/da5/ceb 0 2024-03-18T14:51:14.034 INFO:tasks.workunit.client.1.smithi100.stdout:7/849: dwrite d5/d14/d16/d63/d124/dd1/f5b [0,4194304] 0 2024-03-18T14:51:14.034 INFO:tasks.workunit.client.1.smithi100.stdout:7/850: write d5/f9c [1012152,63381] 0 2024-03-18T14:51:14.034 INFO:tasks.workunit.client.1.smithi100.stdout:7/851: dread - d5/d22/d3b/d9f/da0/db1/dd3/fe0 zero size 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:7/852: dread - d5/d14/d16/d63/d124/dd1/d86/d85/d87/df6/fa6 zero size 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:7/853: write d5/d14/d16/d63/d124/dc3/f12c [264593,105032] 0 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:5/760: rename d29/d2d/d39/d83/dc1/dbb/f5f to d29/d2d/d39/d61/ddd/ffc 0 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/869: rmdir dd/d31/d91/d13b/da9 39 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/870: chown dd/d31/d91/d56/lb4 7295755 1 2024-03-18T14:51:14.035 INFO:tasks.workunit.client.1.smithi100.stdout:0/871: dread - dd/d2b/d11e/f13c zero size 2024-03-18T14:51:14.037 INFO:tasks.workunit.client.1.smithi100.stdout:2/834: unlink d3/d6/ca2 0 2024-03-18T14:51:14.040 INFO:tasks.workunit.client.1.smithi100.stdout:5/761: dread d29/d2d/d39/d61/f6b [0,4194304] 0 2024-03-18T14:51:14.043 INFO:tasks.workunit.client.1.smithi100.stdout:3/850: creat d3/dd/d10d/f125 x:0 0 0 2024-03-18T14:51:14.045 INFO:tasks.workunit.client.1.smithi100.stdout:6/751: creat d6/d12/d2c/fec x:0 0 0 2024-03-18T14:51:14.051 INFO:tasks.workunit.client.1.smithi100.stdout:7/854: rmdir d5/d22/d24/d6e/d7a/dc2/d111/d132 0 2024-03-18T14:51:14.056 INFO:tasks.workunit.client.1.smithi100.stdout:8/873: rename d4/dc/d1f/d59/d6c/ddb to d4/dc/d1f/d33/d128/d19/dc0/d135 0 2024-03-18T14:51:14.056 INFO:tasks.workunit.client.1.smithi100.stdout:8/874: dread - d4/dc/d31/d85/dd6/df9/f116 zero size 2024-03-18T14:51:14.061 INFO:tasks.workunit.client.1.smithi100.stdout:0/872: link dd/d31/d91/d56/d62/f84 dd/d31/d91/d56/d59/f13e 0 2024-03-18T14:51:14.061 INFO:tasks.workunit.client.1.smithi100.stdout:6/752: dwrite d6/d9f/fca [0,4194304] 0 2024-03-18T14:51:14.066 INFO:tasks.workunit.client.1.smithi100.stdout:2/835: mknod d3/d6/d29/c116 0 2024-03-18T14:51:14.068 INFO:tasks.workunit.client.1.smithi100.stdout:3/851: symlink d3/d16/d13/d1f/d3f/l126 0 2024-03-18T14:51:14.071 INFO:tasks.workunit.client.1.smithi100.stdout:8/875: rename d4/dc/dc6 to d4/dc/d1f/d33/d128/d21/d23/d47/d54/d136 0 2024-03-18T14:51:14.071 INFO:tasks.workunit.client.1.smithi100.stdout:6/753: dread d6/d7/f8 [0,4194304] 0 2024-03-18T14:51:14.073 INFO:tasks.workunit.client.1.smithi100.stdout:0/873: dread dd/d20/f5f [0,4194304] 0 2024-03-18T14:51:14.074 INFO:tasks.workunit.client.1.smithi100.stdout:0/874: fdatasync dd/d61/fea 0 2024-03-18T14:51:14.074 INFO:tasks.workunit.client.1.smithi100.stdout:7/855: dwrite d5/d14/d16/d63/d124/dd1/d86/f7f [0,4194304] 0 2024-03-18T14:51:14.075 INFO:tasks.workunit.client.1.smithi100.stdout:5/762: fsync d29/d2d/d39/d61/ddd/ffc 0 2024-03-18T14:51:14.084 INFO:tasks.workunit.client.1.smithi100.stdout:3/852: mkdir d3/dd/d21/dae/d127 0 2024-03-18T14:51:14.086 INFO:tasks.workunit.client.1.smithi100.stdout:8/876: creat d4/dc/d1f/d33/d128/d19/d37/f137 x:0 0 0 2024-03-18T14:51:14.086 INFO:tasks.workunit.client.1.smithi100.stdout:8/877: chown d4/dc/d1f/d33/d128/d19/d2c/da2 172175 1 2024-03-18T14:51:14.086 INFO:tasks.workunit.client.1.smithi100.stdout:8/878: chown d4/dc/d1f/d33/c52 1954 1 2024-03-18T14:51:14.089 INFO:tasks.workunit.client.1.smithi100.stdout:6/754: creat d6/d7/df/fed x:0 0 0 2024-03-18T14:51:14.092 INFO:tasks.workunit.client.1.smithi100.stdout:0/875: dwrite dd/d20/f92 [0,4194304] 0 2024-03-18T14:51:14.092 INFO:tasks.workunit.client.1.smithi100.stdout:5/763: dwrite d29/d2d/d32/f7c [4194304,4194304] 0 2024-03-18T14:51:14.098 INFO:tasks.workunit.client.1.smithi100.stdout:7/856: unlink d5/d14/d16/d63/d124/dc3/ldd 0 2024-03-18T14:51:14.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/879: link d4/d3c/c5e d4/d3c/c138 0 2024-03-18T14:51:14.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/880: dread - d4/dc/d1f/d59/fb1 zero size 2024-03-18T14:51:14.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/881: dread - d4/dc/d1f/d33/d128/d21/d23/d47/d56/ffd zero size 2024-03-18T14:51:14.099 INFO:tasks.workunit.client.1.smithi100.stdout:8/882: dread - d4/dc/d31/d85/dd6/df9/f116 zero size 2024-03-18T14:51:14.102 INFO:tasks.workunit.client.1.smithi100.stdout:6/755: symlink d6/d12/d2c/lee 0 2024-03-18T14:51:14.102 INFO:tasks.workunit.client.1.smithi100.stdout:6/756: chown d6/d7/df/d26/l28 12982824 1 2024-03-18T14:51:14.102 INFO:tasks.workunit.client.1.smithi100.stdout:6/757: fsync d6/d12/d2c/f7e 0 2024-03-18T14:51:14.109 INFO:tasks.workunit.client.1.smithi100.stdout:0/876: symlink dd/d31/d127/l13f 0 2024-03-18T14:51:14.110 INFO:tasks.workunit.client.1.smithi100.stdout:5/764: unlink d29/d2d/d39/d65/d7d/fbd 0 2024-03-18T14:51:14.110 INFO:tasks.workunit.client.1.smithi100.stdout:5/765: write d29/f2b [1622529,80812] 0 2024-03-18T14:51:14.110 INFO:tasks.workunit.client.1.smithi100.stdout:5/766: chown d29/d2d/f6f 28 1 2024-03-18T14:51:14.112 INFO:tasks.workunit.client.1.smithi100.stdout:7/857: symlink d5/d30/l137 0 2024-03-18T14:51:14.115 INFO:tasks.workunit.client.1.smithi100.stdout:8/883: rename d4/dc/d1f/d33/d128/d21/d23/d25/f71 to d4/d5a/f139 0 2024-03-18T14:51:14.115 INFO:tasks.workunit.client.1.smithi100.stdout:8/884: read - d4/dc/d1f/d59/fbd zero size 2024-03-18T14:51:14.115 INFO:tasks.workunit.client.1.smithi100.stdout:8/885: read - d4/dc/d1f/d33/d128/d19/d2c/da2/f12b zero size 2024-03-18T14:51:14.117 INFO:tasks.workunit.client.1.smithi100.stdout:5/767: dread d29/f91 [0,4194304] 0 2024-03-18T14:51:14.123 INFO:tasks.workunit.client.1.smithi100.stdout:0/877: mkdir dd/d61/dcd/d140 0 2024-03-18T14:51:14.127 INFO:tasks.workunit.client.1.smithi100.stdout:7/858: symlink d5/d22/d3b/d9f/da0/db1/dd3/l138 0 2024-03-18T14:51:14.131 INFO:tasks.workunit.client.1.smithi100.stdout:8/886: link d4/dc/d31/d66/fc3 d4/dc/d97/f13a 0 2024-03-18T14:51:14.132 INFO:tasks.workunit.client.1.smithi100.stdout:6/758: getdents d6/d9f 0 2024-03-18T14:51:14.139 INFO:tasks.workunit.client.1.smithi100.stdout:0/878: mkdir dd/d31/d91/d7b/d11c/d141 0 2024-03-18T14:51:14.141 INFO:tasks.workunit.client.1.smithi100.stdout:6/759: getdents d6/d12/da2 0 2024-03-18T14:51:14.141 INFO:tasks.workunit.client.1.smithi100.stdout:6/760: stat d6/d7/df/d72/f85 0 2024-03-18T14:51:14.142 INFO:tasks.workunit.client.1.smithi100.stdout:0/879: unlink dd/c17 0 2024-03-18T14:51:14.144 INFO:tasks.workunit.client.1.smithi100.stdout:6/761: link d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/c46 d6/d7/df/d26/d4e/d66/d36/d4f/cef 0 2024-03-18T14:51:14.163 INFO:tasks.workunit.client.1.smithi100.stdout:8/887: dwrite d4/dc/dd/f44 [0,4194304] 0 2024-03-18T14:51:14.164 INFO:tasks.workunit.client.1.smithi100.stdout:8/888: creat d4/dc/d1f/d33/d128/d19/d2c/f13b x:0 0 0 2024-03-18T14:51:14.165 INFO:tasks.workunit.client.1.smithi100.stdout:8/889: mknod d4/dc/d1f/d33/d128/d19/dc0/d113/c13c 0 2024-03-18T14:51:14.166 INFO:tasks.workunit.client.1.smithi100.stdout:8/890: mknod d4/dc/dd/d48/dd8/c13d 0 2024-03-18T14:51:14.167 INFO:tasks.workunit.client.1.smithi100.stdout:7/859: dwrite d5/d22/d3b/d9f/da0/db1/dd3/ff1 [0,4194304] 0 2024-03-18T14:51:14.168 INFO:tasks.workunit.client.1.smithi100.stdout:8/891: rename d4/dc2/cf4 to d4/dc/d1f/d33/d128/d19/c13e 0 2024-03-18T14:51:14.172 INFO:tasks.workunit.client.1.smithi100.stdout:8/892: rename d4/dc/d1f/d33/d128/d21/f131 to d4/dc/d1f/d33/d128/d19/d37/f13f 0 2024-03-18T14:51:14.172 INFO:tasks.workunit.client.1.smithi100.stdout:8/893: write d4/dc/d31/f69 [2363287,58319] 0 2024-03-18T14:51:14.172 INFO:tasks.workunit.client.1.smithi100.stdout:8/894: fdatasync d4/dc/d1f/d59/d6c/d75/fd1 0 2024-03-18T14:51:14.173 INFO:tasks.workunit.client.1.smithi100.stdout:8/895: creat d4/d3c/f140 x:0 0 0 2024-03-18T14:51:14.173 INFO:tasks.workunit.client.1.smithi100.stdout:0/880: dwrite dd/d31/fe6 [0,4194304] 0 2024-03-18T14:51:14.173 INFO:tasks.workunit.client.1.smithi100.stdout:6/762: dwrite d6/d9f/fca [4194304,4194304] 0 2024-03-18T14:51:14.174 INFO:tasks.workunit.client.1.smithi100.stdout:8/896: rmdir d4/dc/d1f/d33/d128/d19/dc0/d135 39 2024-03-18T14:51:14.178 INFO:tasks.workunit.client.1.smithi100.stdout:0/881: mknod dd/d31/d91/d56/d59/c142 0 2024-03-18T14:51:14.178 INFO:tasks.workunit.client.1.smithi100.stdout:0/882: chown dd/d31/d91/d56/d71/dcc/dcf/lb9 12903809 1 2024-03-18T14:51:14.178 INFO:tasks.workunit.client.1.smithi100.stdout:7/860: dread d5/d14/d16/d44/fc9 [0,4194304] 0 2024-03-18T14:51:14.181 INFO:tasks.workunit.client.1.smithi100.stdout:6/763: mknod d6/da1/dd7/cf0 0 2024-03-18T14:51:14.190 INFO:tasks.workunit.client.1.smithi100.stdout:8/897: symlink d4/dc/d1f/d33/d128/d19/db0/dd5/l141 0 2024-03-18T14:51:14.190 INFO:tasks.workunit.client.1.smithi100.stdout:8/898: creat d4/dc/dd/d48/f142 x:0 0 0 2024-03-18T14:51:14.193 INFO:tasks.workunit.client.1.smithi100.stdout:7/861: dwrite d5/d22/d24/dac/fb6 [0,4194304] 0 2024-03-18T14:51:14.193 INFO:tasks.workunit.client.1.smithi100.stdout:7/862: chown d5/d14/d16/d63/cb8 64 1 2024-03-18T14:51:14.194 INFO:tasks.workunit.client.1.smithi100.stdout:0/883: mkdir dd/d31/d91/d65/d143 0 2024-03-18T14:51:14.200 INFO:tasks.workunit.client.1.smithi100.stdout:7/863: rename d5/d14/d16/d63/d124/dd1/d86/d85/d87/df6/f73 to d5/d14/d16/d63/d124/d66/f139 0 2024-03-18T14:51:14.202 INFO:tasks.workunit.client.1.smithi100.stdout:0/884: unlink dd/d31/d91/d56/de0/l130 0 2024-03-18T14:51:14.205 INFO:tasks.workunit.client.1.smithi100.stdout:6/764: truncate d6/d7/df/d72/f33 996081 0 2024-03-18T14:51:14.205 INFO:tasks.workunit.client.1.smithi100.stdout:7/864: symlink d5/d22/d3b/d9f/da0/db1/l13a 0 2024-03-18T14:51:14.207 INFO:tasks.workunit.client.1.smithi100.stdout:8/899: dwrite d4/dc/d31/d85/d8e/ff6 [0,4194304] 0 2024-03-18T14:51:14.207 INFO:tasks.workunit.client.1.smithi100.stdout:0/885: symlink dd/d2b/d95/d79/l144 0 2024-03-18T14:51:14.211 INFO:tasks.workunit.client.1.smithi100.stdout:6/765: mkdir d6/d7/df/df1 0 2024-03-18T14:51:14.211 INFO:tasks.workunit.client.1.smithi100.stdout:6/766: fsync d6/d7/df/f2a 0 2024-03-18T14:51:14.212 INFO:tasks.workunit.client.1.smithi100.stdout:1/838: sync 2024-03-18T14:51:14.212 INFO:tasks.workunit.client.1.smithi100.stdout:4/819: sync 2024-03-18T14:51:14.212 INFO:tasks.workunit.client.1.smithi100.stdout:4/820: fdatasync d9/d27/d62/f9a 0 2024-03-18T14:51:14.212 INFO:tasks.workunit.client.1.smithi100.stdout:1/839: write d8/d30/d61/f63 [8872164,69702] 0 2024-03-18T14:51:14.217 INFO:tasks.workunit.client.1.smithi100.stdout:7/865: dwrite d5/d14/d16/d44/fa7 [4194304,4194304] 0 2024-03-18T14:51:14.217 INFO:tasks.workunit.client.1.smithi100.stdout:7/866: chown d5/d14/d16/d63/d124/dd1/d86/d85/d87/df6/c115 1238 1 2024-03-18T14:51:14.224 INFO:tasks.workunit.client.1.smithi100.stdout:8/900: mkdir d4/dc/d1f/d33/d128/d21/d23/d47/d54/d124/d143 0 2024-03-18T14:51:14.226 INFO:tasks.workunit.client.1.smithi100.stdout:0/886: getdents dd/d31/d127 0 2024-03-18T14:51:14.227 INFO:tasks.workunit.client.1.smithi100.stdout:7/867: dread d5/d14/d16/d63/d124/dd1/f51 [0,4194304] 0 2024-03-18T14:51:14.227 INFO:tasks.workunit.client.1.smithi100.stdout:6/767: symlink d6/d7/df/d72/d65/d7a/lf2 0 2024-03-18T14:51:14.227 INFO:tasks.workunit.client.1.smithi100.stdout:6/768: chown d6/d7/df/d72/c6a 506 1 2024-03-18T14:51:14.227 INFO:tasks.workunit.client.1.smithi100.stdout:6/769: write d6/d7/df/d26/d4e/d66/fd9 [748178,44887] 0 2024-03-18T14:51:14.228 INFO:tasks.workunit.client.1.smithi100.stdout:4/821: symlink d9/d13/lf6 0 2024-03-18T14:51:14.229 INFO:tasks.workunit.client.1.smithi100.stdout:1/840: creat d8/d2a/d37/d50/d6f/d81/f10a x:0 0 0 2024-03-18T14:51:14.231 INFO:tasks.workunit.client.1.smithi100.stdout:5/768: sync 2024-03-18T14:51:14.231 INFO:tasks.workunit.client.1.smithi100.stdout:2/836: sync 2024-03-18T14:51:14.231 INFO:tasks.workunit.client.1.smithi100.stdout:3/853: sync 2024-03-18T14:51:14.231 INFO:tasks.workunit.client.1.smithi100.stdout:2/837: creat d3/da/de/d20/d82/d95/f117 x:0 0 0 2024-03-18T14:51:14.232 INFO:tasks.workunit.client.1.smithi100.stdout:2/838: write d3/da/de/d20/f50 [5513038,105967] 0 2024-03-18T14:51:14.232 INFO:tasks.workunit.client.1.smithi100.stdout:9/895: sync 2024-03-18T14:51:14.233 INFO:tasks.workunit.client.1.smithi100.stdout:8/901: truncate d4/dc/d31/d85/fb2 1638909 0 2024-03-18T14:51:14.234 INFO:tasks.workunit.client.1.smithi100.stdout:8/902: write d4/dc/d1f/d33/d128/d19/f133 [386946,52867] 0 2024-03-18T14:51:14.234 INFO:tasks.workunit.client.1.smithi100.stdout:8/903: chown d4/dc/d1f/d33/d128/d21/d23/d47/f5b 1 1 2024-03-18T14:51:14.235 INFO:tasks.workunit.client.1.smithi100.stdout:0/887: dread dd/d20/f4d [0,4194304] 0 2024-03-18T14:51:14.235 INFO:tasks.workunit.client.1.smithi100.stdout:7/868: creat d5/d14/d16/d63/f13b x:0 0 0 2024-03-18T14:51:14.237 INFO:tasks.workunit.client.1.smithi100.stdout:6/770: creat d6/d7/df/d26/d4e/db5/de1/ff3 x:0 0 0 2024-03-18T14:51:14.238 INFO:tasks.workunit.client.1.smithi100.stdout:6/771: chown d6/d7/df/d26/d4e/d66/f68 0 1 2024-03-18T14:51:14.238 INFO:tasks.workunit.client.1.smithi100.stdout:6/772: write d6/d7/fc5 [5016005,3464] 0 2024-03-18T14:51:14.240 INFO:tasks.workunit.client.1.smithi100.stdout:4/822: symlink d9/d13/lf7 0 2024-03-18T14:51:14.240 INFO:tasks.workunit.client.1.smithi100.stdout:4/823: write d9/d27/f29 [152528,2225] 0 2024-03-18T14:51:14.240 INFO:tasks.workunit.client.1.smithi100.stdout:4/824: getdents d9/d13/d5a/d5b/da0/dd8/df0 0 2024-03-18T14:51:14.240 INFO:tasks.workunit.client.1.smithi100.stdout:4/825: write d9/d27/fa9 [1114645,99388] 0 2024-03-18T14:51:14.247 INFO:tasks.workunit.client.1.smithi100.stdout:5/769: dread f15 [0,4194304] 0 2024-03-18T14:51:14.247 INFO:tasks.workunit.client.1.smithi100.stdout:5/770: chown d29/d2d/d39/f46 0 1 2024-03-18T14:51:14.250 INFO:tasks.workunit.client.1.smithi100.stdout:2/839: unlink d3/d1a/lef 0 2024-03-18T14:51:14.250 INFO:tasks.workunit.client.1.smithi100.stdout:2/840: readlink d3/d1a/d76/lce 0 2024-03-18T14:51:14.250 INFO:tasks.workunit.client.1.smithi100.stdout:2/841: dread - d3/da/de/d6e/ff3 zero size 2024-03-18T14:51:14.251 INFO:tasks.workunit.client.1.smithi100.stdout:9/896: mknod da/d51/d55/c120 0 2024-03-18T14:51:14.252 INFO:tasks.workunit.client.1.smithi100.stdout:8/904: mkdir d4/dc/d1f/d33/d128/d21/d23/d47/d144 0 2024-03-18T14:51:14.252 INFO:tasks.workunit.client.1.smithi100.stdout:8/905: readlink d4/dc/d1f/d33/d128/d21/d23/d25/l3e 0 2024-03-18T14:51:14.252 INFO:tasks.workunit.client.1.smithi100.stdout:8/906: truncate d4/dc/d1f/d33/d128/d21/d23/d47/d54/fa5 901641 0 2024-03-18T14:51:14.256 INFO:tasks.workunit.client.1.smithi100.stdout:8/907: dread d4/dc/d1f/d33/f8d [0,4194304] 0 2024-03-18T14:51:14.256 INFO:tasks.workunit.client.1.smithi100.stdout:0/888: mkdir dd/d31/d91/d56/d71/dcc/dcf/d120/d145 0 2024-03-18T14:51:14.256 INFO:tasks.workunit.client.1.smithi100.stdout:0/889: fdatasync dd/d2b/f4c 0 2024-03-18T14:51:14.258 INFO:tasks.workunit.client.1.smithi100.stdout:7/869: rename d5/d14/d16/d63/d124/dd1/d86/d85/d87/df6/f65 to d5/d90/da1/f13c 0 2024-03-18T14:51:14.258 INFO:tasks.workunit.client.1.smithi100.stdout:7/870: fdatasync d5/d30/d4d/d5a/f82 0 2024-03-18T14:51:14.259 INFO:tasks.workunit.client.1.smithi100.stdout:3/854: dwrite d3/d16/d13/d17/dab/fdb [0,4194304] 0 2024-03-18T14:51:14.259 INFO:tasks.workunit.client.1.smithi100.stdout:3/855: creat d3/dd/d21/dcd/d103/f128 x:0 0 0 2024-03-18T14:51:14.260 INFO:tasks.workunit.client.1.smithi100.stdout:1/841: dwrite d8/d1e/d11/d8a/dda/ff2 [0,4194304] 0 2024-03-18T14:51:14.268 INFO:tasks.workunit.client.1.smithi100.stdout:9/897: mkdir da/dd/d29/d121 0 2024-03-18T14:51:14.268 INFO:tasks.workunit.client.1.smithi100.stdout:9/898: fdatasync da/d2c/d2f/f4a 0 2024-03-18T14:51:14.269 INFO:tasks.workunit.client.1.smithi100.stdout:3/856: dread d3/fa [0,4194304] 0 2024-03-18T14:51:14.269 INFO:tasks.workunit.client.1.smithi100.stdout:6/773: dwrite d6/d12/d78/fb9 [0,4194304] 0 2024-03-18T14:51:14.269 INFO:tasks.workunit.client.1.smithi100.stdout:8/908: chown d4/dc/d1f/d33/d128/d19/ca9 5 1 2024-03-18T14:51:14.272 INFO:tasks.workunit.client.1.smithi100.stdout:4/826: rename d9/d13/d5a/da4/fe0 to d9/d13/ff8 0 2024-03-18T14:51:14.272 INFO:tasks.workunit.client.1.smithi100.stdout:4/827: write d9/d3a/f99 [328084,4901] 0 2024-03-18T14:51:14.273 INFO:tasks.workunit.client.1.smithi100.stdout:5/771: dwrite d29/d2d/d39/d61/f89 [0,4194304] 0 2024-03-18T14:51:14.274 INFO:tasks.workunit.client.1.smithi100.stdout:7/871: symlink d5/d14/d16/d63/d124/dd1/d86/de6/l13d 0 2024-03-18T14:51:14.287 INFO:tasks.workunit.client.1.smithi100.stdout:0/890: dwrite dd/f42 [0,4194304] 0 2024-03-18T14:51:14.288 INFO:tasks.workunit.client.1.smithi100.stdout:0/891: truncate dd/d31/d91/d13b/da9/dac/f135 1019308 0 2024-03-18T14:51:14.288 INFO:tasks.workunit.client.1.smithi100.stdout:0/892: readlink dd/d31/d91/d56/lc2 0 2024-03-18T14:51:14.288 INFO:tasks.workunit.client.1.smithi100.stdout:9/899: dwrite da/dc3/fd9 [0,4194304] 0 2024-03-18T14:51:14.288 INFO:tasks.workunit.client.1.smithi100.stdout:1/842: dwrite d8/d1e/d11/d2d/f6e [0,4194304] 0 2024-03-18T14:51:14.296 INFO:tasks.workunit.client.1.smithi100.stdout:3/857: mknod d3/d16/d13/d1f/d3f/d4f/d65/c129 0 2024-03-18T14:51:14.298 INFO:tasks.workunit.client.1.smithi100.stdout:6/774: link d6/f2f d6/d12/d78/ff4 0 2024-03-18T14:51:14.298 INFO:tasks.workunit.client.1.smithi100.stdout:6/775: dread - d6/fd4 zero size 2024-03-18T14:51:14.301 INFO:tasks.workunit.client.1.smithi100.stdout:8/909: rename d4/dc/d1f/d33/d128/d19/d37/cba to d4/dc/d1f/d33/d128/d19/db0/dd5/c145 0 2024-03-18T14:51:14.304 INFO:tasks.workunit.client.1.smithi100.stdout:4/828: mkdir d9/d13/d5a/d5b/df9 0 2024-03-18T14:51:14.308 INFO:tasks.workunit.client.1.smithi100.stdout:0/893: dwrite dd/d61/fea [0,4194304] 0 2024-03-18T14:51:14.310 INFO:tasks.workunit.client.1.smithi100.stdout:6/776: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/f80 [0,4194304] 0 2024-03-18T14:51:14.310 INFO:tasks.workunit.client.1.smithi100.stdout:6/777: chown d6/f6c 158567236 1 2024-03-18T14:51:14.311 INFO:tasks.workunit.client.1.smithi100.stdout:7/872: mkdir d5/d14/d16/d63/d124/dd1/d86/d13e 0 2024-03-18T14:51:14.312 INFO:tasks.workunit.client.1.smithi100.stdout:7/873: stat d5/d14/c64 0 2024-03-18T14:51:14.312 INFO:tasks.workunit.client.1.smithi100.stdout:7/874: stat d5/d14/d16/d63/d124/dd1/d86/f60 0 2024-03-18T14:51:14.312 INFO:tasks.workunit.client.1.smithi100.stdout:2/842: write d3/d6/d29/d36/ddf/fed [1100989,50827] 0 2024-03-18T14:51:14.312 INFO:tasks.workunit.client.1.smithi100.stdout:2/843: stat d3/d6/d29/d36/d4a/l88 0 2024-03-18T14:51:14.313 INFO:tasks.workunit.client.1.smithi100.stdout:2/844: write d3/da/de/d20/d82/f113 [905361,125739] 0 2024-03-18T14:51:14.318 INFO:tasks.workunit.client.1.smithi100.stdout:9/900: mknod da/dd/d3e/d11e/d8b/c122 0 2024-03-18T14:51:14.319 INFO:tasks.workunit.client.1.smithi100.stdout:8/910: dwrite d4/dc/d31/d85/d8e/ff6 [0,4194304] 0 2024-03-18T14:51:14.322 INFO:tasks.workunit.client.1.smithi100.stdout:1/843: truncate d8/d1e/d11/d27/d83/ff6 1808803 0 2024-03-18T14:51:14.322 INFO:tasks.workunit.client.1.smithi100.stdout:1/844: dread d8/d1e/d11/d8a/fa8 [0,4194304] 0 2024-03-18T14:51:14.325 INFO:tasks.workunit.client.1.smithi100.stdout:3/858: link d3/d16/d13/d17/cc6 d3/dd/d21/dae/d127/c12a 0 2024-03-18T14:51:14.327 INFO:tasks.workunit.client.1.smithi100.stdout:5/772: getdents d29/d2d/d39/d61 0 2024-03-18T14:51:14.334 INFO:tasks.workunit.client.1.smithi100.stdout:0/894: rename dd/d31/d91/l9a to dd/d31/d91/d56/d71/dcc/dcf/l146 0 2024-03-18T14:51:14.348 INFO:tasks.workunit.client.1.smithi100.stdout:6/778: truncate d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/fa3 3593133 0 2024-03-18T14:51:14.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/911: truncate d4/f90 37594 0 2024-03-18T14:51:14.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/912: write d4/dc/d1f/d59/d6c/d75/dea/feb [2260480,80676] 0 2024-03-18T14:51:14.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/913: chown d4/dc/d1f/d33/d128/d21/d23/d47/d54/d124/d143 3608 1 2024-03-18T14:51:14.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/914: dread - d4/dc/d1f/d59/d6c/fdd zero size 2024-03-18T14:51:14.357 INFO:tasks.workunit.client.1.smithi100.stdout:8/915: chown d4/dc/d1f/d33/d128/d19/d2c/da2/cab 24427 1 2024-03-18T14:51:14.371 INFO:tasks.workunit.client.1.smithi100.stdout:1/845: rename d8/d30/d61/d8d/dde/de6/f4b to d8/d30/f10b 0 2024-03-18T14:51:14.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/895: truncate dd/d2b/d95/d79/fa4 1558878 0 2024-03-18T14:51:14.373 INFO:tasks.workunit.client.1.smithi100.stdout:0/896: dread dd/d31/d91/d65/fc8 [0,4194304] 0 2024-03-18T14:51:14.374 INFO:tasks.workunit.client.1.smithi100.stdout:1/846: dread d8/d1e/d11/d27/d3f/d76/fb0 [0,4194304] 0 2024-03-18T14:51:14.375 INFO:tasks.workunit.client.1.smithi100.stdout:6/779: creat d6/d7/df/d72/d8b/ff5 x:0 0 0 2024-03-18T14:51:14.375 INFO:tasks.workunit.client.1.smithi100.stdout:6/780: stat d6/d7/df/d26/d4e/d66/d36/d4f/db8 0 2024-03-18T14:51:14.375 INFO:tasks.workunit.client.1.smithi100.stdout:6/781: write d6/f6c [271948,32175] 0 2024-03-18T14:51:14.379 INFO:tasks.workunit.client.1.smithi100.stdout:8/916: dwrite d4/dc/d1f/d33/d128/f53 [0,4194304] 0 2024-03-18T14:51:14.380 INFO:tasks.workunit.client.1.smithi100.stdout:8/917: dread - d4/dc/d1f/d33/d128/d19/d37/f137 zero size 2024-03-18T14:51:14.384 INFO:tasks.workunit.client.1.smithi100.stdout:8/918: dread d4/dc/d1f/d33/d128/d21/d23/d47/f5b [4194304,4194304] 0 2024-03-18T14:51:14.384 INFO:tasks.workunit.client.1.smithi100.stdout:8/919: creat d4/dc/d31/d85/dd6/f146 x:0 0 0 2024-03-18T14:51:14.384 INFO:tasks.workunit.client.1.smithi100.stdout:8/920: creat d4/dc/d31/d85/d8e/f147 x:0 0 0 2024-03-18T14:51:14.384 INFO:tasks.workunit.client.1.smithi100.stdout:8/921: fdatasync d4/dc/d1f/d59/f64 0 2024-03-18T14:51:14.385 INFO:tasks.workunit.client.1.smithi100.stdout:8/922: chown d4/dc/d31/lb4 1153304677 1 2024-03-18T14:51:14.385 INFO:tasks.workunit.client.1.smithi100.stdout:8/923: write d4/dc/d1f/d33/d128/d19/fe0 [5099906,21685] 0 2024-03-18T14:51:14.385 INFO:tasks.workunit.client.1.smithi100.stdout:8/924: fdatasync d4/dc/d31/d85/d8e/f100 0 2024-03-18T14:51:14.387 INFO:tasks.workunit.client.1.smithi100.stdout:7/875: rmdir d5/d14 39 2024-03-18T14:51:14.387 INFO:tasks.workunit.client.1.smithi100.stdout:7/876: dread - d5/d90/da1/d128/f136 zero size 2024-03-18T14:51:14.391 INFO:tasks.workunit.client.1.smithi100.stdout:3/859: rmdir d3/da5 39 2024-03-18T14:51:14.391 INFO:tasks.workunit.client.1.smithi100.stdout:3/860: stat d3/d16/d13/d1f/d3f 0 2024-03-18T14:51:14.392 INFO:tasks.workunit.client.1.smithi100.stdout:1/847: dwrite d8/d1e/d11/d27/f4d [0,4194304] 0 2024-03-18T14:51:14.392 INFO:tasks.workunit.client.1.smithi100.stdout:1/848: creat d8/d1e/d11/d8a/dda/f10c x:0 0 0 2024-03-18T14:51:14.393 INFO:tasks.workunit.client.1.smithi100.stdout:1/849: creat d8/d2a/d37/d50/d6f/d71/f10d x:0 0 0 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/773: rename l1c to d29/d2d/d39/d83/dc1/dbb/d9e/dd7/df9/lfd 0 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/774: getdents d29/d2d/d39/d71/d96/dc8 0 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/775: write d29/d2d/d39/d83/dc1/dbb/f76 [1530269,113477] 0 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/776: write d29/d2d/d39/d61/ddd/fef [772735,129049] 0 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/777: dread - d29/d2d/d39/d71/d96/fe0 zero size 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/778: chown d29/d2d/d39/d83/dc1/dbb/fa4 52 1 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/779: dread - d29/d2d/d39/d74/fa5 zero size 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/780: dread - d29/d2d/d39/d61/ff3 zero size 2024-03-18T14:51:14.395 INFO:tasks.workunit.client.1.smithi100.stdout:5/781: fdatasync d29/d2d/f6f 0 2024-03-18T14:51:14.414 INFO:tasks.workunit.client.1.smithi100.stdout:1/850: dwrite d8/d1e/d11/f1b [0,4194304] 0 2024-03-18T14:51:14.414 INFO:tasks.workunit.client.1.smithi100.stdout:1/851: creat d8/d1e/d11/d6d/d59/f10e x:0 0 0 2024-03-18T14:51:14.414 INFO:tasks.workunit.client.1.smithi100.stdout:1/852: stat d8/d30/d61/l70 0 2024-03-18T14:51:14.417 INFO:tasks.workunit.client.1.smithi100.stdout:1/853: dread d8/d2a/f41 [0,4194304] 0 2024-03-18T14:51:14.418 INFO:tasks.workunit.client.1.smithi100.stdout:8/925: symlink d4/dc/d31/d85/l148 0 2024-03-18T14:51:14.419 INFO:tasks.workunit.client.1.smithi100.stdout:8/926: write d4/d5a/dbb/f6b [777792,86728] 0 2024-03-18T14:51:14.419 INFO:tasks.workunit.client.1.smithi100.stdout:7/877: creat d5/d14/d16/d63/d124/dd1/d86/de6/f13f x:0 0 0 2024-03-18T14:51:14.424 INFO:tasks.workunit.client.1.smithi100.stdout:3/861: unlink d3/d16/d13/d1f/d7c/cdd 0 2024-03-18T14:51:14.433 INFO:tasks.workunit.client.1.smithi100.stdout:6/782: rename d6/d7/df/d26/d4e/d66/d36/d4f/dcf/fd1 to d6/d7/df/d26/d4e/d66/d36/d4f/db8/ff6 0 2024-03-18T14:51:14.443 INFO:tasks.workunit.client.1.smithi100.stdout:0/897: rmdir dd/d61 39 2024-03-18T14:51:14.450 INFO:tasks.workunit.client.1.smithi100.stdout:7/878: dwrite d5/d14/d16/d63/d124/d66/f54 [0,4194304] 0 2024-03-18T14:51:14.453 INFO:tasks.workunit.client.1.smithi100.stdout:1/854: mknod d8/d30/d61/da2/c10f 0 2024-03-18T14:51:14.455 INFO:tasks.workunit.client.1.smithi100.stdout:7/879: dread d5/d22/d3b/d9f/da0/fa8 [0,4194304] 0 2024-03-18T14:51:14.459 INFO:tasks.workunit.client.1.smithi100.stdout:8/927: rmdir d4/d5a/dbb/d12c 0 2024-03-18T14:51:14.465 INFO:tasks.workunit.client.1.smithi100.stdout:3/862: unlink d3/dd/dd0/lde 0 2024-03-18T14:51:14.465 INFO:tasks.workunit.client.1.smithi100.stdout:3/863: stat d3/dd/d21 0 2024-03-18T14:51:14.468 INFO:tasks.workunit.client.1.smithi100.stdout:6/783: symlink d6/d7/df/d26/d4e/d66/d36/d4f/lf7 0 2024-03-18T14:51:14.469 INFO:tasks.workunit.client.1.smithi100.stdout:5/782: getdents d29/dd0 0 2024-03-18T14:51:14.469 INFO:tasks.workunit.client.1.smithi100.stdout:5/783: fdatasync d29/d2d/d39/d71/f77 0 2024-03-18T14:51:14.470 INFO:tasks.workunit.client.1.smithi100.stdout:5/784: truncate d29/f50 1084860 0 2024-03-18T14:51:14.470 INFO:tasks.workunit.client.1.smithi100.stdout:5/785: chown d29/d2d/d39/d83/dc1/dbb/lfb 0 1 2024-03-18T14:51:14.472 INFO:tasks.workunit.client.1.smithi100.stdout:0/898: mknod dd/d31/d91/d56/d71/dcc/dcf/d120/d145/c147 0 2024-03-18T14:51:14.476 INFO:tasks.workunit.client.1.smithi100.stdout:1/855: dwrite d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:14.477 INFO:tasks.workunit.client.1.smithi100.stdout:7/880: creat d5/d22/d3b/df5/d113/f140 x:0 0 0 2024-03-18T14:51:14.483 INFO:tasks.workunit.client.1.smithi100.stdout:8/928: link d4/d5a/lfe d4/dc/d1f/d33/d128/d21/d23/d47/d54/l149 0 2024-03-18T14:51:14.485 INFO:tasks.workunit.client.1.smithi100.stdout:3/864: truncate d3/dd/d21/fa6 4552774 0 2024-03-18T14:51:14.485 INFO:tasks.workunit.client.1.smithi100.stdout:3/865: fdatasync d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb 0 2024-03-18T14:51:14.490 INFO:tasks.workunit.client.1.smithi100.stdout:6/784: truncate f4 6611812 0 2024-03-18T14:51:14.490 INFO:tasks.workunit.client.1.smithi100.stdout:6/785: read - d6/d7/df/d26/d4e/db5/dba/fd3 zero size 2024-03-18T14:51:14.490 INFO:tasks.workunit.client.1.smithi100.stdout:7/881: write d5/d14/d16/f1d [164885,45509] 0 2024-03-18T14:51:14.502 INFO:tasks.workunit.client.1.smithi100.stdout:8/929: unlink d4/f90 0 2024-03-18T14:51:14.502 INFO:tasks.workunit.client.1.smithi100.stdout:8/930: fdatasync d4/d3c/fe8 0 2024-03-18T14:51:14.502 INFO:tasks.workunit.client.1.smithi100.stdout:8/931: dread - d4/dc/d31/d85/d8e/f147 zero size 2024-03-18T14:51:14.502 INFO:tasks.workunit.client.1.smithi100.stdout:8/932: write d4/dc/d31/d85/fb7 [1056350,70305] 0 2024-03-18T14:51:14.510 INFO:tasks.workunit.client.1.smithi100.stdout:5/786: dwrite d29/d2d/d8f/fb8 [0,4194304] 0 2024-03-18T14:51:14.510 INFO:tasks.workunit.client.1.smithi100.stdout:1/856: dwrite d8/d1e/d11/d27/d83/ff6 [0,4194304] 0 2024-03-18T14:51:14.514 INFO:tasks.workunit.client.1.smithi100.stdout:3/866: dwrite d3/fa [4194304,4194304] 0 2024-03-18T14:51:14.514 INFO:tasks.workunit.client.1.smithi100.stdout:3/867: chown d3/dd/d21/d3e/df8/db9/fb1 244589385 1 2024-03-18T14:51:14.515 INFO:tasks.workunit.client.1.smithi100.stdout:3/868: chown d3/dd/d21/d3e/f32 1887632 1 2024-03-18T14:51:14.527 INFO:tasks.workunit.client.1.smithi100.stdout:7/882: symlink d5/d22/d3b/d9f/da0/l141 0 2024-03-18T14:51:14.527 INFO:tasks.workunit.client.1.smithi100.stdout:7/883: write d5/d22/d3b/d9f/da0/db1/dd3/f118 [163346,55084] 0 2024-03-18T14:51:14.531 INFO:tasks.workunit.client.1.smithi100.stdout:6/786: rename d6/d7/df/d72/d65/c74 to d6/d7/df/d26/d4e/d66/d36/d4f/dcf/cf8 0 2024-03-18T14:51:14.537 INFO:tasks.workunit.client.1.smithi100.stdout:7/884: mkdir d5/d30/d142 0 2024-03-18T14:51:14.539 INFO:tasks.workunit.client.1.smithi100.stdout:6/787: creat d6/d7/df/d72/d65/d7a/da5/ff9 x:0 0 0 2024-03-18T14:51:14.539 INFO:tasks.workunit.client.1.smithi100.stdout:6/788: stat d6/d7 0 2024-03-18T14:51:14.541 INFO:tasks.workunit.client.1.smithi100.stdout:3/869: mknod d3/dd/d21/c12b 0 2024-03-18T14:51:14.541 INFO:tasks.workunit.client.1.smithi100.stdout:7/885: dread d5/d22/d24/fcd [0,4194304] 0 2024-03-18T14:51:14.543 INFO:tasks.workunit.client.1.smithi100.stdout:3/870: creat d3/dd/f12c x:0 0 0 2024-03-18T14:51:14.544 INFO:tasks.workunit.client.1.smithi100.stdout:3/871: read d3/d16/dca/fcf [162392,72126] 0 2024-03-18T14:51:14.544 INFO:tasks.workunit.client.1.smithi100.stdout:3/872: chown d3/d16/d13/d17/c64 121 1 2024-03-18T14:51:14.544 INFO:tasks.workunit.client.1.smithi100.stdout:3/873: chown d3/d16/d13/d17/d7d 1600764847 1 2024-03-18T14:51:14.547 INFO:tasks.workunit.client.1.smithi100.stdout:7/886: rename d5/d22/d3b/d9f/da0/db1/fda to d5/d14/d16/d63/d124/d66/f143 0 2024-03-18T14:51:14.547 INFO:tasks.workunit.client.1.smithi100.stdout:3/874: dread d3/d16/d13/d1f/d3f/d4f/d60/f94 [0,4194304] 0 2024-03-18T14:51:14.553 INFO:tasks.workunit.client.1.smithi100.stdout:1/857: dwrite d8/d1e/d11/d6d/d59/f10e [0,4194304] 0 2024-03-18T14:51:14.554 INFO:tasks.workunit.client.1.smithi100.stdout:8/933: dwrite d4/dc/d1f/d59/d6c/d75/dea/feb [0,4194304] 0 2024-03-18T14:51:14.554 INFO:tasks.workunit.client.1.smithi100.stdout:6/789: read d6/d7/df/d26/d4e/d66/f68 [2493076,93266] 0 2024-03-18T14:51:14.557 INFO:tasks.workunit.client.1.smithi100.stdout:5/787: dwrite d29/d2d/d39/d61/f89 [0,4194304] 0 2024-03-18T14:51:14.571 INFO:tasks.workunit.client.1.smithi100.stdout:1/858: mknod d8/d30/d61/d8d/dde/c110 0 2024-03-18T14:51:14.573 INFO:tasks.workunit.client.1.smithi100.stdout:8/934: symlink d4/dc/d1f/d33/d128/d19/d37/l14a 0 2024-03-18T14:51:14.573 INFO:tasks.workunit.client.1.smithi100.stdout:8/935: write f1 [4453728,108289] 0 2024-03-18T14:51:14.584 INFO:tasks.workunit.client.1.smithi100.stdout:5/788: mkdir d29/d2d/d39/dfe 0 2024-03-18T14:51:14.585 INFO:tasks.workunit.client.1.smithi100.stdout:8/936: dread d4/f15 [0,4194304] 0 2024-03-18T14:51:14.586 INFO:tasks.workunit.client.1.smithi100.stdout:8/937: dread - d4/dc/d31/d85/fcb zero size 2024-03-18T14:51:14.592 INFO:tasks.workunit.client.1.smithi100.stdout:3/875: dwrite d3/d16/d13/d1f/d3f/d4f/d60/da9/fcb [0,4194304] 0 2024-03-18T14:51:14.593 INFO:tasks.workunit.client.1.smithi100.stdout:1/859: dwrite d8/d1e/d11/d27/d83/fd4 [0,4194304] 0 2024-03-18T14:51:14.593 INFO:tasks.workunit.client.1.smithi100.stdout:1/860: write d8/d1e/d11/d27/d3f/daa/dd5/ddc/fba [5147880,68802] 0 2024-03-18T14:51:14.598 INFO:tasks.workunit.client.1.smithi100.stdout:5/789: dread d29/d2d/d39/d65/f6a [0,4194304] 0 2024-03-18T14:51:14.598 INFO:tasks.workunit.client.1.smithi100.stdout:8/938: dread d4/dc/d31/f69 [0,4194304] 0 2024-03-18T14:51:14.599 INFO:tasks.workunit.client.1.smithi100.stdout:8/939: chown d4/dc/d1f/d33/d128/d19/fe0 8952431 1 2024-03-18T14:51:14.599 INFO:tasks.workunit.client.1.smithi100.stdout:6/790: dwrite d6/d7/df/d72/d48/d76/fad [0,4194304] 0 2024-03-18T14:51:14.599 INFO:tasks.workunit.client.1.smithi100.stdout:5/790: mkdir d29/d2d/d39/dfe/dff 0 2024-03-18T14:51:14.599 INFO:tasks.workunit.client.1.smithi100.stdout:8/940: symlink d4/dc/d1f/l14b 0 2024-03-18T14:51:14.600 INFO:tasks.workunit.client.1.smithi100.stdout:8/941: write d4/d3c/fe8 [381647,42153] 0 2024-03-18T14:51:14.601 INFO:tasks.workunit.client.1.smithi100.stdout:8/942: fsync d4/dc/d1f/d33/d128/f63 0 2024-03-18T14:51:14.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/791: mknod d6/d7/df/d26/d4e/db5/de1/cfa 0 2024-03-18T14:51:14.602 INFO:tasks.workunit.client.1.smithi100.stdout:6/792: write d6/d7/df/d26/d4e/d66/d36/fde [1272496,31162] 0 2024-03-18T14:51:14.602 INFO:tasks.workunit.client.1.smithi100.stdout:1/861: symlink d8/l111 0 2024-03-18T14:51:14.603 INFO:tasks.workunit.client.1.smithi100.stdout:8/943: mknod d4/d3c/c14c 0 2024-03-18T14:51:14.607 INFO:tasks.workunit.client.1.smithi100.stdout:1/862: rename d8/d30/d61/d8d/dde/c110 to d8/d30/c112 0 2024-03-18T14:51:14.608 INFO:tasks.workunit.client.1.smithi100.stdout:1/863: fsync d8/fab 0 2024-03-18T14:51:14.608 INFO:tasks.workunit.client.1.smithi100.stdout:4/829: sync 2024-03-18T14:51:14.609 INFO:tasks.workunit.client.1.smithi100.stdout:2/845: sync 2024-03-18T14:51:14.609 INFO:tasks.workunit.client.1.smithi100.stdout:9/901: sync 2024-03-18T14:51:14.611 INFO:tasks.workunit.client.1.smithi100.stdout:5/791: write d29/d2d/d39/d83/dc1/dbb/f53 [779596,87331] 0 2024-03-18T14:51:14.612 INFO:tasks.workunit.client.1.smithi100.stdout:8/944: dread d4/dc/d1f/d33/d128/d21/d23/d47/d54/fa5 [0,4194304] 0 2024-03-18T14:51:14.612 INFO:tasks.workunit.client.1.smithi100.stdout:8/945: stat d4/dc/d31/d85/d8e/f100 0 2024-03-18T14:51:14.616 INFO:tasks.workunit.client.1.smithi100.stdout:6/793: mknod d6/d7/df/d72/d8b/cfb 0 2024-03-18T14:51:14.616 INFO:tasks.workunit.client.1.smithi100.stdout:6/794: chown d6/d7/c42 68817 1 2024-03-18T14:51:14.618 INFO:tasks.workunit.client.1.smithi100.stdout:2/846: creat d3/da/de/d20/dea/d40/d7d/f118 x:0 0 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:3/876: dwrite d3/dd/f12c [0,4194304] 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:3/877: truncate d3/d16/d13/d1f/d3f/d4f/de0/ffd 5041595 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:3/878: fsync d3/d16/d13/d1f/d3f/d4f/d60/f63 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:3/879: fdatasync d3/dd/d21/d3e/fcc 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:1/864: dread d8/d1e/d11/d27/d73/d89/f9d [0,4194304] 0 2024-03-18T14:51:14.621 INFO:tasks.workunit.client.1.smithi100.stdout:4/830: creat d9/d27/d62/d7b/d82/dac/ffa x:0 0 0 2024-03-18T14:51:14.623 INFO:tasks.workunit.client.1.smithi100.stdout:9/902: symlink da/d88/da9/l123 0 2024-03-18T14:51:14.623 INFO:tasks.workunit.client.1.smithi100.stdout:9/903: creat da/dd/d29/d71/df5/f124 x:0 0 0 2024-03-18T14:51:14.623 INFO:tasks.workunit.client.1.smithi100.stdout:9/904: fdatasync f9 0 2024-03-18T14:51:14.625 INFO:tasks.workunit.client.1.smithi100.stdout:5/792: unlink d29/d2d/dbe/dc2/fce 0 2024-03-18T14:51:14.626 INFO:tasks.workunit.client.1.smithi100.stdout:4/831: dread d9/d13/d5a/d5b/f70 [0,4194304] 0 2024-03-18T14:51:14.626 INFO:tasks.workunit.client.1.smithi100.stdout:8/946: mknod d4/dc/dd/d48/c14d 0 2024-03-18T14:51:14.627 INFO:tasks.workunit.client.1.smithi100.stdout:8/947: dread - d4/dc/dd/d48/f142 zero size 2024-03-18T14:51:14.627 INFO:tasks.workunit.client.1.smithi100.stdout:2/847: mknod d3/da/db8/c119 0 2024-03-18T14:51:14.628 INFO:tasks.workunit.client.1.smithi100.stdout:2/848: chown d3/d6/d5f/d6a/fc8 978421665 1 2024-03-18T14:51:14.631 INFO:tasks.workunit.client.1.smithi100.stdout:8/948: dread d4/d5a/f139 [0,4194304] 0 2024-03-18T14:51:14.634 INFO:tasks.workunit.client.1.smithi100.stdout:6/795: dwrite d6/d7/df/d26/d4e/fd5 [0,4194304] 0 2024-03-18T14:51:14.634 INFO:tasks.workunit.client.1.smithi100.stdout:6/796: dread - d6/d7/df/d26/d4e/d66/fd0 zero size 2024-03-18T14:51:14.634 INFO:tasks.workunit.client.1.smithi100.stdout:6/797: dread - d6/d7/df/d26/d4e/db5/dba/fd3 zero size 2024-03-18T14:51:14.639 INFO:tasks.workunit.client.1.smithi100.stdout:3/880: dwrite d3/d16/d13/d17/d7d/fd4 [0,4194304] 0 2024-03-18T14:51:14.645 INFO:tasks.workunit.client.1.smithi100.stdout:1/865: mknod d8/d2a/d37/d50/dc5/c113 0 2024-03-18T14:51:14.649 INFO:tasks.workunit.client.1.smithi100.stdout:9/905: mknod da/d51/c125 0 2024-03-18T14:51:14.649 INFO:tasks.workunit.client.1.smithi100.stdout:5/793: mknod d29/d2d/d39/d71/d96/c100 0 2024-03-18T14:51:14.650 INFO:tasks.workunit.client.1.smithi100.stdout:9/906: read da/d2c/d70/fc4 [250837,43089] 0 2024-03-18T14:51:14.651 INFO:tasks.workunit.client.1.smithi100.stdout:4/832: rename d9/d13/d5a/cdb to d9/d13/d5a/d5b/da0/dd8/df0/cfb 0 2024-03-18T14:51:14.652 INFO:tasks.workunit.client.1.smithi100.stdout:2/849: mkdir d3/da/de/d20/d82/dad/dc5/d11a 0 2024-03-18T14:51:14.652 INFO:tasks.workunit.client.1.smithi100.stdout:2/850: fdatasync d3/dd4/ff9 0 2024-03-18T14:51:14.653 INFO:tasks.workunit.client.1.smithi100.stdout:8/949: mknod d4/dc/d1f/d33/d128/d19/d37/c14e 0 2024-03-18T14:51:14.653 INFO:tasks.workunit.client.1.smithi100.stdout:8/950: read - d4/dc/d1f/d33/d128/d19/d37/f137 zero size 2024-03-18T14:51:14.658 INFO:tasks.workunit.client.1.smithi100.stdout:1/866: mknod d8/d2a/c114 0 2024-03-18T14:51:14.659 INFO:tasks.workunit.client.1.smithi100.stdout:5/794: unlink d29/d2d/d39/d61/ddd/le4 0 2024-03-18T14:51:14.661 INFO:tasks.workunit.client.1.smithi100.stdout:9/907: symlink da/dd/d29/d9e/dcd/l126 0 2024-03-18T14:51:14.665 INFO:tasks.workunit.client.1.smithi100.stdout:4/833: mknod d9/d13/d5a/d5b/da0/dd8/cfc 0 2024-03-18T14:51:14.669 INFO:tasks.workunit.client.1.smithi100.stdout:3/881: mknod d3/dd/d21/c12d 0 2024-03-18T14:51:14.673 INFO:tasks.workunit.client.1.smithi100.stdout:1/867: mknod d8/d2a/d37/c115 0 2024-03-18T14:51:14.676 INFO:tasks.workunit.client.1.smithi100.stdout:5/795: truncate d29/d2d/d39/d71/fd3 2375590 0 2024-03-18T14:51:14.681 INFO:tasks.workunit.client.1.smithi100.stdout:6/798: rename d6/d12/d2c/le2 to d6/d7/df/d72/d65/lfc 0 2024-03-18T14:51:14.682 INFO:tasks.workunit.client.1.smithi100.stdout:8/951: dwrite d4/dc/d1f/d33/d128/d19/d37/f137 [0,4194304] 0 2024-03-18T14:51:14.682 INFO:tasks.workunit.client.1.smithi100.stdout:3/882: mknod d3/dd/d21/d3e/df8/c12e 0 2024-03-18T14:51:14.683 INFO:tasks.workunit.client.1.smithi100.stdout:2/851: dwrite d3/da/de/d20/f24 [4194304,4194304] 0 2024-03-18T14:51:14.683 INFO:tasks.workunit.client.1.smithi100.stdout:2/852: read - d3/da/de/d6e/f115 zero size 2024-03-18T14:51:14.684 INFO:tasks.workunit.client.1.smithi100.stdout:9/908: dwrite da/dd/d3c/fc9 [0,4194304] 0 2024-03-18T14:51:14.685 INFO:tasks.workunit.client.1.smithi100.stdout:9/909: chown da/dd/d3c/d6b/cdf 31731866 1 2024-03-18T14:51:14.685 INFO:tasks.workunit.client.1.smithi100.stdout:1/868: creat d8/d1e/d11/d27/d3f/f116 x:0 0 0 2024-03-18T14:51:14.688 INFO:tasks.workunit.client.1.smithi100.stdout:4/834: rename d9/d13/dae to d9/d13/d39/dfd 0 2024-03-18T14:51:14.688 INFO:tasks.workunit.client.1.smithi100.stdout:4/835: dread - d9/d17/fdd zero size 2024-03-18T14:51:14.688 INFO:tasks.workunit.client.1.smithi100.stdout:4/836: fsync d9/d17/f34 0 2024-03-18T14:51:14.688 INFO:tasks.workunit.client.1.smithi100.stdout:4/837: chown d9/d13/d25 521607 1 2024-03-18T14:51:14.689 INFO:tasks.workunit.client.1.smithi100.stdout:4/838: chown d9/f10 2024 1 2024-03-18T14:51:14.689 INFO:tasks.workunit.client.1.smithi100.stdout:6/799: mknod d6/d7/df/d26/d4e/db5/de1/cfd 0 2024-03-18T14:51:14.690 INFO:tasks.workunit.client.1.smithi100.stdout:6/800: stat d6/d7/df/d26/d4e/d66/d36/d4f/d71/cc8 0 2024-03-18T14:51:14.690 INFO:tasks.workunit.client.1.smithi100.stdout:6/801: chown d6/da1/dd7/cf0 10624 1 2024-03-18T14:51:14.693 INFO:tasks.workunit.client.1.smithi100.stdout:5/796: dwrite d29/f9b [0,4194304] 0 2024-03-18T14:51:14.695 INFO:tasks.workunit.client.1.smithi100.stdout:8/952: creat d4/dc/d1f/d33/d128/d21/d23/d47/d56/f14f x:0 0 0 2024-03-18T14:51:14.701 INFO:tasks.workunit.client.1.smithi100.stdout:1/869: dwrite d8/d1e/d11/d8a/dda/f10c [0,4194304] 0 2024-03-18T14:51:14.703 INFO:tasks.workunit.client.1.smithi100.stdout:9/910: mknod da/dd/d3c/d4b/c127 0 2024-03-18T14:51:14.703 INFO:tasks.workunit.client.1.smithi100.stdout:9/911: chown da/dd/f8f 6931032 1 2024-03-18T14:51:14.706 INFO:tasks.workunit.client.1.smithi100.stdout:3/883: rename d3/dd/d21/d3e/c43 to d3/d16/d13/d1f/d3f/d4f/d60/da9/c12f 0 2024-03-18T14:51:14.707 INFO:tasks.workunit.client.1.smithi100.stdout:4/839: link d9/d27/da2/fc3 d9/d27/da2/ffe 0 2024-03-18T14:51:14.713 INFO:tasks.workunit.client.1.smithi100.stdout:5/797: creat d29/d2d/d39/d61/f101 x:0 0 0 2024-03-18T14:51:14.715 INFO:tasks.workunit.client.1.smithi100.stdout:8/953: symlink d4/dc/dd/d48/l150 0 2024-03-18T14:51:14.717 INFO:tasks.workunit.client.1.smithi100.stdout:2/853: getdents d3/da/de/d20/dea/d40/d53 0 2024-03-18T14:51:14.717 INFO:tasks.workunit.client.1.smithi100.stdout:2/854: write d3/d1a/d6c/f8b [3844246,34562] 0 2024-03-18T14:51:14.733 INFO:tasks.workunit.client.1.smithi100.stdout:5/798: dwrite d29/f50 [0,4194304] 0 2024-03-18T14:51:14.734 INFO:tasks.workunit.client.1.smithi100.stdout:5/799: creat d29/d2d/d39/d61/ddd/f102 x:0 0 0 2024-03-18T14:51:14.734 INFO:tasks.workunit.client.1.smithi100.stdout:5/800: chown d29/d4a/c7f 6569 1 2024-03-18T14:51:14.734 INFO:tasks.workunit.client.1.smithi100.stdout:5/801: read d29/d2d/d39/d60/f70 [531468,62876] 0 2024-03-18T14:51:14.734 INFO:tasks.workunit.client.1.smithi100.stdout:3/884: dwrite d3/d16/d13/d17/f61 [0,4194304] 0 2024-03-18T14:51:14.737 INFO:tasks.workunit.client.1.smithi100.stdout:1/870: mknod d8/d1e/d11/d27/d3f/daa/c117 0 2024-03-18T14:51:14.737 INFO:tasks.workunit.client.1.smithi100.stdout:1/871: write d8/d1e/d11/d2d/f6e [4456569,37864] 0 2024-03-18T14:51:14.737 INFO:tasks.workunit.client.1.smithi100.stdout:1/872: truncate d8/d1e/d11/fc3 4984262 0 2024-03-18T14:51:14.738 INFO:tasks.workunit.client.1.smithi100.stdout:1/873: chown d8/d1e/d11/d27/d3f/daa 109118 1 2024-03-18T14:51:14.739 INFO:tasks.workunit.client.1.smithi100.stdout:9/912: symlink da/dd/d29/d71/d72/db2/l128 0 2024-03-18T14:51:14.741 INFO:tasks.workunit.client.1.smithi100.stdout:6/802: rename d6/da1/dd7 to d6/d7/df/d26/d4e/db5/dba/dfe 0 2024-03-18T14:51:14.741 INFO:tasks.workunit.client.1.smithi100.stdout:6/803: creat d6/d7/df/d26/d4e/d66/fff x:0 0 0 2024-03-18T14:51:14.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:14 smithi012 ceph-mon[29182]: pgmap v11: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 212 MiB/s rd, 241 MiB/s wr, 376 op/s 2024-03-18T14:51:14.744 INFO:tasks.workunit.client.1.smithi100.stdout:4/840: creat d9/da6/fff x:0 0 0 2024-03-18T14:51:14.745 INFO:tasks.workunit.client.1.smithi100.stdout:8/954: rmdir d4/dc/d1f 39 2024-03-18T14:51:14.746 INFO:tasks.workunit.client.1.smithi100.stdout:7/887: sync 2024-03-18T14:51:14.746 INFO:tasks.workunit.client.1.smithi100.stdout:0/899: sync 2024-03-18T14:51:14.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/888: dread d5/d14/d16/f1a [0,4194304] 0 2024-03-18T14:51:14.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/889: creat d5/d22/d3b/df5/f144 x:0 0 0 2024-03-18T14:51:14.753 INFO:tasks.workunit.client.1.smithi100.stdout:7/890: fdatasync d5/d22/d24/d6e/d7a/fe3 0 2024-03-18T14:51:14.754 INFO:tasks.workunit.client.1.smithi100.stdout:5/802: truncate d29/d2d/d32/d92/fb9 991845 0 2024-03-18T14:51:14.758 INFO:tasks.workunit.client.1.smithi100.stdout:3/885: mkdir d3/dd/dd0/d130 0 2024-03-18T14:51:14.759 INFO:tasks.workunit.client.1.smithi100.stdout:7/891: dread d5/d14/d16/d63/d124/dfe/f117 [0,4194304] 0 2024-03-18T14:51:14.763 INFO:tasks.workunit.client.1.smithi100.stdout:1/874: truncate d8/d1e/d11/d6d/d59/fd7 2241156 0 2024-03-18T14:51:14.765 INFO:tasks.workunit.client.1.smithi100.stdout:9/913: mknod da/dc3/d104/c129 0 2024-03-18T14:51:14.769 INFO:tasks.workunit.client.1.smithi100.stdout:6/804: symlink d6/d7/df/d26/d4e/d66/d36/d4f/dcf/l100 0 2024-03-18T14:51:14.773 INFO:tasks.workunit.client.1.smithi100.stdout:4/841: mknod d9/d27/d62/d7b/d82/dac/c100 0 2024-03-18T14:51:14.773 INFO:tasks.workunit.client.1.smithi100.stdout:4/842: truncate d9/da6/fff 977634 0 2024-03-18T14:51:14.774 INFO:tasks.workunit.client.1.smithi100.stdout:8/955: mkdir d4/d5a/dbb/d151 0 2024-03-18T14:51:14.775 INFO:tasks.workunit.client.1.smithi100.stdout:9/914: dread da/dd/d3e/d11e/d4d/fad [0,4194304] 0 2024-03-18T14:51:14.776 INFO:tasks.workunit.client.1.smithi100.stdout:0/900: symlink dd/d20/db5/l148 0 2024-03-18T14:51:14.778 INFO:tasks.workunit.client.1.smithi100.stdout:3/886: dwrite d3/d16/d13/d17/fe5 [0,4194304] 0 2024-03-18T14:51:14.778 INFO:tasks.workunit.client.1.smithi100.stdout:3/887: read - d3/dd/d21/d3e/fc0 zero size 2024-03-18T14:51:14.781 INFO:tasks.workunit.client.1.smithi100.stdout:2/855: rmdir d3/d6/d29 39 2024-03-18T14:51:14.783 INFO:tasks.workunit.client.1.smithi100.stdout:8/956: dread d4/dc/dd/d48/f117 [0,4194304] 0 2024-03-18T14:51:14.783 INFO:tasks.workunit.client.1.smithi100.stdout:8/957: creat d4/dc/d1f/d33/d128/d19/d2c/da2/f152 x:0 0 0 2024-03-18T14:51:14.787 INFO:tasks.workunit.client.1.smithi100.stdout:5/803: rename d29/d2d/d39/d83/dc1/lcb to d29/d2d/d39/d71/d96/dc8/l103 0 2024-03-18T14:51:14.787 INFO:tasks.workunit.client.1.smithi100.stdout:5/804: chown d29/d2d/d39/d61/cba 56148584 1 2024-03-18T14:51:14.788 INFO:tasks.workunit.client.1.smithi100.stdout:3/888: dread d3/dd/d49/f70 [0,4194304] 0 2024-03-18T14:51:14.789 INFO:tasks.workunit.client.1.smithi100.stdout:7/892: mknod d5/d22/d3b/d9b/c145 0 2024-03-18T14:51:14.793 INFO:tasks.workunit.client.1.smithi100.stdout:6/805: rmdir d6/d9f 39 2024-03-18T14:51:14.793 INFO:tasks.workunit.client.1.smithi100.stdout:6/806: read - d6/fd4 zero size 2024-03-18T14:51:14.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:14 smithi100 ceph-mon[37807]: pgmap v11: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 212 MiB/s rd, 241 MiB/s wr, 376 op/s 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:9/915: symlink da/dc3/d104/l12a 0 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:9/916: dread - da/d88/f10f zero size 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:9/917: write da/dd/d3c/d4b/f58 [4842157,62122] 0 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:9/918: write da/dd/d29/d71/d72/f78 [3267689,60019] 0 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:9/919: write da/dd/d3e/d11e/f63 [926902,2158] 0 2024-03-18T14:51:14.804 INFO:tasks.workunit.client.1.smithi100.stdout:2/856: dwrite d3/dd4/f100 [0,4194304] 0 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:2/857: chown d3/d1a/ccc 2976 1 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:0/901: creat dd/d31/d91/d56/d71/dcc/dcf/d120/d145/f149 x:0 0 0 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:8/958: getdents d4/dc/d1f/d33/d128/d21/d23/d119 0 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:8/959: fsync d4/dc/d1f/d33/d128/d19/d37/f61 0 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:8/960: fdatasync d4/f43 0 2024-03-18T14:51:14.805 INFO:tasks.workunit.client.1.smithi100.stdout:8/961: readlink d4/dc/dd/l79 0 2024-03-18T14:51:14.808 INFO:tasks.workunit.client.1.smithi100.stdout:4/843: write d9/d13/d25/d51/f60 [1915439,14932] 0 2024-03-18T14:51:14.809 INFO:tasks.workunit.client.1.smithi100.stdout:3/889: mknod d3/dd/dd0/c131 0 2024-03-18T14:51:14.809 INFO:tasks.workunit.client.1.smithi100.stdout:3/890: write d3/d16/d13/d17/f7f [876033,4692] 0 2024-03-18T14:51:14.809 INFO:tasks.workunit.client.1.smithi100.stdout:3/891: truncate d3/d16/d13/d17/f4c 3409852 0 2024-03-18T14:51:14.810 INFO:tasks.workunit.client.1.smithi100.stdout:7/893: symlink d5/d14/d16/d63/d124/dfe/l146 0 2024-03-18T14:51:14.811 INFO:tasks.workunit.client.1.smithi100.stdout:1/875: symlink d8/dd/l118 0 2024-03-18T14:51:14.812 INFO:tasks.workunit.client.1.smithi100.stdout:9/920: link da/dd/d3b/fc6 da/dd/d29/d71/d72/df7/f12b 0 2024-03-18T14:51:14.813 INFO:tasks.workunit.client.1.smithi100.stdout:6/807: dwrite d6/f91 [0,4194304] 0 2024-03-18T14:51:14.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/962: getdents d4/dc 0 2024-03-18T14:51:14.817 INFO:tasks.workunit.client.1.smithi100.stdout:8/963: write d4/ff [4729671,64208] 0 2024-03-18T14:51:14.820 INFO:tasks.workunit.client.1.smithi100.stdout:5/805: rmdir d29/d2d/d32/d92 39 2024-03-18T14:51:14.823 INFO:tasks.workunit.client.1.smithi100.stdout:4/844: creat d9/d27/f101 x:0 0 0 2024-03-18T14:51:14.823 INFO:tasks.workunit.client.1.smithi100.stdout:4/845: creat d9/d13/d5a/d5b/da0/dcd/f102 x:0 0 0 2024-03-18T14:51:14.825 INFO:tasks.workunit.client.1.smithi100.stdout:4/846: dread d9/d27/f29 [0,4194304] 0 2024-03-18T14:51:14.825 INFO:tasks.workunit.client.1.smithi100.stdout:4/847: read d9/d27/d42/f6a [2634733,105262] 0 2024-03-18T14:51:14.827 INFO:tasks.workunit.client.1.smithi100.stdout:0/902: dwrite dd/d2b/d11e/fe7 [0,4194304] 0 2024-03-18T14:51:14.833 INFO:tasks.workunit.client.1.smithi100.stdout:9/921: creat da/dcb/f12c x:0 0 0 2024-03-18T14:51:14.839 INFO:tasks.workunit.client.1.smithi100.stdout:8/964: creat d4/dc/d1f/d33/d128/d19/db0/dd5/f153 x:0 0 0 2024-03-18T14:51:14.841 INFO:tasks.workunit.client.1.smithi100.stdout:0/903: dread dd/f85 [4194304,4194304] 0 2024-03-18T14:51:14.842 INFO:tasks.workunit.client.1.smithi100.stdout:4/848: rmdir d9/dc8 39 2024-03-18T14:51:14.845 INFO:tasks.workunit.client.1.smithi100.stdout:9/922: mkdir da/dd/d3c/d4b/dd4/d12d 0 2024-03-18T14:51:14.845 INFO:tasks.workunit.client.1.smithi100.stdout:6/808: rmdir d6/d7/df/d72/d65/d7a/da5 39 2024-03-18T14:51:14.846 INFO:tasks.workunit.client.1.smithi100.stdout:2/858: dwrite d3/da/de/d20/f50 [0,4194304] 0 2024-03-18T14:51:14.847 INFO:tasks.workunit.client.1.smithi100.stdout:7/894: dwrite d5/d14/d16/d63/d124/d66/f54 [0,4194304] 0 2024-03-18T14:51:14.848 INFO:tasks.workunit.client.1.smithi100.stdout:7/895: stat d5/d22/d3b/d9f/da0/db1/dd3/fe0 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:8/965: creat d4/dc/d1f/d33/d128/d19/f154 x:0 0 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:1/876: dwrite d8/d2a/fc7 [0,4194304] 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/849: truncate d9/d27/d62/d7b/d82/fef 3632176 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/850: write d9/d27/d62/d7b/f8e [733370,76737] 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/851: write d9/d13/d5a/fcb [779450,72634] 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/852: chown d9/d13/d5a/d5b/da0 60383 1 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/853: truncate d9/d13/d39/fad 38468 0 2024-03-18T14:51:14.849 INFO:tasks.workunit.client.1.smithi100.stdout:4/854: creat d9/d13/d5a/d5b/da0/dcd/f103 x:0 0 0 2024-03-18T14:51:14.850 INFO:tasks.workunit.client.1.smithi100.stdout:3/892: dwrite d3/dd/d21/d3e/f79 [4194304,4194304] 0 2024-03-18T14:51:14.851 INFO:tasks.workunit.client.1.smithi100.stdout:5/806: dwrite d29/d2d/d32/fd6 [0,4194304] 0 2024-03-18T14:51:14.851 INFO:tasks.workunit.client.1.smithi100.stdout:5/807: write d29/d2d/d39/f4e [4272827,125923] 0 2024-03-18T14:51:14.855 INFO:tasks.workunit.client.1.smithi100.stdout:0/904: write dd/f8f [422880,26869] 0 2024-03-18T14:51:14.855 INFO:tasks.workunit.client.1.smithi100.stdout:0/905: fdatasync dd/d20/f2c 0 2024-03-18T14:51:14.858 INFO:tasks.workunit.client.1.smithi100.stdout:3/893: rmdir d3/dd/d21/d3e/df8/db9 39 2024-03-18T14:51:14.858 INFO:tasks.workunit.client.1.smithi100.stdout:9/923: dread da/dd/f10 [0,4194304] 0 2024-03-18T14:51:14.860 INFO:tasks.workunit.client.1.smithi100.stdout:6/809: mkdir d6/d7/df/d72/d65/d101 0 2024-03-18T14:51:14.860 INFO:tasks.workunit.client.1.smithi100.stdout:2/859: write d3/d6/d5f/d63/f72 [2034127,111065] 0 2024-03-18T14:51:14.861 INFO:tasks.workunit.client.1.smithi100.stdout:4/855: dread d9/d13/d5a/d5b/da0/dcd/fd3 [0,4194304] 0 2024-03-18T14:51:14.861 INFO:tasks.workunit.client.1.smithi100.stdout:4/856: dread - d9/d13/d5a/d5b/da0/dcd/f103 zero size 2024-03-18T14:51:14.862 INFO:tasks.workunit.client.1.smithi100.stdout:8/966: write d4/dc/d1f/d33/d128/f63 [902839,59008] 0 2024-03-18T14:51:14.863 INFO:tasks.workunit.client.1.smithi100.stdout:8/967: readlink d4/dc/d1f/d33/d128/d19/db0/dd5/l141 0 2024-03-18T14:51:14.867 INFO:tasks.workunit.client.1.smithi100.stdout:5/808: mknod d29/d2d/d39/d60/c104 0 2024-03-18T14:51:14.870 INFO:tasks.workunit.client.1.smithi100.stdout:0/906: creat dd/d31/d91/d13b/da9/dac/f14a x:0 0 0 2024-03-18T14:51:14.871 INFO:tasks.workunit.client.1.smithi100.stdout:3/894: mknod d3/d16/dca/d10e/c132 0 2024-03-18T14:51:14.871 INFO:tasks.workunit.client.1.smithi100.stdout:3/895: write d3/d16/d13/d17/d7d/fa3 [1286696,76627] 0 2024-03-18T14:51:14.871 INFO:tasks.workunit.client.1.smithi100.stdout:3/896: write d3/dd/f28 [1228727,41622] 0 2024-03-18T14:51:14.875 INFO:tasks.workunit.client.1.smithi100.stdout:4/857: dread d9/f11 [0,4194304] 0 2024-03-18T14:51:14.876 INFO:tasks.workunit.client.1.smithi100.stdout:6/810: rmdir d6/d7/df/d72/d65/d101 0 2024-03-18T14:51:14.878 INFO:tasks.workunit.client.1.smithi100.stdout:2/860: link d3/fc d3/d1a/d76/f11b 0 2024-03-18T14:51:14.879 INFO:tasks.workunit.client.1.smithi100.stdout:9/924: dwrite da/dd/d29/d71/d72/f118 [0,4194304] 0 2024-03-18T14:51:14.879 INFO:tasks.workunit.client.1.smithi100.stdout:9/925: write da/dd/d3b/d103/ffd [346850,110269] 0 2024-03-18T14:51:14.879 INFO:tasks.workunit.client.1.smithi100.stdout:9/926: truncate da/dd/d3b/d103/ffd 958943 0 2024-03-18T14:51:14.880 INFO:tasks.workunit.client.1.smithi100.stdout:1/877: dwrite d8/d2a/f47 [0,4194304] 0 2024-03-18T14:51:14.880 INFO:tasks.workunit.client.1.smithi100.stdout:1/878: fdatasync d8/d2a/d37/f66 0 2024-03-18T14:51:14.881 INFO:tasks.workunit.client.1.smithi100.stdout:7/896: dwrite d5/d14/f1e [0,4194304] 0 2024-03-18T14:51:14.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/968: unlink d4/dc/d1f/d33/d128/c6f 0 2024-03-18T14:51:14.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/969: dread - d4/dc/d1f/d33/d128/d19/d37/fb6 zero size 2024-03-18T14:51:14.883 INFO:tasks.workunit.client.1.smithi100.stdout:8/970: creat d4/dc/dd/d48/f155 x:0 0 0 2024-03-18T14:51:14.886 INFO:tasks.workunit.client.1.smithi100.stdout:5/809: symlink d29/d2d/d39/d61/ddd/l105 0 2024-03-18T14:51:14.886 INFO:tasks.workunit.client.1.smithi100.stdout:5/810: chown d29/d2d/d39/d71/d96/c9d 6 1 2024-03-18T14:51:14.887 INFO:tasks.workunit.client.1.smithi100.stdout:5/811: creat d29/d2d/d39/f106 x:0 0 0 2024-03-18T14:51:14.889 INFO:tasks.workunit.client.1.smithi100.stdout:0/907: truncate dd/d20/f2d 788241 0 2024-03-18T14:51:14.890 INFO:tasks.workunit.client.1.smithi100.stdout:2/861: write d3/d6/d29/d36/d49/fb4 [1658237,8514] 0 2024-03-18T14:51:14.897 INFO:tasks.workunit.client.1.smithi100.stdout:4/858: mknod d9/dc8/c104 0 2024-03-18T14:51:14.897 INFO:tasks.workunit.client.1.smithi100.stdout:1/879: read d8/d30/f40 [2309755,25143] 0 2024-03-18T14:51:14.897 INFO:tasks.workunit.client.1.smithi100.stdout:4/859: readlink d9/d27/da2/lb7 0 2024-03-18T14:51:14.897 INFO:tasks.workunit.client.1.smithi100.stdout:4/860: write d9/d13/d5a/d5b/da0/dcd/f103 [128781,129553] 0 2024-03-18T14:51:14.898 INFO:tasks.workunit.client.1.smithi100.stdout:4/861: read d9/da6/fff [132860,113729] 0 2024-03-18T14:51:14.898 INFO:tasks.workunit.client.1.smithi100.stdout:4/862: truncate d9/d13/ff2 940993 0 2024-03-18T14:51:14.900 INFO:tasks.workunit.client.1.smithi100.stdout:6/811: rename d6/d7/df/d72/d65/lfc to d6/d7/df/d26/d4e/d66/d36/d4f/d71/d27/d64/l102 0 2024-03-18T14:51:14.900 INFO:tasks.workunit.client.1.smithi100.stdout:9/927: dwrite da/dd/d3e/d11e/d8b/f3f [0,4194304] 0 2024-03-18T14:51:14.902 INFO:tasks.workunit.client.1.smithi100.stdout:3/897: dwrite d3/d16/d13/d17/d7d/f101 [0,4194304] 0 2024-03-18T14:51:14.913 INFO:tasks.workunit.client.1.smithi100.stdout:4/863: dread d9/d13/d5a/fcb [0,4194304] 0 2024-03-18T14:51:14.913 INFO:tasks.workunit.client.1.smithi100.stdout:4/864: write d9/d27/f101 [201883,109233] 0 2024-03-18T14:51:14.913 INFO:tasks.workunit.client.1.smithi100.stdout:2/862: dwrite f0 [0,4194304] 0 2024-03-18T14:51:14.914 INFO:tasks.workunit.client.1.smithi100.stdout:5/812: dwrite d29/d2d/d39/fde [0,4194304] 0 2024-03-18T14:51:14.914 INFO:tasks.workunit.client.1.smithi100.stdout:5/813: readlink d29/d2d/lc4 0 2024-03-18T14:51:14.916 INFO:tasks.workunit.client.1.smithi100.stdout:7/897: creat d5/d22/d24/d6e/d7a/dc2/d111/d129/f147 x:0 0 0 2024-03-18T14:51:14.919 INFO:tasks.workunit.client.1.smithi100.stdout:0/908: dwrite dd/d31/d91/d56/ffa [0,4194304] 0 2024-03-18T14:51:14.927 INFO:tasks.workunit.client.1.smithi100.stdout:1/880: creat d8/d1e/d11/d27/d3f/daa/dd1/f119 x:0 0 0 2024-03-18T14:51:14.929 INFO:tasks.workunit.client.1.smithi100.stdout:8/971: rename d4/dc/d1f/d33/d128/d21/d23/d47/d56/l8c to d4/dc/d1f/d33/d128/d21/d23/d47/d54/d136/l156 0 2024-03-18T14:51:14.929 INFO:tasks.workunit.client.1.smithi100.stdout:8/972: chown d4/dc/d1f/d33/d128/d21/d23/l3b 0 1 2024-03-18T14:51:14.929 INFO:tasks.workunit.client.1.smithi100.stdout:0/909: write dd/d20/d44/f5e [292901,104543] 0 2024-03-18T14:51:14.931 INFO:tasks.workunit.client.1.smithi100.stdout:6/812: mkdir d6/d12/d103 0 2024-03-18T14:51:14.935 INFO:tasks.workunit.client.1.smithi100.stdout:3/898: symlink d3/d16/d13/l133 0 2024-03-18T14:51:14.936 INFO:tasks.workunit.client.1.smithi100.stdout:4/865: rmdir d9 39 2024-03-18T14:51:14.936 INFO:tasks.workunit.client.1.smithi100.stdout:4/866: readlink d9/d13/l44 0 2024-03-18T14:51:14.941 INFO:tasks.workunit.client.1.smithi100.stdout:5/814: creat d29/d2d/d39/d83/dc1/dbb/d9e/dd7/f107 x:0 0 0 2024-03-18T14:51:14.943 INFO:tasks.workunit.client.1.smithi100.stdout:7/898: link d5/d14/d16/d63/d124/dd1/d86/d85/d87/df6/ce7 d5/d30/d4d/db4/c148 0 2024-03-18T14:51:14.946 INFO:tasks.workunit.client.1.smithi100.stdout:8/973: dwrite d4/dc/d1f/d33/d128/d21/d23/d47/d56/ffd [0,4194304] 0 2024-03-18T14:51:14.946 INFO:tasks.workunit.client.1.smithi100.stdout:1/881: symlink d8/d1e/l11a 0 2024-03-18T14:51:14.949 INFO:tasks.workunit.client.1.smithi100.stdout:2/863: rename d3/da/de/d20/dea/d4f/fb5 to d3/d1a/d76/f11c 0 2024-03-18T14:51:14.949 INFO:tasks.workunit.client.1.smithi100.stdout:3/899: dwrite d3/f29 [4194304,4194304] 0 2024-03-18T14:51:14.949 INFO:tasks.workunit.client.1.smithi100.stdout:3/900: creat d3/dd/d21/dcd/d103/f134 x:0 0 0 2024-03-18T14:51:14.949 INFO:tasks.workunit.client.1.smithi100.stdout:3/901: readlink d3/d16/d13/l41 0 2024-03-18T14:51:14.953 INFO:tasks.workunit.client.1.smithi100.stdout:0/910: truncate dd/d20/f32 41722 0 2024-03-18T14:51:14.953 INFO:tasks.workunit.client.1.smithi100.stdout:0/911: chown dd 11995 1 2024-03-18T14:51:14.955 INFO:tasks.workunit.client.1.smithi100.stdout:9/928: rmdir da/dd/d29/d71/df5 39 2024-03-18T14:51:14.956 INFO:tasks.workunit.client.1.smithi100.stdout:1/882: dread d8/d2a/d37/f39 [0,4194304] 0 2024-03-18T14:51:14.957 INFO:tasks.workunit.client.1.smithi100.stdout:4/867: symlink d9/d13/d5a/d5b/df9/l105 0 2024-03-18T14:51:14.957 INFO:tasks.workunit.client.1.smithi100.stdout:4/868: fsync d9/d13/d5a/da4/fee 0 2024-03-18T14:51:14.958 INFO:tasks.workunit.client.1.smithi100.stdout:5/815: mknod d29/d2d/d39/d71/d96/c108 0 2024-03-18T14:51:14.962 INFO:tasks.workunit.client.1.smithi100.stdout:7/899: dwrite d5/d22/d3b/d47/fd5 [0,4194304] 0 2024-03-18T14:51:14.963 INFO:tasks.workunit.client.1.smithi100.stdout:8/974: unlink d4/dc/d1f/d33/d128/f53 0 2024-03-18T14:51:14.967 INFO:tasks.workunit.client.1.smithi100.stdout:2/864: symlink d3/da/db8/d111/l11d 0 2024-03-18T14:51:14.972 INFO:tasks.workunit.client.1.smithi100.stdout:3/902: mknod d3/d16/d13/d1f/d7c/c135 0 2024-03-18T14:51:14.973 INFO:tasks.workunit.client.1.smithi100.stdout:0/912: mknod dd/d31/d127/d131/c14b 0 2024-03-18T14:51:14.976 INFO:tasks.workunit.client.1.smithi100.stdout:6/813: rmdir d6/d7/df/d72/d8b 39 2024-03-18T14:51:14.978 INFO:tasks.workunit.client.1.smithi100.stdout:9/929: creat da/dd/df1/f12e x:0 0 0 2024-03-18T14:51:14.978 INFO:tasks.workunit.client.1.smithi100.stdout:9/930: fdatasync da/dd/d3b/d103/fe6 0 2024-03-18T14:51:14.978 INFO:tasks.workunit.client.1.smithi100.stdout:5/816: dwrite d29/d2d/d39/d61/ddd/f102 [0,4194304] 0 2024-03-18T14:51:14.978 INFO:tasks.workunit.client.1.smithi100.stdout:1/883: dwrite d8/d1e/d11/d27/d73/d89/f9d [0,4194304] 0 2024-03-18T14:51:14.980 INFO:tasks.workunit.client.1.smithi100.stdout:5/817: dread d29/d2d/d39/d71/f77 [0,4194304] 0 2024-03-18T14:51:14.980 INFO:tasks.workunit.client.1.smithi100.stdout:5/818: dread - d29/d2d/d32/d92/f9a zero size 2024-03-18T14:51:14.980 INFO:tasks.workunit.client.1.smithi100.stdout:5/819: fdatasync d29/d2d/d8f/fb8 0 2024-03-18T14:51:14.986 INFO:tasks.workunit.client.1.smithi100.stdout:4/869: creat d9/d13/d5a/da4/f106 x:0 0 0 2024-03-18T14:51:14.993 INFO:tasks.workunit.client.1.smithi100.stdout:7/900: dwrite d5/d14/d16/d63/d124/ff2 [0,4194304] 0 2024-03-18T14:51:14.993 INFO:tasks.workunit.client.1.smithi100.stdout:7/901: creat d5/d22/d24/dac/f149 x:0 0 0 2024-03-18T14:51:14.997 INFO:tasks.workunit.client.1.smithi100.stdout:2/865: mknod d3/d6/d29/d36/d49/c11e 0 2024-03-18T14:51:14.997 INFO:tasks.workunit.client.1.smithi100.stdout:3/903: dwrite d3/d16/d13/d1f/d3f/d4f/d65/feb [0,4194304] 0 2024-03-18T14:51:14.997 INFO:tasks.workunit.client.1.smithi100.stdout:3/904: stat d3/d16/d13/d1f/ca8 0 2024-03-18T14:51:14.999 INFO:tasks.workunit.client.1.smithi100.stdout:0/913: symlink dd/d31/d91/d56/de0/d113/l14c 0 2024-03-18T14:51:15.006 INFO:tasks.workunit.client.1.smithi100.stdout:1/884: dwrite d8/d1e/d11/d8a/dda/ff2 [0,4194304] 0 2024-03-18T14:51:15.006 INFO:tasks.workunit.client.1.smithi100.stdout:1/885: stat d8/d1e/d11/d6d/d59/cff 0 2024-03-18T14:51:15.011 INFO:tasks.workunit.client.1.smithi100.stdout:4/870: rmdir d9/d27/d42 39 2024-03-18T14:51:15.011 INFO:tasks.workunit.client.1.smithi100.stdout:4/871: dread - d9/d27/da2/faf zero size 2024-03-18T14:51:15.012 INFO:tasks.workunit.client.1.smithi100.stdout:4/872: truncate d9/d13/d5a/d5b/da0/dd8/fed 343929 0 2024-03-18T14:51:15.018 INFO:tasks.workunit.client.1.smithi100.stdout:4/873: read d9/fd7 [1216577,88178] 0 2024-03-18T14:51:15.018 INFO:tasks.workunit.client.1.smithi100.stdout:4/874: write d9/fb [1310201,6635] 0 2024-03-18T14:51:15.018 INFO:tasks.workunit.client.1.smithi100.stdout:4/875: creat d9/d13/d39/d76/da3/dca/f107 x:0 0 0 2024-03-18T14:51:15.018 INFO:tasks.workunit.client.1.smithi100.stdout:4/876: write d9/f7f [561342,116405] 0 2024-03-18T14:51:15.022 INFO:tasks.workunit.client.1.smithi100.stdout:7/902: symlink d5/d22/d3b/d9b/l14a 0 2024-03-18T14:51:15.022 INFO:tasks.workunit.client.1.smithi100.stdout:7/903: write d5/d30/d4d/ff8 [903561,51529] 0 2024-03-18T14:51:15.022 INFO:tasks.workunit.client.1.smithi100.stdout:7/904: chown d5/d14/d16/d63/d124/dd1/d86/d85/d87/lff 1069 1 2024-03-18T14:51:15.028 INFO:tasks.workunit.client.1.smithi100.stdout:6/814: write f4 [5745687,114123] 0 2024-03-18T14:51:15.028 INFO:tasks.workunit.client.1.smithi100.stdout:6/815: fdatasync d6/d7/df/d26/f45 0 2024-03-18T14:51:15.032 INFO:tasks.workunit.client.1.smithi100.stdout:8/975: rename d4/dc/d1f/d33/d128/d19/d103 to d4/d157 0 2024-03-18T14:51:15.036 INFO:tasks.workunit.client.1.smithi100.stdout:2/866: mkdir d3/da/de/d11f 0 2024-03-18T14:51:15.036 INFO:tasks.workunit.client.1.smithi100.stdout:2/867: dread - d3/dc3/ff2 zero size 2024-03-18T14:51:15.037 INFO:tasks.workunit.client.1.smithi100.stdout:3/905: rmdir d3/d16/dca/d10e 39 2024-03-18T14:51:15.037 INFO:tasks.workunit.client.1.smithi100.stdout:2/868: write d3/d6/d29/d36/ddf/fed [1239204,12286] 0 2024-03-18T14:51:15.037 INFO:tasks.workunit.client.1.smithi100.stdout:6/816: dread d6/d12/d2c/fc4 [4194304,4194304] 0 2024-03-18T14:51:15.044 INFO:tasks.workunit.client.1.smithi100.stdout:0/914: unlink dd/d31/d91/d56/d71/cb2 0 2024-03-18T14:51:15.048 INFO:tasks.workunit.client.1.smithi100.stdout:8/976: dwrite d4/dc/d1f/d33/d128/d19/d37/f137 [4194304,4194304] 0 2024-03-18T14:51:15.051 INFO:tasks.workunit.client.1.smithi100.stdout:8/977: dread d4/dc/d1f/d33/d128/f63 [0,4194304] 0 2024-03-18T14:51:15.060 INFO:tasks.workunit.client.1.smithi100.stdout:1/886: truncate d8/d30/d61/d8d/fc1 4181773 0 2024-03-18T14:51:15.069 INFO:tasks.workunit.client.1.smithi100.stdout:4/877: rename d9/d13/d5a/c87 to d9/c108 0 2024-03-18T14:51:15.072 INFO:tasks.workunit.client.1.smithi100.stdout:3/906: creat d3/d16/d13/d1f/d37/f136 x:0 0 0 2024-03-18T14:51:15.072 INFO:tasks.workunit.client.1.smithi100.stdout:3/907: creat d3/d16/d13/d1f/d3f/d4f/de0/de6/f137 x:0 0 0 2024-03-18T14:51:15.072 INFO:tasks.workunit.client.1.smithi100.stdout:3/908: chown d3/d16/d13/d1f/f78 44182786 1 2024-03-18T14:51:15.072 INFO:tasks.workunit.client.1.smithi100.stdout:3/909: chown d3/da5/fc7 6760775 1 2024-03-18T14:51:15.073 INFO:tasks.workunit.client.1.smithi100.stdout:2/869: mknod d3/da/db8/c120 0 2024-03-18T14:51:15.074 INFO:tasks.workunit.client.1.smithi100.stdout:2/870: read d3/d6/d29/d36/d49/f51 [1103904,27113] 0 2024-03-18T14:51:15.074 INFO:tasks.workunit.client.1.smithi100.stdout:2/871: fsync d3/d6/d5f/f6d 0 2024-03-18T14:51:15.074 INFO:tasks.workunit.client.1.smithi100.stdout:4/878: dread d9/f32 [4194304,4194304] 0 2024-03-18T14:51:15.076 INFO:tasks.workunit.client.1.smithi100.stdout:0/915: link dd/d31/d91/d7b/fad dd/d31/d91/d65/d143/f14d 0 2024-03-18T14:51:15.076 INFO:tasks.workunit.client.1.smithi100.stdout:0/916: chown dd/l11b 15359882 1 2024-03-18T14:51:15.077 INFO:tasks.workunit.client.1.smithi100.stdout:4/879: dread d9/d13/d39/d76/f94 [0,4194304] 0 2024-03-18T14:51:15.078 INFO:tasks.workunit.client.1.smithi100.stdout:1/887: dwrite d8/d2a/d37/f5f [4194304,4194304] 0 2024-03-18T14:51:15.078 INFO:tasks.workunit.client.1.smithi100.stdout:1/888: fdatasync d8/d2a/d37/f5f 0 2024-03-18T14:51:15.080 INFO:tasks.workunit.client.1.smithi100.stdout:8/978: mknod d4/d5a/c158 0 2024-03-18T14:51:15.081 INFO:tasks.workunit.client.1.smithi100.stdout:8/979: chown d4/dc/d1f/d33/d128/d19/db0/dd5/f104 3 1 2024-03-18T14:51:15.084 INFO:tasks.workunit.client.1.smithi100.stdout:9/931: write da/dd/d3b/d103/db0/feb [253749,35650] 0 2024-03-18T14:51:15.089 INFO:tasks.workunit.client.1.smithi100.stdout:1/889: dread d8/d2a/d37/d50/f85 [0,4194304] 0 2024-03-18T14:51:15.093 INFO:tasks.workunit.client.1.smithi100.stdout:3/910: dwrite d3/dd/d49/fb7 [0,4194304] 0 2024-03-18T14:51:15.095 INFO:tasks.workunit.client.1.smithi100.stdout:6/817: rename d6/d7/df/d26/f56 to d6/d12/da2/f104 0 2024-03-18T14:51:15.101 INFO:tasks.workunit.client.1.smithi100.stdout:4/880: mkdir d9/d27/d62/d109 0 2024-03-18T14:51:15.101 INFO:tasks.workunit.client.1.smithi100.stdout:4/881: readlink d9/d27/d62/d7b/l7e 0 2024-03-18T14:51:15.103 INFO:tasks.workunit.client.1.smithi100.stdout:1/890: dwrite d8/d1e/d11/d27/f8c [0,4194304] 0 2024-03-18T14:51:15.108 INFO:tasks.workunit.client.1.smithi100.stdout:6/818: dread d6/d7/df/f7c [0,4194304] 0 2024-03-18T14:51:15.108 INFO:tasks.workunit.client.1.smithi100.stdout:6/819: truncate d6/d7/df/d26/d4e/d53/fdc 499181 0 2024-03-18T14:51:15.108 INFO:tasks.workunit.client.1.smithi100.stdout:8/980: symlink d4/d5a/l159 0 2024-03-18T14:51:15.108 INFO:tasks.workunit.client.1.smithi100.stdout:8/981: write d4/dc/d1f/d33/d128/d19/d2c/f13b [880817,54715] 0 2024-03-18T14:51:15.109 INFO:tasks.workunit.client.1.smithi100.stdout:9/932: getdents da/dd/d29 0 2024-03-18T14:51:15.117 INFO:tasks.workunit.client.1.smithi100.stdout:3/911: symlink d3/dd/d21/d3e/df8/db9/df0/l138 0 2024-03-18T14:51:15.122 INFO:tasks.workunit.client.1.smithi100.stdout:6/820: dwrite d6/d7/df/d26/d4e/d66/d36/d4f/db8/ff6 [0,4194304] 0 2024-03-18T14:51:15.122 INFO:tasks.workunit.client.1.smithi100.stdout:6/821: write d6/d12/d2c/fc1 [428108,48825] 0 2024-03-18T14:51:15.128 INFO:tasks.workunit.client.1.smithi100.stdout:2/872: rename d3/da/de/d20/f90 to d3/da/de/d20/dea/d40/d7d/f121 0 2024-03-18T14:51:15.128 INFO:tasks.workunit.client.1.smithi100.stdout:2/873: fsync d3/da/de/d20/dea/d40/f104 0 2024-03-18T14:51:15.132 INFO:tasks.workunit.client.1.smithi100.stdout:4/882: unlink d9/d13/lc4 0 2024-03-18T14:51:15.132 INFO:tasks.workunit.client.1.smithi100.stdout:4/883: write d9/d13/fe8 [508196,115957] 0 2024-03-18T14:51:15.132 INFO:tasks.workunit.client.1.smithi100.stdout:4/884: readlink d9/d13/d39/d76/lb8 0 2024-03-18T14:51:15.132 INFO:tasks.workunit.client.1.smithi100.stdout:4/885: chown d9/d13/d5a/d5b/da0/dd8/df0/cfb 2362 1 2024-03-18T14:51:15.134 INFO:tasks.workunit.client.1.smithi100.stdout:2/874: dread d3/d6/d5f/fc6 [0,4194304] 0 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/912: dwrite d3/d16/d13/d17/f7f [0,4194304] 0 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/913: write d3/dd/d49/f8f [1691000,36623] 0 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/914: write d3/dd/d49/f87 [4283853,119118] 0 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/915: dread - d3/d16/d13/d1f/d3f/d4f/de0/de6/f113 zero size 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/916: truncate d3/dd/d49/f87 5181204 0 2024-03-18T14:51:15.136 INFO:tasks.workunit.client.1.smithi100.stdout:3/917: stat d3/d16/f45 0 2024-03-18T14:51:15.137 INFO:tasks.workunit.client.1.smithi100.stdout:3/918: fdatasync d3/dd/f38 0 2024-03-18T14:51:15.137 INFO:tasks.workunit.client.1.smithi100.stdout:3/919: truncate d3/dd/d21/d3e/df8/db9/fb1 678437 0 2024-03-18T14:51:15.137 INFO:tasks.workunit.client.1.smithi100.stdout:3/920: chown d3/dd/l19 939392 1 2024-03-18T14:51:15.142 INFO:tasks.workunit.client.1.smithi100.stdout:1/891: unlink d8/d1e/c58 0 2024-03-18T14:51:15.145 INFO:tasks.workunit.client.1.smithi100.stdout:8/982: creat d4/d96/f15a x:0 0 0 2024-03-18T14:51:15.147 INFO:tasks.workunit.client.1.smithi100.stdout:8/983: dread d4/dc/d1f/d59/f64 [0,4194304] 0 2024-03-18T14:51:15.147 INFO:tasks.workunit.client.1.smithi100.stdout:8/984: truncate f1 5493659 0 2024-03-18T14:51:15.152 INFO:tasks.workunit.client.1.smithi100.stdout:6/822: creat d6/d7/df/d26/f105 x:0 0 0 2024-03-18T14:51:15.152 INFO:tasks.workunit.client.1.smithi100.stdout:6/823: chown d6/d7/df/d26/d4e/d66/d36/c99 45 1 2024-03-18T14:51:15.152 INFO:tasks.workunit.client.1.smithi100.stdout:6/824: stat d6/d7/df/d26/d4e/d66/d36/l60 0 2024-03-18T14:51:15.152 INFO:tasks.workunit.client.1.smithi100.stdout:1/892: dread d8/d1e/d11/d27/d73/d89/fa4 [4194304,4194304] 0 2024-03-18T14:51:15.155 INFO:tasks.workunit.client.1.smithi100.stdout:0/917: rename dd/d31/f10b to dd/d61/dcd/d140/f14e 0 2024-03-18T14:51:15.155 INFO:tasks.workunit.client.1.smithi100.stdout:0/918: chown dd/d2b/cd4 4605406 1 2024-03-18T14:51:15.157 INFO:tasks.workunit.client.1.smithi100.stdout:4/886: dwrite d9/d3a/fe4 [0,4194304] 0 2024-03-18T14:51:15.157 INFO:tasks.workunit.client.1.smithi100.stdout:4/887: fdatasync d9/d13/d25/fa5 0 2024-03-18T14:51:15.159 INFO:tasks.workunit.client.1.smithi100.stdout:8/985: write d4/dc/d1f/d59/d6c/f98 [3387940,5869] 0 2024-03-18T14:51:15.159 INFO:tasks.workunit.client.1.smithi100.stdout:8/986: dread - d4/dc/d31/f122 zero size 2024-03-18T14:51:15.160 INFO:tasks.workunit.client.1.smithi100.stdout:3/921: stat d3/dd/d21/c11e 0 2024-03-18T14:51:15.161 INFO:tasks.workunit.client.1.smithi100.stdout:2/875: dwrite d3/da/de/d20/f23 [0,4194304] 0 2024-03-18T14:51:15.163 INFO:tasks.workunit.client.1.smithi100.stdout:6/825: symlink d6/d12/l106 0 2024-03-18T14:51:15.166 INFO:tasks.workunit.client.1.smithi100.stdout:4/888: creat d9/d27/d42/f10a x:0 0 0 2024-03-18T14:51:15.166 INFO:tasks.workunit.client.1.smithi100.stdout:4/889: write d9/d17/fdd [813517,963] 0 2024-03-18T14:51:15.168 INFO:tasks.workunit.client.1.smithi100.stdout:8/987: unlink d4/dc/d1f/d33/d128/d19/d2c/fec 0 2024-03-18T14:51:15.168 INFO:tasks.workunit.client.1.smithi100.stdout:8/988: truncate d4/dc/d1f/d33/d128/d19/dc0/d113/f12a 764565 0 2024-03-18T14:51:15.168 INFO:tasks.workunit.client.1.smithi100.stdout:0/919: dread dd/d31/fe6 [0,4194304] 0 2024-03-18T14:51:15.170 INFO:tasks.workunit.client.1.smithi100.stdout:2/876: truncate d3/d6/d29/d36/d49/f7a 635952 0 2024-03-18T14:51:15.190 INFO:tasks.workunit.client.1.smithi100.stdout:1/893: rename d8/dd/c5c to d8/d2a/d37/d50/d6f/d71/db7/c11b 0 2024-03-18T14:51:15.193 INFO:tasks.workunit.client.1.smithi100.stdout:0/920: symlink dd/d20/d44/l14f 0 2024-03-18T14:51:15.193 INFO:tasks.workunit.client.1.smithi100.stdout:0/921: chown dd/c19 40 1 2024-03-18T14:51:15.193 INFO:tasks.workunit.client.1.smithi100.stdout:0/922: write dd/d20/f32 [307189,73876] 0 2024-03-18T14:51:15.194 INFO:tasks.workunit.client.1.smithi100.stdout:0/923: chown dd/d31/d91/d65/d143 1 1 2024-03-18T14:51:15.194 INFO:tasks.workunit.client.1.smithi100.stdout:3/922: mknod d3/c139 0 2024-03-18T14:51:15.197 INFO:tasks.workunit.client.1.smithi100.stdout:1/894: dread d8/f7e [0,4194304] 0 2024-03-18T14:51:15.198 INFO:tasks.workunit.client.1.smithi100.stdout:1/895: creat d8/d1e/d11/d27/d73/f11c x:0 0 0 2024-03-18T14:51:15.198 INFO:tasks.workunit.client.1.smithi100.stdout:1/896: chown d8/d30/d61/f63 335869 1 2024-03-18T14:51:15.198 INFO:tasks.workunit.client.1.smithi100.stdout:2/877: mknod d3/d8c/c122 0 2024-03-18T14:51:15.198 INFO:tasks.workunit.client.1.smithi100.stdout:2/878: creat d3/da/de/d20/d82/dad/de2/f123 x:0 0 0 2024-03-18T14:51:15.200 INFO:tasks.workunit.client.1.smithi100.stdout:6/826: getdents d6 0 2024-03-18T14:51:15.200 INFO:tasks.workunit.client.1.smithi100.stdout:6/827: chown d6/d12/d78/ce6 1 1 2024-03-18T14:51:15.200 INFO:tasks.workunit.client.1.smithi100.stdout:6/828: write d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [4836223,56721] 0 2024-03-18T14:51:15.200 INFO:tasks.workunit.client.1.smithi100.stdout:6/829: fdatasync d6/d12/d2c/f7e 0 2024-03-18T14:51:15.201 INFO:tasks.workunit.client.1.smithi100.stdout:4/890: rename d9/d13/d5a/da4/fee to d9/d13/d39/f10b 0 2024-03-18T14:51:15.203 INFO:tasks.workunit.client.1.smithi100.stdout:0/924: truncate dd/d61/fea 3368432 0 2024-03-18T14:51:15.205 INFO:tasks.workunit.client.1.smithi100.stdout:3/923: unlink d3/d16/d13/d1f/d3f/d4f/d60/da9/f11b 0 2024-03-18T14:51:15.209 INFO:tasks.workunit.client.1.smithi100.stdout:5/820: sync 2024-03-18T14:51:15.209 INFO:tasks.workunit.client.1.smithi100.stdout:1/897: truncate d8/d1e/d11/d27/f8c 1912675 0 2024-03-18T14:51:15.209 INFO:tasks.workunit.client.1.smithi100.stdout:7/905: sync 2024-03-18T14:51:15.209 INFO:tasks.workunit.client.1.smithi100.stdout:7/906: creat d5/d90/da1/d128/f14b x:0 0 0 2024-03-18T14:51:15.213 INFO:tasks.workunit.client.1.smithi100.stdout:2/879: creat d3/da/db8/f124 x:0 0 0 2024-03-18T14:51:15.218 INFO:tasks.workunit.client.1.smithi100.stdout:2/880: write d3/da/de/d20/dea/f38 [3912982,32677] 0 2024-03-18T14:51:15.219 INFO:tasks.workunit.client.1.smithi100.stdout:8/989: rename d4/dc/d1f/d59/d6c/d75 to d4/dc/d1f/d33/d128/d21/d23/d47/d54/d124/d15b 0 2024-03-18T14:51:15.223 INFO:tasks.workunit.client.1.smithi100.stdout:0/925: mkdir dd/d2b/d95/d150 0 2024-03-18T14:51:15.224 INFO:tasks.workunit.client.1.smithi100.stdout:0/926: write dd/d31/d91/d56/d71/f109 [716187,51126] 0 2024-03-18T14:51:15.227 INFO:tasks.workunit.client.1.smithi100.stdout:3/924: mknod d3/d16/d13/d1f/d3f/d4f/de0/c13a 0 2024-03-18T14:51:15.227 INFO:tasks.workunit.client.1.smithi100.stdout:3/925: creat d3/dd/d10d/f13b x:0 0 0 2024-03-18T14:51:15.227 INFO:tasks.workunit.client.1.smithi100.stdout:3/926: truncate d3/dd/d49/f72 934783 0 2024-03-18T14:51:15.228 INFO:tasks.workunit.client.1.smithi100.stdout:3/927: write d3/da5/ff9 [1041441,125826] 0 2024-03-18T14:51:15.229 INFO:tasks.workunit.client.1.smithi100.stdout:5/821: symlink d29/d2d/d39/dfe/l109 0 2024-03-18T14:51:15.235 INFO:tasks.workunit.client.1.smithi100.stdout:7/907: mknod d5/d14/d16/d63/d124/d66/c14c 0 2024-03-18T14:51:15.236 INFO:tasks.workunit.client.1.smithi100.stdout:7/908: readlink d5/d30/l137 0 2024-03-18T14:51:15.237 INFO:tasks.workunit.client.1.smithi100.stdout:7/909: write d5/d22/d24/d6e/d7a/fe3 [417157,125424] 0 2024-03-18T14:51:15.237 INFO:tasks.workunit.client.1.smithi100.stdout:6/830: rmdir d6/d7/df/d26/d4e/d66/d36/d4f/db8 39 2024-03-18T14:51:15.240 INFO:tasks.workunit.client.1.smithi100.stdout:2/881: creat d3/da/db8/d111/f125 x:0 0 0 2024-03-18T14:51:15.241 INFO:tasks.workunit.client.1.smithi100.stdout:4/891: rename d9/d27/d62/d7b/cba to d9/d13/d39/d76/da3/dca/c10c 0 2024-03-18T14:51:15.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/928: mknod d3/d16/d13/d1f/d3f/d4f/c13c 0 2024-03-18T14:51:15.244 INFO:tasks.workunit.client.1.smithi100.stdout:3/929: write d3/dd/d49/f87 [575455,42594] 0 2024-03-18T14:51:15.245 INFO:tasks.workunit.client.1.smithi100.stdout:1/898: mkdir d8/d2a/d37/d50/d11d 0 2024-03-18T14:51:15.246 INFO:tasks.workunit.client.1.smithi100.stdout:6/831: unlink d6/d7/df/d26/d4e/d66/d36/d4f/d8a/l92 0 2024-03-18T14:51:15.247 INFO:tasks.workunit.client.1.smithi100.stdout:0/927: rename dd/d31/d91/d65/ff5 to dd/d31/d91/d56/d121/f151 0 2024-03-18T14:51:15.248 INFO:tasks.workunit.client.1.smithi100.stdout:0/928: dread - dd/d2b/fc0 zero size 2024-03-18T14:51:15.248 INFO:tasks.workunit.client.1.smithi100.stdout:6/832: mkdir d6/d7/df/d26/d4e/db5/de1/d107 0 2024-03-18T14:51:15.249 INFO:tasks.workunit.client.1.smithi100.stdout:6/833: dread - d6/d7/df/d72/d8b/ff5 zero size 2024-03-18T14:51:15.250 INFO:tasks.workunit.client.1.smithi100.stdout:4/892: rename d9/d13/d5a/d5b/da0/dcd/fd3 to d9/d27/d62/d109/f10d 0 2024-03-18T14:51:15.251 INFO:tasks.workunit.client.1.smithi100.stdout:0/929: truncate dd/d31/f57 385457 0 2024-03-18T14:51:15.251 INFO:tasks.workunit.client.1.smithi100.stdout:4/893: write d9/d3a/fc6 [2564791,119222] 0 2024-03-18T14:51:15.253 INFO:tasks.workunit.client.1.smithi100.stdout:3/930: rename d3/d16/l9d to d3/d16/d13/d1f/l13d 0 2024-03-18T14:51:15.254 INFO:tasks.workunit.client.1.smithi100.stdout:8/990: dwrite d4/dc/d1f/d33/d128/d19/d37/f137 [4194304,4194304] 0 2024-03-18T14:51:15.254 INFO:tasks.workunit.client.1.smithi100.stdout:0/930: symlink dd/d20/ddc/l152 0 2024-03-18T14:51:15.255 INFO:tasks.workunit.client.1.smithi100.stdout:5/822: dwrite d29/f34 [0,4194304] 0 2024-03-18T14:51:15.255 INFO:tasks.workunit.client.1.smithi100.stdout:4/894: creat d9/d13/d39/dfd/f10e x:0 0 0 2024-03-18T14:51:15.255 INFO:tasks.workunit.client.1.smithi100.stdout:2/882: dwrite d3/da/de/d20/d82/fd8 [0,4194304] 0 2024-03-18T14:51:15.257 INFO:tasks.workunit.client.1.smithi100.stdout:6/834: rmdir d6/d12 39 2024-03-18T14:51:15.260 INFO:tasks.workunit.client.1.smithi100.stdout:1/899: rename d8/d1e/d11/d6d/d59/c80 to d8/df5/c11e 0 2024-03-18T14:51:15.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/931: write dd/d20/f5f [1858158,17493] 0 2024-03-18T14:51:15.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/932: fsync dd/d61/fde 0 2024-03-18T14:51:15.263 INFO:tasks.workunit.client.1.smithi100.stdout:0/933: write dd/d2b/d11e/fbe [1409806,39088] 0 2024-03-18T14:51:15.264 INFO:tasks.workunit.client.1.smithi100.stdout:8/991: mkdir d4/dc2/d15c 0 2024-03-18T14:51:15.265 INFO:tasks.workunit.client.1.smithi100.stdout:5/823: symlink d29/d2d/d39/d83/dc1/l10a 0 2024-03-18T14:51:15.265 INFO:tasks.workunit.client.1.smithi100.stdout:5/824: readlink l1d 0 2024-03-18T14:51:15.275 INFO:tasks.workunit.client.1.smithi100.stdout:6/835: symlink d6/d7/df/df1/l108 0 2024-03-18T14:51:15.280 INFO:tasks.workunit.client.1.smithi100.stdout:2/883: rename d3/d6/d5f/d6a/fc9 to d3/da/de/d20/dea/d40/dac/f126 0 2024-03-18T14:51:15.284 INFO:tasks.workunit.client.1.smithi100.stdout:0/934: link dd/d31/d91/d56/de0/f12c dd/d31/d91/d56/d62/f153 0 2024-03-18T14:51:15.285 INFO:tasks.workunit.client.1.smithi100.stdout:0/935: read dd/d20/db5/f6d [223477,95190] 0 2024-03-18T14:51:15.285 INFO:tasks.workunit.client.1.smithi100.stdout:6/836: dread d6/d7/df/d72/f85 [0,4194304] 0 2024-03-18T14:51:15.287 INFO:tasks.workunit.client.1.smithi100.stdout:0/936: write dd/d31/d91/d56/d62/fed [824469,126217] 0 2024-03-18T14:51:15.287 INFO:tasks.workunit.client.1.smithi100.stdout:0/937: chown dd/d31/d91/d56/d59/c83 185 1 2024-03-18T14:51:15.287 INFO:tasks.workunit.client.1.smithi100.stdout:3/931: dwrite d3/fd5 [4194304,4194304] 0 2024-03-18T14:51:15.289 INFO:tasks.workunit.client.1.smithi100.stdout:4/895: dwrite d9/d3a/f69 [0,4194304] 0 2024-03-18T14:51:15.289 INFO:tasks.workunit.client.1.smithi100.stdout:7/910: dwrite d5/d22/d24/d6e/d7a/fe3 [0,4194304] 0 2024-03-18T14:51:15.290 INFO:tasks.workunit.client.1.smithi100.stdout:4/896: dread d9/d13/d39/d76/f94 [0,4194304] 0 2024-03-18T14:51:15.293 INFO:tasks.workunit.client.1.smithi100.stdout:8/992: rename d4/dc/dd/c83 to d4/dc/d1f/c15d 0 2024-03-18T14:51:15.293 INFO:tasks.workunit.client.1.smithi100.stdout:8/993: chown d4/dc/d97/l12f 1251153 1 2024-03-18T14:51:15.295 INFO:tasks.workunit.client.1.smithi100.stdout:2/884: truncate d3/d6/f44 2107551 0 2024-03-18T14:51:15.304 INFO:tasks.workunit.client.1.smithi100.stdout:3/932: write d3/d16/d13/f3b [3103028,126218] 0 2024-03-18T14:51:15.304 INFO:tasks.workunit.client.1.smithi100.stdout:1/900: dwrite d8/fb6 [0,4194304] 0 2024-03-18T14:51:15.306 INFO:tasks.workunit.client.1.smithi100.stdout:6/837: dread d6/d7/df/d26/d4e/d66/d36/d4f/d71/f5b [0,4194304] 0 2024-03-18T14:51:15.307 INFO:tasks.workunit.client.1.smithi100.stdout:7/911: creat d5/d14/d16/d63/d124/d66/d4f/dcb/f14d x:0 0 0 2024-03-18T14:51:15.307 INFO:tasks.workunit.client.1.smithi100.stdout:7/912: readlink d5/d14/d16/d63/d124/dd1/la3 0 2024-03-18T14:51:15.308 INFO:tasks.workunit.client.1.smithi100.stdout:7/913: dread - d5/d22/d3b/d9f/da0/db1/dd3/fe0 zero size 2024-03-18T14:51:15.308 INFO:tasks.workunit.client.1.smithi100.stdout:4/897: mknod d9/d27/d62/c10f 0 2024-03-18T14:51:15.311 INFO:tasks.workunit.client.1.smithi100.stdout:7/914: read d5/d14/d16/d63/d124/dfe/f117 [1651815,90879] 0 2024-03-18T14:51:15.311 INFO:tasks.workunit.client.1.smithi100.stdout:6/838: dread d6/d12/d78/fb2 [0,4194304] 0 2024-03-18T14:51:15.313 INFO:tasks.workunit.client.1.smithi100.stdout:0/938: rename dd/d61/fe8 to dd/d31/d91/d65/f154 0 2024-03-18T14:51:15.314 INFO:tasks.workunit.client.1.smithi100.stdout:8/994: mkdir d4/dc/d1f/d33/d128/d21/d15e 0 2024-03-18T14:51:15.315 INFO:tasks.workunit.client.1.smithi100.stdout:2/885: mknod d3/da/de/d20/d107/c127 0 2024-03-18T14:51:15.315 INFO:tasks.workunit.client.1.smithi100.stdout:2/886: chown d3/da/de/f3c 13545 1 2024-03-18T14:51:15.315 INFO:tasks.workunit.client.1.smithi100.stdout:2/887: stat d3/c7 0 2024-03-18T14:51:15.322 INFO:tasks.workunit.client.1.smithi100.stdout:1/901: dwrite d8/f52 [0,4194304] 0 2024-03-18T14:51:15.322 INFO:tasks.workunit.client.1.smithi100.stdout:1/902: write d8/fee [797111,60684] 0 2024-03-18T14:51:15.325 INFO:tasks.workunit.client.1.smithi100.stdout:2/888: read d3/d6/d29/d36/f4e [1837851,48548] 0 2024-03-18T14:51:15.325 INFO:tasks.workunit.client.1.smithi100.stdout:2/889: fdatasync d3/d6/d5f/f68 0 2024-03-18T14:51:15.327 INFO:tasks.workunit.client.1.smithi100.stdout:4/898: mkdir d9/d13/d39/dfd/d110 0 2024-03-18T14:51:15.327 INFO:tasks.workunit.client.1.smithi100.stdout:4/899: truncate d9/d17/f34 7962875 0 2024-03-18T14:51:15.329 INFO:tasks.workunit.client.1.smithi100.stdout:7/915: dwrite d5/d22/d3b/d9f/da0/db1/dd3/fe0 [0,4194304] 0 2024-03-18T14:51:15.330 INFO:tasks.workunit.client.1.smithi100.stdout:6/839: symlink d6/d7/df/d26/d4e/d66/d36/d4f/d71/l109 0 2024-03-18T14:51:15.330 INFO:tasks.workunit.client.1.smithi100.stdout:2/890: dread d3/f10 [0,4194304] 0 2024-03-18T14:51:15.331 INFO:tasks.workunit.client.1.smithi100.stdout:8/995: truncate d4/dc/dd/f44 1601865 0 2024-03-18T14:51:15.332 INFO:tasks.workunit.client.1.smithi100.stdout:1/903: mknod d8/d1e/d11/d8a/c11f 0 2024-03-18T14:51:15.337 INFO:tasks.workunit.client.1.smithi100.stdout:7/916: symlink d5/d30/d4d/db4/d116/l14e 0 2024-03-18T14:51:15.342 INFO:tasks.workunit.client.1.smithi100.stdout:6/840: creat d6/d7/df/d26/d4e/d66/d36/d4f/de4/f10a x:0 0 0 2024-03-18T14:51:15.346 INFO:tasks.workunit.client.1.smithi100.stdout:2/891: rename d3/d6/d29/d36/d49/c61 to d3/d6/c128 0 2024-03-18T14:51:15.348 INFO:tasks.workunit.client.1.smithi100.stdout:9/933: sync 2024-03-18T14:51:15.349 INFO:tasks.workunit.client.1.smithi100.stdout:9/934: getdents da/d2c/d2f/d114 0 2024-03-18T14:51:15.349 INFO:tasks.workunit.client.1.smithi100.stdout:8/996: stat d4/dc/d1f/d33/d128/d21/d23/d47/d54/l149 0 2024-03-18T14:51:15.352 INFO:tasks.workunit.client.1.smithi100.stdout:1/904: unlink d8/d30/d61/d8d/le2 0 2024-03-18T14:51:15.354 INFO:tasks.workunit.client.1.smithi100.stdout:6/841: truncate d6/fb3 227937 0 2024-03-18T14:51:15.355 INFO:tasks.workunit.client.1.smithi100.stdout:6/842: write d6/d7/df/d26/d4e/d66/fd9 [1762115,111680] 0 2024-03-18T14:51:15.358 INFO:tasks.workunit.client.1.smithi100.stdout:9/935: creat da/dd/d29/d71/d72/db2/d119/f12f x:0 0 0 2024-03-18T14:51:15.360 INFO:tasks.workunit.client.1.smithi100.stdout:8/997: symlink d4/dc/d1f/d33/l15f 0 2024-03-18T14:51:15.360 INFO:tasks.workunit.client.1.smithi100.stdout:9/936: write da/dd/d3e/d11e/d4d/fad [4747577,25719] 0 2024-03-18T14:51:15.362 INFO:tasks.workunit.client.1.smithi100.stdout:7/917: dwrite d5/d22/d24/d6e/d7a/f89 [0,4194304] 0 2024-03-18T14:51:15.362 INFO:tasks.workunit.client.1.smithi100.stdout:7/918: chown d5/d14/d16/d63/d124/dc3/d110 12490 1 2024-03-18T14:51:15.362 INFO:tasks.workunit.client.1.smithi100.stdout:7/919: write d5/d22/d3b/d9f/da0/db1/dd3/f114 [1383858,10427] 0 2024-03-18T14:51:15.362 INFO:tasks.workunit.client.1.smithi100.stdout:2/892: dwrite d3/da/de/d6e/fcd [0,4194304] 0 2024-03-18T14:51:15.366 INFO:tasks.workunit.client.1.smithi100.stdout:6/843: creat d6/d7/df/d26/d4e/d66/d36/d4f/d71/d7f/f10b x:0 0 0 2024-03-18T14:51:15.367 INFO:tasks.workunit.client.1.smithi100.stdout:6/844: chown d6/d7/df/d26/d4e/d66/d36/d4f/cef 0 1 2024-03-18T14:51:15.368 INFO:tasks.workunit.client.1.smithi100.stdout:8/998: creat d4/dc/d1f/d33/f160 x:0 0 0 2024-03-18T14:51:15.369 INFO:tasks.workunit.client.1.smithi100.stdout:9/937: dread da/d88/da9/fc7 [0,4194304] 0 2024-03-18T14:51:15.369 INFO:tasks.workunit.client.1.smithi100.stdout:9/938: readlink da/l57 0 2024-03-18T14:51:15.369 INFO:tasks.workunit.client.1.smithi100.stdout:9/939: write da/dd/d29/d71/df5/f124 [664344,9285] 0 2024-03-18T14:51:15.373 INFO:tasks.workunit.client.1.smithi100.stdout:1/905: rmdir d8/d1e/d11/d2d 39 2024-03-18T14:51:15.373 INFO:tasks.workunit.client.1.smithi100.stdout:1/906: write d8/d1e/d11/f60 [392606,7148] 0 2024-03-18T14:51:15.373 INFO:tasks.workunit.client.1.smithi100.stdout:1/907: fsync d8/fb6 0 2024-03-18T14:51:15.377 INFO:tasks.workunit.client.1.smithi100.stdout:2/893: symlink d3/da/de/d20/d82/dad/dc5/l129 0 2024-03-18T14:51:15.377 INFO:tasks.workunit.client.1.smithi100.stdout:2/894: chown d3/da/de/c31 205753 1 2024-03-18T14:51:15.377 INFO:tasks.workunit.client.1.smithi100.stdout:2/895: creat d3/da/de/d20/dea/d40/d7d/f12a x:0 0 0 2024-03-18T14:51:15.378 INFO:tasks.workunit.client.1.smithi100.stdout:2/896: readlink d3/da/de/d20/d82/d95/la0 0 2024-03-18T14:51:15.380 INFO:tasks.workunit.client.1.smithi100.stdout:8/999: truncate d4/dc/d1f/d33/d128/f63 1972235 0 2024-03-18T14:51:15.381 INFO:tasks.workunit.client.1.smithi100.stdout:9/940: mknod da/dd/d3b/d103/c130 0 2024-03-18T14:51:15.383 INFO:tasks.workunit.client.1.smithi100.stdout:6/845: dwrite d6/d12/da2/faf [0,4194304] 0 2024-03-18T14:51:15.383 INFO:tasks.workunit.client.1.smithi100.stdout:6/846: fdatasync d6/f6c 0 2024-03-18T14:51:15.384 INFO:tasks.workunit.client.1.smithi100.stdout:1/908: rename d8/d1e/d11/f60 to d8/d1e/d11/d27/d3f/daa/dd5/f120 0 2024-03-18T14:51:15.384 INFO:tasks.workunit.client.1.smithi100.stdout:1/909: chown d8/d1e/d11/d8a/f97 171 1 2024-03-18T14:51:15.384 INFO:tasks.workunit.client.1.smithi100.stdout:1/910: write d8/d30/d61/d8d/dde/de6/fe7 [207405,18201] 0 2024-03-18T14:51:15.391 INFO:tasks.workunit.client.1.smithi100.stdout:2/897: creat d3/da/de/d20/d107/f12b x:0 0 0 2024-03-18T14:51:15.392 INFO:tasks.workunit.client.1.smithi100.stdout:6/847: mknod d6/c10c 0 2024-03-18T14:51:15.394 INFO:tasks.workunit.client.1.smithi100.stdout:9/941: rename da/dd/fec to da/dd/d3e/d11e/dd6/f131 0 2024-03-18T14:51:15.396 INFO:tasks.workunit.client.1.smithi100.stdout:2/898: rmdir d3/d6/d29 39 2024-03-18T14:51:15.396 INFO:tasks.workunit.client.1.smithi100.stdout:2/899: truncate d3/da/de/d20/d82/fee 686416 0 2024-03-18T14:51:15.397 INFO:tasks.workunit.client.1.smithi100.stdout:2/900: read d3/da/de/f28 [3553560,46076] 0 2024-03-18T14:51:15.397 INFO:tasks.workunit.client.1.smithi100.stdout:6/848: mknod d6/d7/df/d26/d4e/d66/c10d 0 2024-03-18T14:51:15.397 INFO:tasks.workunit.client.1.smithi100.stdout:6/849: dread - d6/d7/df/d26/d4e/d66/d36/d4f/fae zero size 2024-03-18T14:51:15.398 INFO:tasks.workunit.client.1.smithi100.stdout:6/850: truncate d6/d7/df/d72/fd8 1037632 0 2024-03-18T14:51:15.398 INFO:tasks.workunit.client.1.smithi100.stdout:9/942: creat da/dc3/ddb/f132 x:0 0 0 2024-03-18T14:51:15.400 INFO:tasks.workunit.client.1.smithi100.stdout:6/851: creat d6/d7/df/d26/d4e/db5/dba/dfe/f10e x:0 0 0 2024-03-18T14:51:15.401 INFO:tasks.workunit.client.1.smithi100.stdout:9/943: mknod da/dc3/c133 0 2024-03-18T14:51:15.403 INFO:tasks.workunit.client.1.smithi100.stdout:6/852: creat d6/d9f/f10f x:0 0 0 2024-03-18T14:51:15.421 INFO:tasks.workunit.client.1.smithi100.stdout:9/944: dwrite da/dc3/dfc/f75 [0,4194304] 0 2024-03-18T14:51:15.426 INFO:tasks.workunit.client.1.smithi100.stdout:9/945: creat da/dd/d3e/d11e/dd6/de0/f134 x:0 0 0 2024-03-18T14:51:15.429 INFO:tasks.workunit.client.1.smithi100.stdout:9/946: link da/dd/d3e/d11e/d4d/f56 da/dd/d29/d9e/dcd/f135 0 2024-03-18T14:51:15.430 INFO:tasks.workunit.client.1.smithi100.stdout:9/947: fsync da/dc3/ddb/f132 0 2024-03-18T14:51:15.432 INFO:tasks.workunit.client.1.smithi100.stdout:9/948: dread da/d2c/f77 [0,4194304] 0 2024-03-18T14:51:15.434 INFO:tasks.workunit.client.1.smithi100.stdout:6/853: truncate d6/d7/df/d26/f84 1665776 0 2024-03-18T14:51:15.442 INFO:tasks.workunit.client.1.smithi100.stdout:9/949: dread da/dc3/dfc/f6e [0,4194304] 0 2024-03-18T14:51:15.443 INFO:tasks.workunit.client.1.smithi100.stdout:9/950: symlink da/dd/d3b/d103/l136 0 2024-03-18T14:51:15.445 INFO:tasks.workunit.client.1.smithi100.stdout:9/951: rmdir da/dc3 39 2024-03-18T14:51:15.447 INFO:tasks.workunit.client.1.smithi100.stdout:6/854: dwrite d6/d7/df/d26/d4e/db5/dba/fd3 [0,4194304] 0 2024-03-18T14:51:15.451 INFO:tasks.workunit.client.1.smithi100.stdout:6/855: mkdir d6/d7/df/d26/d4e/d53/d110 0 2024-03-18T14:51:15.451 INFO:tasks.workunit.client.1.smithi100.stdout:6/856: stat d6/d12/da2/faf 0 2024-03-18T14:51:15.456 INFO:tasks.workunit.client.1.smithi100.stdout:6/857: creat d6/d7/df/d72/d48/d76/f111 x:0 0 0 2024-03-18T14:51:15.459 INFO:tasks.workunit.client.1.smithi100.stdout:9/952: dwrite da/d88/fff [0,4194304] 0 2024-03-18T14:51:15.467 INFO:tasks.workunit.client.1.smithi100.stdout:9/953: write da/dd/d3b/f115 [684861,27730] 0 2024-03-18T14:51:15.473 INFO:tasks.workunit.client.1.smithi100.stdout:6/858: rename d6/d7/df/d26/d4e/d66 to d6/d7/df/d26/d4e/d53/d110/d112 0 2024-03-18T14:51:15.473 INFO:tasks.workunit.client.1.smithi100.stdout:6/859: creat d6/d7/df/d26/d4e/d53/f113 x:0 0 0 2024-03-18T14:51:15.476 INFO:tasks.workunit.client.1.smithi100.stdout:9/954: mkdir da/dd/d3c/d4b/d137 0 2024-03-18T14:51:15.480 INFO:tasks.workunit.client.1.smithi100.stdout:9/955: mknod da/dd/d3b/c138 0 2024-03-18T14:51:15.486 INFO:tasks.workunit.client.1.smithi100.stdout:9/956: dread da/d88/f8c [0,4194304] 0 2024-03-18T14:51:15.490 INFO:tasks.workunit.client.1.smithi100.stdout:6/860: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/f58 [0,4194304] 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:6/861: dwrite d6/d7/df/d26/d4e/d53/d110/d112/fd9 [0,4194304] 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:6/862: fsync d6/d7/df/d26/d4e/d53/f113 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:6/863: truncate d6/d7/df/d72/fd8 1608680 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:6/864: fsync f4 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:6/865: fsync d6/d7/f8d 0 2024-03-18T14:51:15.511 INFO:tasks.workunit.client.1.smithi100.stdout:9/957: dwrite da/d88/f8c [0,4194304] 0 2024-03-18T14:51:15.517 INFO:tasks.workunit.client.1.smithi100.stdout:9/958: mkdir da/dd/d3c/d139 0 2024-03-18T14:51:15.522 INFO:tasks.workunit.client.1.smithi100.stdout:6/866: rmdir d6/da1 0 2024-03-18T14:51:15.524 INFO:tasks.workunit.client.1.smithi100.stdout:6/867: truncate d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/ff6 6991645 0 2024-03-18T14:51:15.539 INFO:tasks.workunit.client.1.smithi100.stdout:5/825: sync 2024-03-18T14:51:15.539 INFO:tasks.workunit.client.1.smithi100.stdout:5/826: fdatasync d29/f34 0 2024-03-18T14:51:15.541 INFO:tasks.workunit.client.1.smithi100.stdout:5/827: dread d29/d2d/dbe/dc2/ff1 [0,4194304] 0 2024-03-18T14:51:15.541 INFO:tasks.workunit.client.1.smithi100.stdout:6/868: dwrite d6/d7/df/d72/d48/d76/f111 [0,4194304] 0 2024-03-18T14:51:15.542 INFO:tasks.workunit.client.1.smithi100.stdout:5/828: mkdir d29/d2d/d39/d83/dc1/d10b 0 2024-03-18T14:51:15.547 INFO:tasks.workunit.client.1.smithi100.stdout:6/869: creat d6/d12/d78/f114 x:0 0 0 2024-03-18T14:51:15.555 INFO:tasks.workunit.client.1.smithi100.stdout:6/870: dread d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/ff6 [4194304,4194304] 0 2024-03-18T14:51:15.555 INFO:tasks.workunit.client.1.smithi100.stdout:6/871: fsync d6/d7/df/d26/d4e/d53/d110/d112/f2d 0 2024-03-18T14:51:15.555 INFO:tasks.workunit.client.1.smithi100.stdout:6/872: creat d6/d7/df/d72/d65/d7a/f115 x:0 0 0 2024-03-18T14:51:15.559 INFO:tasks.workunit.client.1.smithi100.stdout:5/829: dwrite d29/d2d/d39/d65/d7d/fe2 [0,4194304] 0 2024-03-18T14:51:15.562 INFO:tasks.workunit.client.1.smithi100.stdout:6/873: dread f4 [4194304,4194304] 0 2024-03-18T14:51:15.564 INFO:tasks.workunit.client.1.smithi100.stdout:5/830: getdents d29/d2d/d39/d71 0 2024-03-18T14:51:15.564 INFO:tasks.workunit.client.1.smithi100.stdout:5/831: fdatasync d29/d2d/d39/d60/fb5 0 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/874: rmdir d6/d7/df/d72/d65 39 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/875: fdatasync d6/fe5 0 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/876: read d6/d12/da2/f104 [3608392,47911] 0 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/877: fdatasync d6/d9f/f10f 0 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/878: write d6/d12/f43 [47571,35124] 0 2024-03-18T14:51:15.566 INFO:tasks.workunit.client.1.smithi100.stdout:6/879: creat d6/d7/df/d26/d4e/f116 x:0 0 0 2024-03-18T14:51:15.567 INFO:tasks.workunit.client.1.smithi100.stdout:5/832: symlink d29/d2d/d39/d61/l10c 0 2024-03-18T14:51:15.568 INFO:tasks.workunit.client.1.smithi100.stdout:6/880: dread d6/d7/df/d72/f33 [0,4194304] 0 2024-03-18T14:51:15.569 INFO:tasks.workunit.client.1.smithi100.stdout:6/881: fsync d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/ff6 0 2024-03-18T14:51:15.569 INFO:tasks.workunit.client.1.smithi100.stdout:6/882: truncate d6/d12/d78/ff4 1525363 0 2024-03-18T14:51:15.569 INFO:tasks.workunit.client.1.smithi100.stdout:5/833: write f26 [1657692,24303] 0 2024-03-18T14:51:15.570 INFO:tasks.workunit.client.1.smithi100.stdout:5/834: mkdir d29/dd0/d10d 0 2024-03-18T14:51:15.578 INFO:tasks.workunit.client.1.smithi100.stdout:5/835: dread d29/d4a/f9c [0,4194304] 0 2024-03-18T14:51:15.578 INFO:tasks.workunit.client.1.smithi100.stdout:5/836: creat d29/d2d/f10e x:0 0 0 2024-03-18T14:51:15.580 INFO:tasks.workunit.client.1.smithi100.stdout:5/837: link d29/d2d/d39/f84 d29/d2d/d8f/df4/f10f 0 2024-03-18T14:51:15.580 INFO:tasks.workunit.client.1.smithi100.stdout:5/838: write d29/d2d/d39/d65/f6a [3583046,102608] 0 2024-03-18T14:51:15.582 INFO:tasks.workunit.client.1.smithi100.stdout:6/883: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d27/f70 [0,4194304] 0 2024-03-18T14:51:15.583 INFO:tasks.workunit.client.1.smithi100.stdout:5/839: symlink d29/d2d/d39/d83/dc1/dbb/d9e/dd7/dee/l110 0 2024-03-18T14:51:15.586 INFO:tasks.workunit.client.1.smithi100.stdout:5/840: rename d29/ff7 to d29/d2d/d39/d61/ddd/f111 0 2024-03-18T14:51:15.587 INFO:tasks.workunit.client.1.smithi100.stdout:5/841: link d29/d2d/f97 d29/d2d/d39/d83/dc1/dbb/f112 0 2024-03-18T14:51:15.588 INFO:tasks.workunit.client.1.smithi100.stdout:5/842: creat d29/d2d/d39/d83/dc1/dbb/d9e/dd7/f113 x:0 0 0 2024-03-18T14:51:15.590 INFO:tasks.workunit.client.1.smithi100.stdout:5/843: creat d29/d2d/d8f/f114 x:0 0 0 2024-03-18T14:51:15.592 INFO:tasks.workunit.client.1.smithi100.stdout:5/844: mknod d29/d2d/dbe/dc2/c115 0 2024-03-18T14:51:15.594 INFO:tasks.workunit.client.1.smithi100.stdout:5/845: symlink d29/d2d/dbe/l116 0 2024-03-18T14:51:15.594 INFO:tasks.workunit.client.1.smithi100.stdout:5/846: fdatasync d29/f2c 0 2024-03-18T14:51:15.595 INFO:tasks.workunit.client.1.smithi100.stdout:6/884: dwrite d6/d7/df/d26/f84 [0,4194304] 0 2024-03-18T14:51:15.595 INFO:tasks.workunit.client.1.smithi100.stdout:5/847: creat d29/d2d/d39/dfe/f117 x:0 0 0 2024-03-18T14:51:15.603 INFO:tasks.workunit.client.1.smithi100.stdout:6/885: creat d6/d12/da2/f117 x:0 0 0 2024-03-18T14:51:15.607 INFO:tasks.workunit.client.1.smithi100.stdout:5/848: unlink d29/d2d/f97 0 2024-03-18T14:51:15.610 INFO:tasks.workunit.client.1.smithi100.stdout:6/886: link d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/c87 d6/d7/df/c118 0 2024-03-18T14:51:15.620 INFO:tasks.workunit.client.1.smithi100.stdout:5/849: link d29/d2d/d39/d83/dc1/dbb/d9e/lac d29/d2d/d39/d71/d96/dc8/l118 0 2024-03-18T14:51:15.620 INFO:tasks.workunit.client.1.smithi100.stdout:6/887: symlink d6/d7/l119 0 2024-03-18T14:51:15.624 INFO:tasks.workunit.client.1.smithi100.stdout:5/850: rename l1d to d29/d2d/d39/d65/d7d/l119 0 2024-03-18T14:51:15.627 INFO:tasks.workunit.client.1.smithi100.stdout:6/888: unlink d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/lf7 0 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:0/939: sync 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:3/933: sync 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:2/901: sync 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:9/959: sync 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:1/911: sync 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:2/902: dread - d3/d6/d29/d36/ddf/f114 zero size 2024-03-18T14:51:15.632 INFO:tasks.workunit.client.1.smithi100.stdout:4/900: sync 2024-03-18T14:51:15.633 INFO:tasks.workunit.client.1.smithi100.stdout:7/920: sync 2024-03-18T14:51:15.633 INFO:tasks.workunit.client.1.smithi100.stdout:2/903: dread - d3/da/de/d20/d82/d95/f117 zero size 2024-03-18T14:51:15.633 INFO:tasks.workunit.client.1.smithi100.stdout:3/934: write d3/d16/d13/d1f/d3f/d4f/de0/ffd [3425892,38742] 0 2024-03-18T14:51:15.637 INFO:tasks.workunit.client.1.smithi100.stdout:6/889: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/de8/f11a x:0 0 0 2024-03-18T14:51:15.637 INFO:tasks.workunit.client.1.smithi100.stdout:6/890: chown d6/d12/d2c/fc1 773366 1 2024-03-18T14:51:15.638 INFO:tasks.workunit.client.1.smithi100.stdout:5/851: link d29/d2d/d32/d92/daa/cbc d29/d2d/d39/d83/dc1/dbb/d9e/c11a 0 2024-03-18T14:51:15.638 INFO:tasks.workunit.client.1.smithi100.stdout:5/852: stat d29/d2d/d39/d60 0 2024-03-18T14:51:15.640 INFO:tasks.workunit.client.1.smithi100.stdout:0/940: rename dd/d20/d44/f9e to dd/d31/d91/d56/d71/dcc/dcf/d120/d12f/f155 0 2024-03-18T14:51:15.640 INFO:tasks.workunit.client.1.smithi100.stdout:0/941: creat dd/d31/d91/d56/d62/d6e/f156 x:0 0 0 2024-03-18T14:51:15.640 INFO:tasks.workunit.client.1.smithi100.stdout:0/942: chown dd/l11b 41096 1 2024-03-18T14:51:15.642 INFO:tasks.workunit.client.1.smithi100.stdout:9/960: symlink da/dd/d29/d9e/dcd/l13a 0 2024-03-18T14:51:15.642 INFO:tasks.workunit.client.1.smithi100.stdout:9/961: creat da/dcb/f13b x:0 0 0 2024-03-18T14:51:15.642 INFO:tasks.workunit.client.1.smithi100.stdout:1/912: unlink d8/d1e/d11/d27/l4e 0 2024-03-18T14:51:15.644 INFO:tasks.workunit.client.1.smithi100.stdout:4/901: truncate d9/d17/f34 2391129 0 2024-03-18T14:51:15.647 INFO:tasks.workunit.client.1.smithi100.stdout:2/904: unlink d3/d6/f9 0 2024-03-18T14:51:15.647 INFO:tasks.workunit.client.1.smithi100.stdout:2/905: read d3/d6/f67 [2329282,65470] 0 2024-03-18T14:51:15.647 INFO:tasks.workunit.client.1.smithi100.stdout:0/943: dread dd/d2b/d11e/fe7 [0,4194304] 0 2024-03-18T14:51:15.655 INFO:tasks.workunit.client.1.smithi100.stdout:7/921: link d5/d22/d3b/c6d d5/d30/d4d/d5a/c14f 0 2024-03-18T14:51:15.655 INFO:tasks.workunit.client.1.smithi100.stdout:7/922: write d5/d90/fd0 [3905153,85506] 0 2024-03-18T14:51:15.659 INFO:tasks.workunit.client.1.smithi100.stdout:6/891: getdents d6/d7/df/d72/d65/d7a/da5 0 2024-03-18T14:51:15.662 INFO:tasks.workunit.client.1.smithi100.stdout:2/906: dwrite d3/da/de/d20/f24 [0,4194304] 0 2024-03-18T14:51:15.662 INFO:tasks.workunit.client.1.smithi100.stdout:5/853: rename f7 to d29/d2d/d39/dfe/dff/f11b 0 2024-03-18T14:51:15.674 INFO:tasks.workunit.client.1.smithi100.stdout:4/902: unlink d9/d27/f3d 0 2024-03-18T14:51:15.674 INFO:tasks.workunit.client.1.smithi100.stdout:4/903: chown d9/d13/d5a/da4/f106 20131450 1 2024-03-18T14:51:15.677 INFO:tasks.workunit.client.1.smithi100.stdout:0/944: link dd/d31/d91/d56/d71/dcc/dcf/d120/d145/f149 dd/d2b/d95/d150/f157 0 2024-03-18T14:51:15.677 INFO:tasks.workunit.client.1.smithi100.stdout:0/945: fdatasync dd/d2b/d11e/f13c 0 2024-03-18T14:51:15.677 INFO:tasks.workunit.client.1.smithi100.stdout:0/946: stat dd/d31/d91/d65 0 2024-03-18T14:51:15.680 INFO:tasks.workunit.client.1.smithi100.stdout:3/935: getdents d3/d16/d13/d1f/d7c 0 2024-03-18T14:51:15.680 INFO:tasks.workunit.client.1.smithi100.stdout:3/936: write d3/d16/d13/f10b [352377,128955] 0 2024-03-18T14:51:15.682 INFO:tasks.workunit.client.1.smithi100.stdout:7/923: creat d5/d30/d4d/db4/f150 x:0 0 0 2024-03-18T14:51:15.683 INFO:tasks.workunit.client.1.smithi100.stdout:7/924: dread d5/d22/d3b/d9f/da0/db1/dd3/f118 [0,4194304] 0 2024-03-18T14:51:15.685 INFO:tasks.workunit.client.1.smithi100.stdout:6/892: mkdir d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/d11b 0 2024-03-18T14:51:15.687 INFO:tasks.workunit.client.1.smithi100.stdout:3/937: dread d3/d16/d13/d1f/f59 [0,4194304] 0 2024-03-18T14:51:15.689 INFO:tasks.workunit.client.1.smithi100.stdout:9/962: rename da/dd/d3b/c80 to da/dd/d3c/d4b/c13c 0 2024-03-18T14:51:15.689 INFO:tasks.workunit.client.1.smithi100.stdout:9/963: read - da/dd/d3e/d11e/dd6/de0/f134 zero size 2024-03-18T14:51:15.689 INFO:tasks.workunit.client.1.smithi100.stdout:9/964: chown da/dd/d3e/d11e/d8b/f37 74 1 2024-03-18T14:51:15.692 INFO:tasks.workunit.client.1.smithi100.stdout:5/854: link d29/d2d/d39/d71/d96/cdf d29/d2d/dbe/c11c 0 2024-03-18T14:51:15.703 INFO:tasks.workunit.client.1.smithi100.stdout:1/913: mkdir d8/d1e/d11/d27/d3f/d121 0 2024-03-18T14:51:15.705 INFO:tasks.workunit.client.1.smithi100.stdout:5/855: dwrite d29/dd0/ff6 [0,4194304] 0 2024-03-18T14:51:15.705 INFO:tasks.workunit.client.1.smithi100.stdout:5/856: chown d29/d2d/d39/d65/f98 3956 1 2024-03-18T14:51:15.706 INFO:tasks.workunit.client.1.smithi100.stdout:5/857: creat d29/d2d/d39/d74/f11d x:0 0 0 2024-03-18T14:51:15.706 INFO:tasks.workunit.client.1.smithi100.stdout:5/858: readlink d29/d2d/d39/dfe/l109 0 2024-03-18T14:51:15.706 INFO:tasks.workunit.client.1.smithi100.stdout:5/859: creat d29/d2d/d32/f11e x:0 0 0 2024-03-18T14:51:15.713 INFO:tasks.workunit.client.1.smithi100.stdout:0/947: unlink dd/d2b/fc0 0 2024-03-18T14:51:15.713 INFO:tasks.workunit.client.1.smithi100.stdout:5/860: dread d29/d2d/d39/d61/f6b [0,4194304] 0 2024-03-18T14:51:15.715 INFO:tasks.workunit.client.1.smithi100.stdout:7/925: link d5/d30/l137 d5/d14/d16/d63/d124/dd1/d86/de6/d122/l151 0 2024-03-18T14:51:15.716 INFO:tasks.workunit.client.1.smithi100.stdout:6/893: mkdir d6/d7/df/d26/d4e/d53/d110/d112/d36/d11c 0 2024-03-18T14:51:15.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/894: chown d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8 12 1 2024-03-18T14:51:15.717 INFO:tasks.workunit.client.1.smithi100.stdout:6/895: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/de8/f11d x:0 0 0 2024-03-18T14:51:15.718 INFO:tasks.workunit.client.1.smithi100.stdout:3/938: mkdir d3/d16/d13/d1f/d37/d13e 0 2024-03-18T14:51:15.719 INFO:tasks.workunit.client.1.smithi100.stdout:2/907: getdents d3/da/de/d20/d82/d95 0 2024-03-18T14:51:15.721 INFO:tasks.workunit.client.1.smithi100.stdout:9/965: mkdir da/d51/df4/d13d 0 2024-03-18T14:51:15.721 INFO:tasks.workunit.client.1.smithi100.stdout:9/966: truncate da/dd/fa1 1409632 0 2024-03-18T14:51:15.721 INFO:tasks.workunit.client.1.smithi100.stdout:9/967: chown da/dd/d3e/d11e/dd6/de0/l11f 590995 1 2024-03-18T14:51:15.723 INFO:tasks.workunit.client.1.smithi100.stdout:1/914: unlink d8/d30/f35 0 2024-03-18T14:51:15.727 INFO:tasks.workunit.client.1.smithi100.stdout:4/904: dwrite d9/d27/d42/f4a [0,4194304] 0 2024-03-18T14:51:15.730 INFO:tasks.workunit.client.1.smithi100.stdout:9/968: dread da/dd/d3b/d103/f4c [0,4194304] 0 2024-03-18T14:51:15.730 INFO:tasks.workunit.client.1.smithi100.stdout:9/969: stat da/d2c/d70/lc8 0 2024-03-18T14:51:15.731 INFO:tasks.workunit.client.1.smithi100.stdout:5/861: rename d29/f34 to d29/d2d/f11f 0 2024-03-18T14:51:15.731 INFO:tasks.workunit.client.1.smithi100.stdout:5/862: fdatasync d29/d2d/d39/f4e 0 2024-03-18T14:51:15.734 INFO:tasks.workunit.client.1.smithi100.stdout:4/905: dread d9/d13/d25/fe6 [0,4194304] 0 2024-03-18T14:51:15.737 INFO:tasks.workunit.client.1.smithi100.stdout:7/926: creat d5/d14/d16/d63/d124/dd1/d107/f152 x:0 0 0 2024-03-18T14:51:15.747 INFO:tasks.workunit.client.1.smithi100.stdout:3/939: creat d3/dd/d21/d3e/df8/d74/f13f x:0 0 0 2024-03-18T14:51:15.747 INFO:tasks.workunit.client.1.smithi100.stdout:3/940: chown d3/dd/d21/d3e/df8/db9/l11d 24243481 1 2024-03-18T14:51:15.756 INFO:tasks.workunit.client.1.smithi100.stdout:0/948: rmdir dd/d31/d91/d56/d62 39 2024-03-18T14:51:15.756 INFO:tasks.workunit.client.1.smithi100.stdout:0/949: chown dd/d31/d91/d56/d71/dcc/dcf/c122 0 1 2024-03-18T14:51:15.757 INFO:tasks.workunit.client.1.smithi100.stdout:9/970: mkdir da/dc3/dfc/d13e 0 2024-03-18T14:51:15.760 INFO:tasks.workunit.client.1.smithi100.stdout:2/908: rename d3/d6/d29/d36/d49/lfb to d3/d1a/d6c/l12c 0 2024-03-18T14:51:15.760 INFO:tasks.workunit.client.1.smithi100.stdout:6/896: mknod d6/d7/df/d72/d65/c11e 0 2024-03-18T14:51:15.760 INFO:tasks.workunit.client.1.smithi100.stdout:6/897: chown d6/d12/d103 22371503 1 2024-03-18T14:51:15.769 INFO:tasks.workunit.client.1.smithi100.stdout:4/906: dwrite d9/d13/d5a/d5b/da0/dd8/fed [0,4194304] 0 2024-03-18T14:51:15.769 INFO:tasks.workunit.client.1.smithi100.stdout:4/907: truncate d9/d13/d25/fa5 695006 0 2024-03-18T14:51:15.769 INFO:tasks.workunit.client.1.smithi100.stdout:7/927: dwrite d5/d14/d16/d63/d124/dc3/f127 [0,4194304] 0 2024-03-18T14:51:15.770 INFO:tasks.workunit.client.1.smithi100.stdout:5/863: dwrite d29/d2d/d32/f11e [0,4194304] 0 2024-03-18T14:51:15.770 INFO:tasks.workunit.client.1.smithi100.stdout:5/864: truncate d29/d4a/f54 4861172 0 2024-03-18T14:51:15.770 INFO:tasks.workunit.client.1.smithi100.stdout:0/950: mknod dd/d61/dcd/dff/c158 0 2024-03-18T14:51:15.770 INFO:tasks.workunit.client.1.smithi100.stdout:4/908: dread d9/d27/da2/ffe [0,4194304] 0 2024-03-18T14:51:15.772 INFO:tasks.workunit.client.1.smithi100.stdout:9/971: rename da/dc3/cfe to da/dd/d29/c13f 0 2024-03-18T14:51:15.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/909: link d3/d1a/fbe d3/d1a/d76/f12d 0 2024-03-18T14:51:15.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/910: chown d3/d6/ff7 13665 1 2024-03-18T14:51:15.783 INFO:tasks.workunit.client.1.smithi100.stdout:6/898: creat d6/d7/df/d72/d48/f11f x:0 0 0 2024-03-18T14:51:15.783 INFO:tasks.workunit.client.1.smithi100.stdout:6/899: chown d6/d12/d2c/cb1 391 1 2024-03-18T14:51:15.793 INFO:tasks.workunit.client.1.smithi100.stdout:1/915: mknod d8/d1e/d11/d27/d3f/d121/c122 0 2024-03-18T14:51:15.800 INFO:tasks.workunit.client.1.smithi100.stdout:7/928: dwrite d5/d22/d24/d6e/d105/f109 [0,4194304] 0 2024-03-18T14:51:15.800 INFO:tasks.workunit.client.1.smithi100.stdout:7/929: creat d5/d30/d4d/db4/f153 x:0 0 0 2024-03-18T14:51:15.805 INFO:tasks.workunit.client.1.smithi100.stdout:1/916: write d8/d2a/fc7 [3663847,17650] 0 2024-03-18T14:51:15.806 INFO:tasks.workunit.client.1.smithi100.stdout:4/909: dwrite d9/fb [0,4194304] 0 2024-03-18T14:51:15.806 INFO:tasks.workunit.client.1.smithi100.stdout:9/972: dwrite da/d51/f116 [0,4194304] 0 2024-03-18T14:51:15.806 INFO:tasks.workunit.client.1.smithi100.stdout:1/917: dread d8/d30/d61/d8d/f92 [0,4194304] 0 2024-03-18T14:51:15.806 INFO:tasks.workunit.client.1.smithi100.stdout:4/910: dread d9/d13/ff2 [0,4194304] 0 2024-03-18T14:51:15.818 INFO:tasks.workunit.client.1.smithi100.stdout:0/951: rename dd/d31/d91/f3f to dd/d31/d91/d56/d121/f159 0 2024-03-18T14:51:15.821 INFO:tasks.workunit.client.1.smithi100.stdout:2/911: creat d3/da/de/d11f/f12e x:0 0 0 2024-03-18T14:51:15.827 INFO:tasks.workunit.client.1.smithi100.stdout:7/930: rmdir d5/d22/d3b/d9f 39 2024-03-18T14:51:15.829 INFO:tasks.workunit.client.1.smithi100.stdout:4/911: dwrite d9/d13/d39/f10b [0,4194304] 0 2024-03-18T14:51:15.831 INFO:tasks.workunit.client.1.smithi100.stdout:9/973: mkdir da/dd/d3e/d11e/dd6/d140 0 2024-03-18T14:51:15.832 INFO:tasks.workunit.client.1.smithi100.stdout:2/912: write d3/d6/d29/d36/f9e [1210619,90019] 0 2024-03-18T14:51:15.837 INFO:tasks.workunit.client.1.smithi100.stdout:5/865: getdents d29/d2d/d39 0 2024-03-18T14:51:15.837 INFO:tasks.workunit.client.1.smithi100.stdout:5/866: creat d29/d2d/d39/d71/d96/f120 x:0 0 0 2024-03-18T14:51:15.843 INFO:tasks.workunit.client.1.smithi100.stdout:6/900: rename d6/d7/df/d26/d4e/d53/la8 to d6/d7/df/d72/d48/d76/l120 0 2024-03-18T14:51:15.844 INFO:tasks.workunit.client.1.smithi100.stdout:0/952: link dd/d31/d91/d56/d62/ca7 dd/d20/c15a 0 2024-03-18T14:51:15.852 INFO:tasks.workunit.client.1.smithi100.stdout:5/867: getdents d29/d2d/d39/d83/dc1/dbb 0 2024-03-18T14:51:15.854 INFO:tasks.workunit.client.1.smithi100.stdout:2/913: rename d3/da/db8/d111/f125 to d3/d6/d29/d36/d4a/f12f 0 2024-03-18T14:51:15.858 INFO:tasks.workunit.client.1.smithi100.stdout:0/953: mkdir dd/d20/d44/df9/d15b 0 2024-03-18T14:51:15.858 INFO:tasks.workunit.client.1.smithi100.stdout:0/954: stat dd/d20/d44/f125 0 2024-03-18T14:51:15.858 INFO:tasks.workunit.client.1.smithi100.stdout:0/955: stat dd/d20/db5/l5a 0 2024-03-18T14:51:15.861 INFO:tasks.workunit.client.1.smithi100.stdout:7/931: getdents d5/d14/d16/d63/d124/dd1/d107 0 2024-03-18T14:51:15.862 INFO:tasks.workunit.client.1.smithi100.stdout:9/974: rmdir da/d51 39 2024-03-18T14:51:15.868 INFO:tasks.workunit.client.1.smithi100.stdout:7/932: dread d5/d22/d24/dac/fb6 [0,4194304] 0 2024-03-18T14:51:15.870 INFO:tasks.workunit.client.1.smithi100.stdout:7/933: dread d5/d14/d16/d63/d124/d66/fc1 [0,4194304] 0 2024-03-18T14:51:15.879 INFO:tasks.workunit.client.1.smithi100.stdout:7/934: write d5/d22/d24/d6e/d105/f109 [1697487,66306] 0 2024-03-18T14:51:15.881 INFO:tasks.workunit.client.1.smithi100.stdout:9/975: mkdir da/d51/d141 0 2024-03-18T14:51:15.885 INFO:tasks.workunit.client.1.smithi100.stdout:6/901: rename d6/d7/df/d72/d8b/f8c to d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d27/f121 0 2024-03-18T14:51:15.893 INFO:tasks.workunit.client.1.smithi100.stdout:7/935: creat d5/d22/d3b/d47/f154 x:0 0 0 2024-03-18T14:51:15.895 INFO:tasks.workunit.client.1.smithi100.stdout:5/868: rename d29/d2d/d39/d71/d96/cdf to d29/d2d/d32/c121 0 2024-03-18T14:51:15.897 INFO:tasks.workunit.client.1.smithi100.stdout:6/902: truncate d6/d7/df/d72/d48/d76/fad 903310 0 2024-03-18T14:51:15.898 INFO:tasks.workunit.client.1.smithi100.stdout:7/936: mknod d5/d30/c155 0 2024-03-18T14:51:15.901 INFO:tasks.workunit.client.1.smithi100.stdout:7/937: truncate d5/d22/d24/d6e/d7a/fe3 2000963 0 2024-03-18T14:51:15.902 INFO:tasks.workunit.client.1.smithi100.stdout:2/914: dwrite d3/d1a/d76/fc7 [0,4194304] 0 2024-03-18T14:51:15.904 INFO:tasks.workunit.client.1.smithi100.stdout:6/903: mknod d6/d7/df/d72/d65/c122 0 2024-03-18T14:51:15.904 INFO:tasks.workunit.client.1.smithi100.stdout:6/904: write d6/d7/df/d26/d4e/f82 [858310,40454] 0 2024-03-18T14:51:15.904 INFO:tasks.workunit.client.1.smithi100.stdout:6/905: dread - d6/d12/da2/f117 zero size 2024-03-18T14:51:15.904 INFO:tasks.workunit.client.1.smithi100.stdout:7/938: truncate d5/d14/d16/f1d 186735 0 2024-03-18T14:51:15.909 INFO:tasks.workunit.client.1.smithi100.stdout:2/915: mknod d3/d6/d29/d36/d4a/c130 0 2024-03-18T14:51:15.909 INFO:tasks.workunit.client.1.smithi100.stdout:2/916: write d3/da/de/d20/dea/d40/dac/fe8 [1025709,101921] 0 2024-03-18T14:51:15.909 INFO:tasks.workunit.client.1.smithi100.stdout:2/917: chown d3/da 1211657 1 2024-03-18T14:51:15.909 INFO:tasks.workunit.client.1.smithi100.stdout:2/918: chown d3/d6/d29/fda 985885 1 2024-03-18T14:51:15.910 INFO:tasks.workunit.client.1.smithi100.stdout:2/919: truncate d3/da/de/d20/f23 2787649 0 2024-03-18T14:51:15.910 INFO:tasks.workunit.client.1.smithi100.stdout:2/920: stat d3/da/de/d20/d82/f113 0 2024-03-18T14:51:15.913 INFO:tasks.workunit.client.1.smithi100.stdout:2/921: mkdir d3/d8c/d131 0 2024-03-18T14:51:15.920 INFO:tasks.workunit.client.1.smithi100.stdout:7/939: write d5/d22/d3b/d9f/da0/db1/dd3/fe0 [2864146,112579] 0 2024-03-18T14:51:15.920 INFO:tasks.workunit.client.1.smithi100.stdout:9/976: dwrite da/dd/d29/d71/d72/f118 [0,4194304] 0 2024-03-18T14:51:15.920 INFO:tasks.workunit.client.1.smithi100.stdout:0/956: dwrite dd/d31/f57 [0,4194304] 0 2024-03-18T14:51:15.921 INFO:tasks.workunit.client.1.smithi100.stdout:5/869: dwrite d29/f37 [0,4194304] 0 2024-03-18T14:51:15.921 INFO:tasks.workunit.client.1.smithi100.stdout:9/977: dread da/d2c/f77 [0,4194304] 0 2024-03-18T14:51:15.924 INFO:tasks.workunit.client.1.smithi100.stdout:6/906: dwrite f4 [4194304,4194304] 0 2024-03-18T14:51:15.924 INFO:tasks.workunit.client.1.smithi100.stdout:7/940: creat d5/d14/d16/d63/d124/dd1/d86/d85/d87/f156 x:0 0 0 2024-03-18T14:51:15.925 INFO:tasks.workunit.client.1.smithi100.stdout:7/941: write d5/d14/d16/d63/d124/d66/d4f/dcb/f10a [512286,1718] 0 2024-03-18T14:51:15.925 INFO:tasks.workunit.client.1.smithi100.stdout:7/942: dread - d5/d30/d4d/db4/fdb zero size 2024-03-18T14:51:15.933 INFO:tasks.workunit.client.1.smithi100.stdout:0/957: mkdir dd/d20/d44/df9/d15b/d15c 0 2024-03-18T14:51:15.934 INFO:tasks.workunit.client.1.smithi100.stdout:2/922: dwrite d3/da/de/d20/d82/fd8 [0,4194304] 0 2024-03-18T14:51:15.934 INFO:tasks.workunit.client.1.smithi100.stdout:2/923: read - d3/da/de/d20/d82/dad/de2/f123 zero size 2024-03-18T14:51:15.937 INFO:tasks.workunit.client.1.smithi100.stdout:5/870: dwrite d29/d2d/dbe/dc2/fe7 [0,4194304] 0 2024-03-18T14:51:15.940 INFO:tasks.workunit.client.1.smithi100.stdout:9/978: mknod da/dd/d3e/d11e/c142 0 2024-03-18T14:51:15.946 INFO:tasks.workunit.client.1.smithi100.stdout:7/943: mknod d5/d22/d3b/d9f/da0/db1/c157 0 2024-03-18T14:51:15.946 INFO:tasks.workunit.client.1.smithi100.stdout:7/944: dread - d5/d14/d16/d63/d124/d66/d4f/dcb/f14d zero size 2024-03-18T14:51:15.947 INFO:tasks.workunit.client.1.smithi100.stdout:7/945: read d5/d22/d3b/d9f/da0/fa8 [1202813,45447] 0 2024-03-18T14:51:15.947 INFO:tasks.workunit.client.1.smithi100.stdout:0/958: mknod dd/d31/d91/d56/de0/c15d 0 2024-03-18T14:51:15.947 INFO:tasks.workunit.client.1.smithi100.stdout:0/959: truncate dd/d61/f100 235393 0 2024-03-18T14:51:15.948 INFO:tasks.workunit.client.1.smithi100.stdout:0/960: truncate dd/d31/d91/d56/d71/dcc/dcf/d120/d12f/f155 1280162 0 2024-03-18T14:51:15.956 INFO:tasks.workunit.client.1.smithi100.stdout:5/871: rename d29/d2d/d39/d61/f101 to d29/d2d/d39/d83/f122 0 2024-03-18T14:51:15.957 INFO:tasks.workunit.client.1.smithi100.stdout:9/979: mkdir da/dd/d29/d71/d72/db2/d143 0 2024-03-18T14:51:15.958 INFO:tasks.workunit.client.1.smithi100.stdout:6/907: rmdir d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f 39 2024-03-18T14:51:15.960 INFO:tasks.workunit.client.1.smithi100.stdout:7/946: mknod d5/d14/d16/d63/d124/dd1/d86/d85/d87/de5/deb/c158 0 2024-03-18T14:51:15.960 INFO:tasks.workunit.client.1.smithi100.stdout:0/961: dread dd/d20/f27 [0,4194304] 0 2024-03-18T14:51:15.960 INFO:tasks.workunit.client.1.smithi100.stdout:0/962: fdatasync dd/f1b 0 2024-03-18T14:51:15.960 INFO:tasks.workunit.client.1.smithi100.stdout:7/947: creat d5/d14/d16/d63/d124/dd1/d86/d85/d87/f159 x:0 0 0 2024-03-18T14:51:15.961 INFO:tasks.workunit.client.1.smithi100.stdout:7/948: chown d5/d22/d24/d6e/d7a/dc2/d111/f135 122 1 2024-03-18T14:51:15.961 INFO:tasks.workunit.client.1.smithi100.stdout:2/924: dwrite d3/d1a/d76/f12d [0,4194304] 0 2024-03-18T14:51:15.965 INFO:tasks.workunit.client.1.smithi100.stdout:5/872: rename d29/d4a/c7f to d29/d2d/dbe/dc2/c123 0 2024-03-18T14:51:15.969 INFO:tasks.workunit.client.1.smithi100.stdout:9/980: write da/dd/d29/fbd [2456688,25534] 0 2024-03-18T14:51:15.969 INFO:tasks.workunit.client.1.smithi100.stdout:9/981: read da/dd/f8f [329575,6208] 0 2024-03-18T14:51:15.980 INFO:tasks.workunit.client.1.smithi100.stdout:6/908: dwrite d6/d12/d2c/f4c [0,4194304] 0 2024-03-18T14:51:15.980 INFO:tasks.workunit.client.1.smithi100.stdout:7/949: dwrite d5/d14/f77 [0,4194304] 0 2024-03-18T14:51:15.992 INFO:tasks.workunit.client.1.smithi100.stdout:2/925: chown d3/d6/d5f/d6a 1975 1 2024-03-18T14:51:15.992 INFO:tasks.workunit.client.1.smithi100.stdout:2/926: write d3/d6/d5f/d6a/fc8 [1261150,107040] 0 2024-03-18T14:51:15.993 INFO:tasks.workunit.client.1.smithi100.stdout:5/873: creat d29/d2d/d39/d83/dc1/dbb/d9e/dd7/df9/f124 x:0 0 0 2024-03-18T14:51:15.995 INFO:tasks.workunit.client.1.smithi100.stdout:9/982: truncate da/dd/d29/d71/d72/f9b 765413 0 2024-03-18T14:51:15.995 INFO:tasks.workunit.client.1.smithi100.stdout:9/983: fsync da/d88/fff 0 2024-03-18T14:51:15.995 INFO:tasks.workunit.client.1.smithi100.stdout:9/984: truncate da/dd/d3e/d11e/dd6/de0/f134 1002476 0 2024-03-18T14:51:15.999 INFO:tasks.workunit.client.1.smithi100.stdout:6/909: rmdir d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a 39 2024-03-18T14:51:16.003 INFO:tasks.workunit.client.1.smithi100.stdout:7/950: rename d5/l8 to d5/d14/d16/d63/d124/dc3/l15a 0 2024-03-18T14:51:16.005 INFO:tasks.workunit.client.1.smithi100.stdout:2/927: symlink d3/da/de/d20/dea/d40/dac/l132 0 2024-03-18T14:51:16.010 INFO:tasks.workunit.client.1.smithi100.stdout:5/874: dwrite f2 [0,4194304] 0 2024-03-18T14:51:16.010 INFO:tasks.workunit.client.1.smithi100.stdout:5/875: chown d29/d2d/d39/d61/l10c 1520613 1 2024-03-18T14:51:16.011 INFO:tasks.workunit.client.1.smithi100.stdout:9/985: mkdir da/dd/d29/d144 0 2024-03-18T14:51:16.011 INFO:tasks.workunit.client.1.smithi100.stdout:9/986: fsync da/dd/d29/d71/df5/fb7 0 2024-03-18T14:51:16.012 INFO:tasks.workunit.client.1.smithi100.stdout:6/910: truncate d6/d12/d78/fb2 1084651 0 2024-03-18T14:51:16.016 INFO:tasks.workunit.client.1.smithi100.stdout:3/941: sync 2024-03-18T14:51:16.017 INFO:tasks.workunit.client.1.smithi100.stdout:2/928: creat d3/da/de/d20/dea/d4f/f133 x:0 0 0 2024-03-18T14:51:16.017 INFO:tasks.workunit.client.1.smithi100.stdout:2/929: truncate d3/f10f 14988 0 2024-03-18T14:51:16.019 INFO:tasks.workunit.client.1.smithi100.stdout:9/987: unlink da/dd/d3c/d6b/cdf 0 2024-03-18T14:51:16.020 INFO:tasks.workunit.client.1.smithi100.stdout:6/911: rename d6/d7/df/d72/c6a to d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/c123 0 2024-03-18T14:51:16.021 INFO:tasks.workunit.client.1.smithi100.stdout:6/912: stat d6/d7/df/d26/d4e/db5/cdf 0 2024-03-18T14:51:16.031 INFO:tasks.workunit.client.1.smithi100.stdout:5/876: dwrite d29/d2d/d39/d74/fa5 [0,4194304] 0 2024-03-18T14:51:16.031 INFO:tasks.workunit.client.1.smithi100.stdout:5/877: chown d29/d2d/d39/d61/ddd/f102 174 1 2024-03-18T14:51:16.031 INFO:tasks.workunit.client.1.smithi100.stdout:5/878: write d29/d2d/d32/f68 [5687490,56230] 0 2024-03-18T14:51:16.032 INFO:tasks.workunit.client.1.smithi100.stdout:7/951: rmdir d5/d22/d24 39 2024-03-18T14:51:16.035 INFO:tasks.workunit.client.1.smithi100.stdout:3/942: truncate d3/dd/f12c 230071 0 2024-03-18T14:51:16.038 INFO:tasks.workunit.client.1.smithi100.stdout:2/930: symlink d3/da/de/d20/d82/dad/dc5/d11a/l134 0 2024-03-18T14:51:16.039 INFO:tasks.workunit.client.1.smithi100.stdout:9/988: dwrite da/dc3/ddb/f132 [0,4194304] 0 2024-03-18T14:51:16.040 INFO:tasks.workunit.client.1.smithi100.stdout:9/989: write da/dc3/dfc/fca [2182725,42868] 0 2024-03-18T14:51:16.040 INFO:tasks.workunit.client.1.smithi100.stdout:9/990: fdatasync da/d2c/d70/fc4 0 2024-03-18T14:51:16.040 INFO:tasks.workunit.client.1.smithi100.stdout:6/913: link d6/d7/df/d72/f85 d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/f124 0 2024-03-18T14:51:16.045 INFO:tasks.workunit.client.1.smithi100.stdout:5/879: rmdir d29/d2d/dbe 39 2024-03-18T14:51:16.046 INFO:tasks.workunit.client.1.smithi100.stdout:2/931: dread d3/da/de/f3c [0,4194304] 0 2024-03-18T14:51:16.048 INFO:tasks.workunit.client.1.smithi100.stdout:7/952: mkdir d5/d14/d16/d63/d124/dd1/d86/de6/d122/d15b 0 2024-03-18T14:51:16.057 INFO:tasks.workunit.client.1.smithi100.stdout:6/914: mkdir d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d125 0 2024-03-18T14:51:16.058 INFO:tasks.workunit.client.1.smithi100.stdout:5/880: symlink d29/dd0/l125 0 2024-03-18T14:51:16.060 INFO:tasks.workunit.client.1.smithi100.stdout:2/932: rename d3/da/de/d20/d107/c127 to d3/d6/d29/d36/d4a/c135 0 2024-03-18T14:51:16.066 INFO:tasks.workunit.client.1.smithi100.stdout:5/881: mkdir d29/d2d/d39/d61/d126 0 2024-03-18T14:51:16.066 INFO:tasks.workunit.client.1.smithi100.stdout:5/882: creat d29/d2d/dbe/dc2/f127 x:0 0 0 2024-03-18T14:51:16.077 INFO:tasks.workunit.client.1.smithi100.stdout:5/883: write d29/d2d/d39/d74/fa5 [2530612,29435] 0 2024-03-18T14:51:16.083 INFO:tasks.workunit.client.1.smithi100.stdout:9/991: dwrite da/d2c/d2f/f4a [0,4194304] 0 2024-03-18T14:51:16.083 INFO:tasks.workunit.client.1.smithi100.stdout:7/953: dwrite d5/d22/d3b/d9f/fcf [0,4194304] 0 2024-03-18T14:51:16.084 INFO:tasks.workunit.client.1.smithi100.stdout:6/915: dwrite d6/d9f/f10f [0,4194304] 0 2024-03-18T14:51:16.138 INFO:tasks.workunit.client.1.smithi100.stdout:2/933: mknod d3/da/de/d20/d82/dad/de2/c136 0 2024-03-18T14:51:16.162 INFO:tasks.workunit.client.1.smithi100.stdout:5/884: creat d29/d4a/f128 x:0 0 0 2024-03-18T14:51:16.201 INFO:tasks.workunit.client.1.smithi100.stdout:9/992: rename da/d51/c125 to da/dd/d3c/c145 0 2024-03-18T14:51:16.214 INFO:tasks.workunit.client.1.smithi100.stdout:7/954: creat d5/d14/d16/d44/f15c x:0 0 0 2024-03-18T14:51:16.214 INFO:tasks.workunit.client.1.smithi100.stdout:7/955: dread - d5/d30/d4d/db4/f150 zero size 2024-03-18T14:51:16.234 INFO:tasks.workunit.client.1.smithi100.stdout:6/916: mkdir d6/d9f/d126 0 2024-03-18T14:51:16.315 INFO:tasks.workunit.client.1.smithi100.stdout:1/918: sync 2024-03-18T14:51:16.327 INFO:tasks.workunit.client.1.smithi100.stdout:5/885: getdents d29/d2d/d32/d92/daa 0 2024-03-18T14:51:16.329 INFO:tasks.workunit.client.1.smithi100.stdout:9/993: symlink da/d88/da9/l146 0 2024-03-18T14:51:16.329 INFO:tasks.workunit.client.1.smithi100.stdout:9/994: dread - da/d88/f10f zero size 2024-03-18T14:51:16.330 INFO:tasks.workunit.client.1.smithi100.stdout:5/886: dread d29/d2d/d39/d61/f6b [0,4194304] 0 2024-03-18T14:51:16.330 INFO:tasks.workunit.client.1.smithi100.stdout:7/956: truncate d5/d14/d16/d63/d124/d66/f139 1568492 0 2024-03-18T14:51:16.335 INFO:tasks.workunit.client.1.smithi100.stdout:9/995: mkdir da/dd/d29/d71/d72/db2/d143/d147 0 2024-03-18T14:51:16.335 INFO:tasks.workunit.client.1.smithi100.stdout:9/996: chown da/l57 523262 1 2024-03-18T14:51:16.341 INFO:tasks.workunit.client.1.smithi100.stdout:2/934: link d3/d6/d5f/fc6 d3/d6/d5f/d63/f137 0 2024-03-18T14:51:16.342 INFO:tasks.workunit.client.1.smithi100.stdout:2/935: dread d3/d1a/d76/fb1 [0,4194304] 0 2024-03-18T14:51:16.342 INFO:tasks.workunit.client.1.smithi100.stdout:2/936: chown d3/d6/ff7 3 1 2024-03-18T14:51:16.350 INFO:tasks.workunit.client.1.smithi100.stdout:9/997: getdents da/d51/d55/d7e 0 2024-03-18T14:51:16.365 INFO:tasks.workunit.client.1.smithi100.stdout:4/912: sync 2024-03-18T14:51:16.366 INFO:tasks.workunit.client.1.smithi100.stdout:4/913: mkdir d9/d13/d5a/d5b/df9/d111 0 2024-03-18T14:51:16.376 INFO:tasks.workunit.client.1.smithi100.stdout:0/963: sync 2024-03-18T14:51:16.376 INFO:tasks.workunit.client.1.smithi100.stdout:4/914: dread d9/d27/d42/f6a [0,4194304] 0 2024-03-18T14:51:16.377 INFO:tasks.workunit.client.1.smithi100.stdout:0/964: rmdir dd/d61 39 2024-03-18T14:51:16.377 INFO:tasks.workunit.client.1.smithi100.stdout:0/965: fsync dd/d20/d44/f5e 0 2024-03-18T14:51:16.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/915: symlink d9/da6/l112 0 2024-03-18T14:51:16.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/916: dread - d9/d13/d5a/da4/fe9 zero size 2024-03-18T14:51:16.378 INFO:tasks.workunit.client.1.smithi100.stdout:4/917: write d9/d17/f24 [680097,111887] 0 2024-03-18T14:51:16.379 INFO:tasks.workunit.client.1.smithi100.stdout:0/966: creat dd/d61/dcd/d140/f15e x:0 0 0 2024-03-18T14:51:16.387 INFO:tasks.workunit.client.1.smithi100.stdout:3/943: sync 2024-03-18T14:51:16.387 INFO:tasks.workunit.client.1.smithi100.stdout:3/944: stat d3/dd/d21/dcd/d103 0 2024-03-18T14:51:16.387 INFO:tasks.workunit.client.1.smithi100.stdout:3/945: stat d3/d16/c108 0 2024-03-18T14:51:16.388 INFO:tasks.workunit.client.1.smithi100.stdout:3/946: mkdir d3/d16/d13/d1f/d3f/d4f/d65/d140 0 2024-03-18T14:51:16.388 INFO:tasks.workunit.client.1.smithi100.stdout:3/947: dread - d3/dd/d21/d3e/df8/df2/f11f zero size 2024-03-18T14:51:16.390 INFO:tasks.workunit.client.1.smithi100.stdout:3/948: symlink d3/da5/l141 0 2024-03-18T14:51:16.390 INFO:tasks.workunit.client.1.smithi100.stdout:3/949: write d3/dd/f27 [4760532,28703] 0 2024-03-18T14:51:16.390 INFO:tasks.workunit.client.1.smithi100.stdout:3/950: write d3/dd/d49/f4a [3892390,83784] 0 2024-03-18T14:51:16.415 INFO:tasks.workunit.client.1.smithi100.stdout:3/951: dwrite d3/dd/f6a [0,4194304] 0 2024-03-18T14:51:16.420 INFO:tasks.workunit.client.1.smithi100.stdout:4/918: dwrite d9/d13/ff2 [0,4194304] 0 2024-03-18T14:51:16.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/887: rmdir d29/d2d/d32/d92 39 2024-03-18T14:51:16.421 INFO:tasks.workunit.client.1.smithi100.stdout:5/888: fdatasync d29/d2d/d39/d74/f11d 0 2024-03-18T14:51:16.424 INFO:tasks.workunit.client.1.smithi100.stdout:5/889: dread d29/d4a/f9c [0,4194304] 0 2024-03-18T14:51:16.427 INFO:tasks.workunit.client.1.smithi100.stdout:4/919: creat d9/d13/d5a/d5b/f113 x:0 0 0 2024-03-18T14:51:16.429 INFO:tasks.workunit.client.1.smithi100.stdout:5/890: mkdir d29/d2d/d39/d61/d129 0 2024-03-18T14:51:16.429 INFO:tasks.workunit.client.1.smithi100.stdout:5/891: write d29/d2d/d32/d92/f9a [873555,67450] 0 2024-03-18T14:51:16.429 INFO:tasks.workunit.client.1.smithi100.stdout:5/892: write d29/d2d/d39/d60/f70 [1103272,66311] 0 2024-03-18T14:51:16.430 INFO:tasks.workunit.client.1.smithi100.stdout:4/920: getdents d9/d27 0 2024-03-18T14:51:16.430 INFO:tasks.workunit.client.1.smithi100.stdout:4/921: readlink d9/d13/l66 0 2024-03-18T14:51:16.431 INFO:tasks.workunit.client.1.smithi100.stdout:4/922: chown d9/d27/d62/d7b/d82/c91 2205 1 2024-03-18T14:51:16.434 INFO:tasks.workunit.client.1.smithi100.stdout:1/919: rename d8/d1e/d11/d27/d3f/f49 to d8/d1e/d11/d2d/f123 0 2024-03-18T14:51:16.434 INFO:tasks.workunit.client.1.smithi100.stdout:5/893: unlink d29/d2d/d39/d71/d96/c108 0 2024-03-18T14:51:16.441 INFO:tasks.workunit.client.1.smithi100.stdout:5/894: write d29/d2d/d39/d83/dc1/dbb/fa4 [5403632,78115] 0 2024-03-18T14:51:16.444 INFO:tasks.workunit.client.1.smithi100.stdout:9/998: rename da/dd/d29/c13f to da/dd/d3b/d103/c148 0 2024-03-18T14:51:16.447 INFO:tasks.workunit.client.1.smithi100.stdout:5/895: mknod d29/d2d/d8f/df4/c12a 0 2024-03-18T14:51:16.447 INFO:tasks.workunit.client.1.smithi100.stdout:1/920: read d8/d1e/d11/d6d/d59/f10e [2166672,78706] 0 2024-03-18T14:51:16.448 INFO:tasks.workunit.client.1.smithi100.stdout:4/923: dwrite d9/d3a/f64 [0,4194304] 0 2024-03-18T14:51:16.448 INFO:tasks.workunit.client.1.smithi100.stdout:9/999: mknod da/dd/d29/d144/c149 0 2024-03-18T14:51:16.452 INFO:tasks.workunit.client.1.smithi100.stdout:5/896: unlink d29/d2d/d39/d74/lf0 0 2024-03-18T14:51:16.455 INFO:tasks.workunit.client.1.smithi100.stdout:0/967: unlink dd/d20/f5f 0 2024-03-18T14:51:16.460 INFO:tasks.workunit.client.1.smithi100.stdout:1/921: mknod d8/d30/d61/c124 0 2024-03-18T14:51:16.463 INFO:tasks.workunit.client.1.smithi100.stdout:1/922: dread d8/d30/d61/d8d/f95 [0,4194304] 0 2024-03-18T14:51:16.463 INFO:tasks.workunit.client.1.smithi100.stdout:1/923: fdatasync f5 0 2024-03-18T14:51:16.468 INFO:tasks.workunit.client.1.smithi100.stdout:4/924: dwrite d9/d17/f24 [0,4194304] 0 2024-03-18T14:51:16.475 INFO:tasks.workunit.client.1.smithi100.stdout:1/924: creat d8/d30/d61/d8d/dde/f125 x:0 0 0 2024-03-18T14:51:16.479 INFO:tasks.workunit.client.1.smithi100.stdout:4/925: mknod d9/d13/d5a/d5b/da0/dcd/c114 0 2024-03-18T14:51:16.480 INFO:tasks.workunit.client.1.smithi100.stdout:1/925: rmdir d8/d1e/d11/d6d/d59 39 2024-03-18T14:51:16.480 INFO:tasks.workunit.client.1.smithi100.stdout:1/926: chown d8/d1e/d11/d6d/d59/caf 928354 1 2024-03-18T14:51:16.480 INFO:tasks.workunit.client.1.smithi100.stdout:4/926: write d9/d27/da2/fc3 [276924,67347] 0 2024-03-18T14:51:16.488 INFO:tasks.workunit.client.1.smithi100.stdout:1/927: write d8/d2a/d37/d50/d6f/d71/fd0 [1473578,86984] 0 2024-03-18T14:51:16.490 INFO:tasks.workunit.client.1.smithi100.stdout:7/957: rmdir d5/d22/d3b 39 2024-03-18T14:51:16.490 INFO:tasks.workunit.client.1.smithi100.stdout:7/958: creat d5/d14/d16/d63/d124/dd1/d86/de6/f15d x:0 0 0 2024-03-18T14:51:16.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:16 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:16.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:16 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:16.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:16 smithi012 ceph-mon[29182]: pgmap v12: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 312 MiB/s rd, 360 MiB/s wr, 560 op/s 2024-03-18T14:51:16.493 INFO:tasks.workunit.client.1.smithi100.stdout:1/928: creat d8/df5/f126 x:0 0 0 2024-03-18T14:51:16.495 INFO:tasks.workunit.client.1.smithi100.stdout:4/927: dwrite d9/d27/d62/d7b/f8e [0,4194304] 0 2024-03-18T14:51:16.495 INFO:tasks.workunit.client.1.smithi100.stdout:4/928: fsync d9/d27/f4b 0 2024-03-18T14:51:16.495 INFO:tasks.workunit.client.1.smithi100.stdout:7/959: creat d5/d22/d3b/d9f/f15e x:0 0 0 2024-03-18T14:51:16.500 INFO:tasks.workunit.client.1.smithi100.stdout:4/929: getdents d9/d27/d62 0 2024-03-18T14:51:16.505 INFO:tasks.workunit.client.1.smithi100.stdout:7/960: creat d5/d22/d24/d39/f15f x:0 0 0 2024-03-18T14:51:16.508 INFO:tasks.workunit.client.1.smithi100.stdout:1/929: link d8/l7c d8/d2a/d37/d50/d6f/df3/l127 0 2024-03-18T14:51:16.508 INFO:tasks.workunit.client.1.smithi100.stdout:1/930: fdatasync d8/fab 0 2024-03-18T14:51:16.510 INFO:tasks.workunit.client.1.smithi100.stdout:4/930: creat d9/d13/d39/dfd/d110/f115 x:0 0 0 2024-03-18T14:51:16.510 INFO:tasks.workunit.client.1.smithi100.stdout:4/931: creat d9/d13/d5a/d5b/f116 x:0 0 0 2024-03-18T14:51:16.512 INFO:tasks.workunit.client.1.smithi100.stdout:1/931: getdents d8/d2a/d37/d50/d6f/d81 0 2024-03-18T14:51:16.516 INFO:tasks.workunit.client.1.smithi100.stdout:1/932: creat d8/f128 x:0 0 0 2024-03-18T14:51:16.516 INFO:tasks.workunit.client.1.smithi100.stdout:1/933: write d8/d1e/d11/d2d/f6e [5456401,72677] 0 2024-03-18T14:51:16.519 INFO:tasks.workunit.client.1.smithi100.stdout:4/932: getdents d9/d13/d39/d79 0 2024-03-18T14:51:16.519 INFO:tasks.workunit.client.1.smithi100.stdout:4/933: truncate d9/d27/d62/d7b/fd4 971511 0 2024-03-18T14:51:16.523 INFO:tasks.workunit.client.1.smithi100.stdout:4/934: link d9/d17/l9f d9/d13/d39/dfd/d110/l117 0 2024-03-18T14:51:16.523 INFO:tasks.workunit.client.1.smithi100.stdout:4/935: creat d9/d13/d39/d76/da3/dca/f118 x:0 0 0 2024-03-18T14:51:16.528 INFO:tasks.workunit.client.1.smithi100.stdout:1/934: mkdir d8/d1e/d129 0 2024-03-18T14:51:16.531 INFO:tasks.workunit.client.1.smithi100.stdout:1/935: unlink d8/d30/d61/d8d/dde/de6/fe3 0 2024-03-18T14:51:16.534 INFO:tasks.workunit.client.1.smithi100.stdout:1/936: mknod d8/df5/c12a 0 2024-03-18T14:51:16.537 INFO:tasks.workunit.client.1.smithi100.stdout:1/937: mkdir d8/d2a/d37/d50/d6f/d71/d12b 0 2024-03-18T14:51:16.539 INFO:tasks.workunit.client.1.smithi100.stdout:1/938: creat d8/d1e/d11/d6d/d59/f12c x:0 0 0 2024-03-18T14:51:16.541 INFO:tasks.workunit.client.1.smithi100.stdout:1/939: creat d8/d2a/d37/d50/dc5/f12d x:0 0 0 2024-03-18T14:51:16.541 INFO:tasks.workunit.client.1.smithi100.stdout:1/940: stat f7 0 2024-03-18T14:51:16.543 INFO:tasks.workunit.client.1.smithi100.stdout:1/941: rmdir d8/d30/d61/db4 39 2024-03-18T14:51:16.544 INFO:tasks.workunit.client.1.smithi100.stdout:1/942: readlink d8/dd/l62 0 2024-03-18T14:51:16.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:16 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:16.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:16 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:16.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:16 smithi100 ceph-mon[37807]: pgmap v12: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 312 MiB/s rd, 360 MiB/s wr, 560 op/s 2024-03-18T14:51:16.546 INFO:tasks.workunit.client.1.smithi100.stdout:4/936: dwrite d9/d13/d39/d76/fcc [0,4194304] 0 2024-03-18T14:51:16.546 INFO:tasks.workunit.client.1.smithi100.stdout:4/937: fsync d9/d3a/fb3 0 2024-03-18T14:51:16.547 INFO:tasks.workunit.client.1.smithi100.stdout:1/943: mkdir d8/dd/d12e 0 2024-03-18T14:51:16.548 INFO:tasks.workunit.client.1.smithi100.stdout:1/944: chown d8/d1e/d11/d27/d83 9 1 2024-03-18T14:51:16.556 INFO:tasks.workunit.client.1.smithi100.stdout:0/968: rename dd/d31/d91/d7b/ce3 to dd/d31/d91/d56/c15f 0 2024-03-18T14:51:16.558 INFO:tasks.workunit.client.1.smithi100.stdout:4/938: dread d9/d13/d39/fad [0,4194304] 0 2024-03-18T14:51:16.561 INFO:tasks.workunit.client.1.smithi100.stdout:0/969: rename dd/d31/d91/d7b/cf0 to dd/d20/d44/df9/c160 0 2024-03-18T14:51:16.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/939: mkdir d9/dc8/d119 0 2024-03-18T14:51:16.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/940: fsync d9/f7f 0 2024-03-18T14:51:16.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/941: chown d9/d27/d62/d7b/l8a 3357494 1 2024-03-18T14:51:16.564 INFO:tasks.workunit.client.1.smithi100.stdout:3/952: truncate d3/dd/f27 3190988 0 2024-03-18T14:51:16.565 INFO:tasks.workunit.client.1.smithi100.stdout:0/970: mkdir dd/d31/d91/d56/d71/dcc/dcf/d120/d12f/d161 0 2024-03-18T14:51:16.569 INFO:tasks.workunit.client.1.smithi100.stdout:0/971: dread dd/fa6 [0,4194304] 0 2024-03-18T14:51:16.569 INFO:tasks.workunit.client.1.smithi100.stdout:0/972: write dd/d31/d91/d56/d59/f11d [1260572,61653] 0 2024-03-18T14:51:16.569 INFO:tasks.workunit.client.1.smithi100.stdout:0/973: write dd/d2b/f40 [765379,65772] 0 2024-03-18T14:51:16.572 INFO:tasks.workunit.client.1.smithi100.stdout:0/974: symlink dd/d31/d91/d56/d71/dcc/l162 0 2024-03-18T14:51:16.572 INFO:tasks.workunit.client.1.smithi100.stdout:0/975: readlink dd/d31/d91/d65/l111 0 2024-03-18T14:51:16.576 INFO:tasks.workunit.client.1.smithi100.stdout:0/976: dread dd/d31/d91/d56/d71/dcc/f124 [0,4194304] 0 2024-03-18T14:51:16.576 INFO:tasks.workunit.client.1.smithi100.stdout:0/977: write dd/d20/f27 [4739770,63410] 0 2024-03-18T14:51:16.576 INFO:tasks.workunit.client.1.smithi100.stdout:1/945: dread d8/d1e/d11/d8a/dda/ff2 [0,4194304] 0 2024-03-18T14:51:16.582 INFO:tasks.workunit.client.1.smithi100.stdout:6/917: creat d6/d7/df/d72/f127 x:0 0 0 2024-03-18T14:51:16.585 INFO:tasks.workunit.client.1.smithi100.stdout:6/918: stat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/c29 0 2024-03-18T14:51:16.586 INFO:tasks.workunit.client.1.smithi100.stdout:0/978: dread dd/f42 [0,4194304] 0 2024-03-18T14:51:16.587 INFO:tasks.workunit.client.1.smithi100.stdout:0/979: chown dd/d31/d91/d13b/da9/dac/fc6 82 1 2024-03-18T14:51:16.587 INFO:tasks.workunit.client.1.smithi100.stdout:0/980: stat dd/d31/d91/d56 0 2024-03-18T14:51:16.587 INFO:tasks.workunit.client.1.smithi100.stdout:0/981: fdatasync dd/d31/ffe 0 2024-03-18T14:51:16.588 INFO:tasks.workunit.client.1.smithi100.stdout:0/982: symlink dd/d61/dcd/d140/l163 0 2024-03-18T14:51:16.590 INFO:tasks.workunit.client.1.smithi100.stdout:1/946: read d8/d2a/d37/f66 [1394169,5824] 0 2024-03-18T14:51:16.590 INFO:tasks.workunit.client.1.smithi100.stdout:0/983: dread dd/d61/f100 [0,4194304] 0 2024-03-18T14:51:16.593 INFO:tasks.workunit.client.1.smithi100.stdout:1/947: rename d8/d1e/d11/d6d/d59/fd7 to d8/d1e/d11/d27/d73/d89/f12f 0 2024-03-18T14:51:16.594 INFO:tasks.workunit.client.1.smithi100.stdout:7/961: symlink d5/d22/d3b/d9f/da0/l160 0 2024-03-18T14:51:16.594 INFO:tasks.workunit.client.1.smithi100.stdout:7/962: fsync d5/d14/d16/d63/d124/d66/fc1 0 2024-03-18T14:51:16.595 INFO:tasks.workunit.client.1.smithi100.stdout:1/948: unlink d8/d2a/fc2 0 2024-03-18T14:51:16.597 INFO:tasks.workunit.client.1.smithi100.stdout:7/963: rename d5/d30/d4d/db4/d116/f133 to d5/d22/d3b/d9f/da0/db1/dd3/f161 0 2024-03-18T14:51:16.597 INFO:tasks.workunit.client.1.smithi100.stdout:7/964: write d5/d14/d16/d63/d124/dd1/d86/de6/f15d [641467,18992] 0 2024-03-18T14:51:16.598 INFO:tasks.workunit.client.1.smithi100.stdout:1/949: rmdir d8/d30/d61/db4 39 2024-03-18T14:51:16.599 INFO:tasks.workunit.client.1.smithi100.stdout:7/965: creat d5/d30/d142/f162 x:0 0 0 2024-03-18T14:51:16.599 INFO:tasks.workunit.client.1.smithi100.stdout:7/966: fsync d5/d30/f103 0 2024-03-18T14:51:16.600 INFO:tasks.workunit.client.1.smithi100.stdout:1/950: unlink d8/d1e/d11/d27/d3f/daa/dd5/ddc/fba 0 2024-03-18T14:51:16.601 INFO:tasks.workunit.client.1.smithi100.stdout:6/919: dwrite d6/d9f/fca [0,4194304] 0 2024-03-18T14:51:16.602 INFO:tasks.workunit.client.1.smithi100.stdout:7/967: rename d5/d14/d16/d63/fd6 to d5/d14/d16/d63/d124/dd1/d86/d85/f163 0 2024-03-18T14:51:16.604 INFO:tasks.workunit.client.1.smithi100.stdout:1/951: creat d8/d1e/d11/d27/d73/f130 x:0 0 0 2024-03-18T14:51:16.605 INFO:tasks.workunit.client.1.smithi100.stdout:1/952: read d8/d30/d61/d8d/dde/de6/fe7 [152283,105183] 0 2024-03-18T14:51:16.605 INFO:tasks.workunit.client.1.smithi100.stdout:1/953: chown d8/fab 929953399 1 2024-03-18T14:51:16.606 INFO:tasks.workunit.client.1.smithi100.stdout:1/954: write d8/d30/d61/fcb [160905,47388] 0 2024-03-18T14:51:16.606 INFO:tasks.workunit.client.1.smithi100.stdout:6/920: mknod d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/de4/c128 0 2024-03-18T14:51:16.610 INFO:tasks.workunit.client.1.smithi100.stdout:6/921: mknod d6/d7/df/d26/d4e/db5/dba/c129 0 2024-03-18T14:51:16.611 INFO:tasks.workunit.client.1.smithi100.stdout:7/968: dread d5/d22/d3b/d9f/fc8 [0,4194304] 0 2024-03-18T14:51:16.612 INFO:tasks.workunit.client.1.smithi100.stdout:1/955: mknod d8/c131 0 2024-03-18T14:51:16.613 INFO:tasks.workunit.client.1.smithi100.stdout:6/922: symlink d6/d12/d78/l12a 0 2024-03-18T14:51:16.615 INFO:tasks.workunit.client.1.smithi100.stdout:7/969: mkdir d5/d22/d24/d6e/d7a/dc2/d164 0 2024-03-18T14:51:16.615 INFO:tasks.workunit.client.1.smithi100.stdout:7/970: readlink d5/d22/d24/l12e 0 2024-03-18T14:51:16.618 INFO:tasks.workunit.client.1.smithi100.stdout:6/923: rmdir d6/d7/df/d26 39 2024-03-18T14:51:16.621 INFO:tasks.workunit.client.1.smithi100.stdout:6/924: mknod d6/d7/c12b 0 2024-03-18T14:51:16.622 INFO:tasks.workunit.client.1.smithi100.stdout:1/956: rename d8/d1e/d11/d27/d83 to d8/d1e/d11/d27/d132 0 2024-03-18T14:51:16.622 INFO:tasks.workunit.client.1.smithi100.stdout:1/957: write d8/d2a/d37/d50/dc5/f12d [320203,83600] 0 2024-03-18T14:51:16.642 INFO:tasks.workunit.client.1.smithi100.stdout:6/925: dwrite d6/d7/df/fed [0,4194304] 0 2024-03-18T14:51:16.654 INFO:tasks.workunit.client.1.smithi100.stdout:1/958: rename d8/d2a/d37/d50/d6f to d8/d133 0 2024-03-18T14:51:16.655 INFO:tasks.workunit.client.1.smithi100.stdout:1/959: mknod d8/d1e/d11/d27/d3f/daa/dd5/c134 0 2024-03-18T14:51:16.659 INFO:tasks.workunit.client.1.smithi100.stdout:1/960: write d8/d1e/d11/d27/d73/d89/f12f [107689,40923] 0 2024-03-18T14:51:16.659 INFO:tasks.workunit.client.1.smithi100.stdout:1/961: write d8/d2a/f47 [4612423,88084] 0 2024-03-18T14:51:16.748 INFO:tasks.workunit.client.1.smithi100.stdout:5/897: sync 2024-03-18T14:51:16.748 INFO:tasks.workunit.client.1.smithi100.stdout:2/937: sync 2024-03-18T14:51:16.748 INFO:tasks.workunit.client.1.smithi100.stdout:5/898: chown d29/d2d/d32/l40 2045 1 2024-03-18T14:51:16.749 INFO:tasks.workunit.client.1.smithi100.stdout:2/938: dread d3/d6/d29/d36/d4a/ff5 [0,4194304] 0 2024-03-18T14:51:16.750 INFO:tasks.workunit.client.1.smithi100.stdout:2/939: read - d3/d6/d5f/d63/fd9 zero size 2024-03-18T14:51:16.750 INFO:tasks.workunit.client.1.smithi100.stdout:5/899: creat d29/d2d/d39/d83/f12b x:0 0 0 2024-03-18T14:51:16.750 INFO:tasks.workunit.client.1.smithi100.stdout:5/900: truncate d29/d2d/d39/d83/f122 853773 0 2024-03-18T14:51:16.751 INFO:tasks.workunit.client.1.smithi100.stdout:5/901: chown d29/d2d/d32/d92/daa/la2 0 1 2024-03-18T14:51:16.751 INFO:tasks.workunit.client.1.smithi100.stdout:5/902: chown d29/d2d/d8f/df4 2 1 2024-03-18T14:51:16.751 INFO:tasks.workunit.client.1.smithi100.stdout:5/903: chown d29/d2d/d39/lf8 206259202 1 2024-03-18T14:51:16.753 INFO:tasks.workunit.client.1.smithi100.stdout:0/984: dwrite dd/d2b/f40 [0,4194304] 0 2024-03-18T14:51:16.757 INFO:tasks.workunit.client.1.smithi100.stdout:2/940: creat d3/d6/d29/d36/d49/f138 x:0 0 0 2024-03-18T14:51:16.766 INFO:tasks.workunit.client.1.smithi100.stdout:0/985: unlink dd/d31/c35 0 2024-03-18T14:51:16.769 INFO:tasks.workunit.client.1.smithi100.stdout:5/904: dwrite d29/f2b [0,4194304] 0 2024-03-18T14:51:16.770 INFO:tasks.workunit.client.1.smithi100.stdout:2/941: mknod d3/da/de/d20/dea/d40/d7d/c139 0 2024-03-18T14:51:16.773 INFO:tasks.workunit.client.1.smithi100.stdout:0/986: unlink dd/d2b/d95/d79/l144 0 2024-03-18T14:51:16.775 INFO:tasks.workunit.client.1.smithi100.stdout:2/942: dread d3/da/de/d20/dea/d40/d53/f85 [0,4194304] 0 2024-03-18T14:51:16.777 INFO:tasks.workunit.client.1.smithi100.stdout:5/905: creat d29/d2d/d39/d61/d126/f12c x:0 0 0 2024-03-18T14:51:16.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/943: truncate d3/d6/d5f/d6a/fb2 698153 0 2024-03-18T14:51:16.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/944: chown d3/da/de/d20/d82/dad/dc5 32411809 1 2024-03-18T14:51:16.780 INFO:tasks.workunit.client.1.smithi100.stdout:2/945: chown d3/d6/d5f 17 1 2024-03-18T14:51:16.782 INFO:tasks.workunit.client.1.smithi100.stdout:0/987: write dd/d31/d91/d56/ffa [2788499,104163] 0 2024-03-18T14:51:16.782 INFO:tasks.workunit.client.1.smithi100.stdout:0/988: stat dd/d31/d91/d56/d62/l134 0 2024-03-18T14:51:16.782 INFO:tasks.workunit.client.1.smithi100.stdout:5/906: symlink d29/d2d/d39/d71/l12d 0 2024-03-18T14:51:16.788 INFO:tasks.workunit.client.1.smithi100.stdout:2/946: truncate d3/da/de/d20/d82/fd8 3704685 0 2024-03-18T14:51:16.788 INFO:tasks.workunit.client.1.smithi100.stdout:2/947: dread - d3/d6/d5f/d63/fd9 zero size 2024-03-18T14:51:16.788 INFO:tasks.workunit.client.1.smithi100.stdout:2/948: chown d3/d1a/fbe 0 1 2024-03-18T14:51:16.792 INFO:tasks.workunit.client.1.smithi100.stdout:0/989: mkdir dd/d31/d91/d56/d59/d164 0 2024-03-18T14:51:16.797 INFO:tasks.workunit.client.1.smithi100.stdout:5/907: mknod d29/d2d/d32/d92/daa/c12e 0 2024-03-18T14:51:16.804 INFO:tasks.workunit.client.1.smithi100.stdout:2/949: write d3/da/de/d20/dea/d40/f83 [1591415,73904] 0 2024-03-18T14:51:16.808 INFO:tasks.workunit.client.1.smithi100.stdout:5/908: rename d29/d2d/d39/d71/lc9 to d29/d2d/d39/d65/l12f 0 2024-03-18T14:51:16.810 INFO:tasks.workunit.client.1.smithi100.stdout:2/950: rmdir d3/d6/d5f 39 2024-03-18T14:51:16.811 INFO:tasks.workunit.client.1.smithi100.stdout:5/909: mkdir d29/dd0/d130 0 2024-03-18T14:51:16.813 INFO:tasks.workunit.client.1.smithi100.stdout:2/951: symlink d3/d6/l13a 0 2024-03-18T14:51:16.814 INFO:tasks.workunit.client.1.smithi100.stdout:5/910: mkdir d29/d2d/d39/d131 0 2024-03-18T14:51:16.814 INFO:tasks.workunit.client.1.smithi100.stdout:5/911: fdatasync d29/d2d/d39/d83/dc1/dbb/f6e 0 2024-03-18T14:51:16.816 INFO:tasks.workunit.client.1.smithi100.stdout:2/952: mknod d3/da/de/d20/dea/d40/dac/c13b 0 2024-03-18T14:51:16.818 INFO:tasks.workunit.client.1.smithi100.stdout:2/953: stat d3/d1a/ccc 0 2024-03-18T14:51:16.818 INFO:tasks.workunit.client.1.smithi100.stdout:2/954: fsync d3/da/de/d20/f24 0 2024-03-18T14:51:16.823 INFO:tasks.workunit.client.1.smithi100.stdout:5/912: dread d29/d2d/d39/d83/dc1/dbb/fa4 [0,4194304] 0 2024-03-18T14:51:16.823 INFO:tasks.workunit.client.1.smithi100.stdout:5/913: chown d29/l3d 150 1 2024-03-18T14:51:16.823 INFO:tasks.workunit.client.1.smithi100.stdout:5/914: write d29/f8c [2737393,92344] 0 2024-03-18T14:51:16.823 INFO:tasks.workunit.client.1.smithi100.stdout:5/915: chown d29/d2d/dbe/dc2/fe7 2 1 2024-03-18T14:51:16.823 INFO:tasks.workunit.client.1.smithi100.stdout:2/955: dread d3/d6/d5f/f68 [0,4194304] 0 2024-03-18T14:51:16.827 INFO:tasks.workunit.client.1.smithi100.stdout:2/956: getdents d3/d6/d5f/d63 0 2024-03-18T14:51:16.829 INFO:tasks.workunit.client.1.smithi100.stdout:2/957: unlink d3/da/de/d20/f24 0 2024-03-18T14:51:16.830 INFO:tasks.workunit.client.1.smithi100.stdout:2/958: mknod d3/da/de/d20/c13c 0 2024-03-18T14:51:16.832 INFO:tasks.workunit.client.1.smithi100.stdout:2/959: rename d3/da/de/d20/d82/d95/f117 to d3/da/de/d20/dea/d40/d53/f13d 0 2024-03-18T14:51:16.832 INFO:tasks.workunit.client.1.smithi100.stdout:2/960: readlink d3/d6/d5f/d6a/l97 0 2024-03-18T14:51:16.837 INFO:tasks.workunit.client.1.smithi100.stdout:5/916: dwrite d29/d2d/d39/d61/f89 [4194304,4194304] 0 2024-03-18T14:51:16.840 INFO:tasks.workunit.client.1.smithi100.stdout:5/917: rename d29/d2d/d39/f106 to d29/d2d/d8f/f132 0 2024-03-18T14:51:16.847 INFO:tasks.workunit.client.1.smithi100.stdout:5/918: symlink d29/d2d/d39/d71/d96/l133 0 2024-03-18T14:51:16.847 INFO:tasks.workunit.client.1.smithi100.stdout:5/919: truncate d29/d2d/d39/d61/d126/f12c 87509 0 2024-03-18T14:51:16.847 INFO:tasks.workunit.client.1.smithi100.stdout:2/961: dread d3/d6/d5f/fe5 [0,4194304] 0 2024-03-18T14:51:16.847 INFO:tasks.workunit.client.1.smithi100.stdout:2/962: fsync d3/fba 0 2024-03-18T14:51:16.848 INFO:tasks.workunit.client.1.smithi100.stdout:5/920: unlink d29/d2d/d32/f68 0 2024-03-18T14:51:16.848 INFO:tasks.workunit.client.1.smithi100.stdout:2/963: symlink d3/d1a/l13e 0 2024-03-18T14:51:16.860 INFO:tasks.workunit.client.1.smithi100.stdout:2/964: link d3/da/de/d20/dea/d40/f104 d3/d1a/f13f 0 2024-03-18T14:51:16.871 INFO:tasks.workunit.client.1.smithi100.stdout:2/965: unlink d3/d1a/d76/f10c 0 2024-03-18T14:51:16.871 INFO:tasks.workunit.client.1.smithi100.stdout:2/966: stat d3/dd4/ff9 0 2024-03-18T14:51:16.875 INFO:tasks.workunit.client.1.smithi100.stdout:2/967: mknod d3/d6/d29/c140 0 2024-03-18T14:51:16.883 INFO:tasks.workunit.client.1.smithi100.stdout:2/968: dread d3/da/de/d20/dea/f38 [0,4194304] 0 2024-03-18T14:51:16.885 INFO:tasks.workunit.client.1.smithi100.stdout:2/969: link d3/d6/d29/c140 d3/da/de/d11f/c141 0 2024-03-18T14:51:16.887 INFO:tasks.workunit.client.1.smithi100.stdout:2/970: mknod d3/d6/d29/d36/c142 0 2024-03-18T14:51:16.903 INFO:tasks.workunit.client.1.smithi100.stdout:2/971: dwrite d3/da/de/d20/dea/d40/d7d/f118 [0,4194304] 0 2024-03-18T14:51:16.904 INFO:tasks.workunit.client.1.smithi100.stdout:2/972: write d3/d6/d5f/d63/fd9 [131315,55492] 0 2024-03-18T14:51:16.907 INFO:tasks.workunit.client.1.smithi100.stdout:2/973: mknod d3/dc3/c143 0 2024-03-18T14:51:16.921 INFO:tasks.workunit.client.1.smithi100.stdout:2/974: rmdir d3/da/de/d20/d82/dad/de2 39 2024-03-18T14:51:16.922 INFO:tasks.workunit.client.1.smithi100.stdout:4/942: sync 2024-03-18T14:51:16.922 INFO:tasks.workunit.client.1.smithi100.stdout:6/926: sync 2024-03-18T14:51:16.922 INFO:tasks.workunit.client.1.smithi100.stdout:7/971: sync 2024-03-18T14:51:16.922 INFO:tasks.workunit.client.1.smithi100.stdout:1/962: sync 2024-03-18T14:51:16.922 INFO:tasks.workunit.client.1.smithi100.stdout:3/953: sync 2024-03-18T14:51:16.923 INFO:tasks.workunit.client.1.smithi100.stdout:6/927: readlink d6/d12/l106 0 2024-03-18T14:51:16.925 INFO:tasks.workunit.client.1.smithi100.stdout:7/972: link d5/d22/db3/c11d d5/d22/d24/d6e/d105/c165 0 2024-03-18T14:51:16.926 INFO:tasks.workunit.client.1.smithi100.stdout:7/973: read - d5/d14/d16/d63/d124/dd1/d86/de6/d122/f131 zero size 2024-03-18T14:51:16.930 INFO:tasks.workunit.client.1.smithi100.stdout:1/963: rename d8/d2a/d37/d50/f5e to d8/d30/d109/f135 0 2024-03-18T14:51:16.932 INFO:tasks.workunit.client.1.smithi100.stdout:1/964: rmdir d8/d30/d109 39 2024-03-18T14:51:16.935 INFO:tasks.workunit.client.1.smithi100.stdout:6/928: rename d6/d7/df/d26/d4e/f82 to d6/d9f/d126/f12c 0 2024-03-18T14:51:16.935 INFO:tasks.workunit.client.1.smithi100.stdout:6/929: read - d6/d12/d78/f114 zero size 2024-03-18T14:51:16.938 INFO:tasks.workunit.client.1.smithi100.stdout:1/965: creat d8/d1e/d11/d27/d132/f136 x:0 0 0 2024-03-18T14:51:16.941 INFO:tasks.workunit.client.1.smithi100.stdout:7/974: rename d5/d14/d16/d63/d124/d66/c14c to d5/d14/d16/d63/c166 0 2024-03-18T14:51:16.943 INFO:tasks.workunit.client.1.smithi100.stdout:6/930: link d6/d7/df/d72/d48/f11f d6/d7/df/d26/d4e/d53/d110/d112/d36/f12d 0 2024-03-18T14:51:16.945 INFO:tasks.workunit.client.1.smithi100.stdout:1/966: creat d8/d1e/d11/d27/d73/d89/d104/f137 x:0 0 0 2024-03-18T14:51:16.946 INFO:tasks.workunit.client.1.smithi100.stdout:1/967: chown d8/d1e/d11/d27/d3f/daa/dd5/c134 1 1 2024-03-18T14:51:16.946 INFO:tasks.workunit.client.1.smithi100.stdout:7/975: mknod d5/d22/d24/d6e/d7a/dc2/d111/d129/c167 0 2024-03-18T14:51:16.948 INFO:tasks.workunit.client.1.smithi100.stdout:4/943: write d9/da6/fb6 [2927104,60220] 0 2024-03-18T14:51:16.948 INFO:tasks.workunit.client.1.smithi100.stdout:4/944: chown d9/d13/d39/d76 2357 1 2024-03-18T14:51:16.949 INFO:tasks.workunit.client.1.smithi100.stdout:3/954: dwrite d3/d16/d13/d1f/d3f/d4f/f68 [0,4194304] 0 2024-03-18T14:51:16.949 INFO:tasks.workunit.client.1.smithi100.stdout:2/975: dwrite d3/da/de/d20/dea/d40/dac/fbf [0,4194304] 0 2024-03-18T14:51:16.950 INFO:tasks.workunit.client.1.smithi100.stdout:1/968: creat d8/d30/d61/db4/f138 x:0 0 0 2024-03-18T14:51:16.951 INFO:tasks.workunit.client.1.smithi100.stdout:7/976: unlink d5/d14/d16/d63/cb8 0 2024-03-18T14:51:16.955 INFO:tasks.workunit.client.1.smithi100.stdout:6/931: getdents d6/d7/df/d26/d4e/d53 0 2024-03-18T14:51:16.960 INFO:tasks.workunit.client.1.smithi100.stdout:4/945: unlink d9/d27/c4e 0 2024-03-18T14:51:16.962 INFO:tasks.workunit.client.1.smithi100.stdout:2/976: dwrite d3/fba [0,4194304] 0 2024-03-18T14:51:16.964 INFO:tasks.workunit.client.1.smithi100.stdout:6/932: write d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/f58 [3420830,22164] 0 2024-03-18T14:51:16.967 INFO:tasks.workunit.client.1.smithi100.stdout:3/955: symlink d3/dd/l142 0 2024-03-18T14:51:16.968 INFO:tasks.workunit.client.1.smithi100.stdout:1/969: mknod d8/d30/d109/c139 0 2024-03-18T14:51:16.970 INFO:tasks.workunit.client.1.smithi100.stdout:7/977: mkdir d5/d22/d24/d168 0 2024-03-18T14:51:16.972 INFO:tasks.workunit.client.1.smithi100.stdout:4/946: rename d9/d27/d42/f4a to d9/d13/d5a/d5b/df9/d111/f11a 0 2024-03-18T14:51:16.973 INFO:tasks.workunit.client.1.smithi100.stdout:1/970: dread d8/d1e/d11/d6d/d59/f10e [0,4194304] 0 2024-03-18T14:51:16.973 INFO:tasks.workunit.client.1.smithi100.stdout:1/971: truncate d8/d1e/d11/d27/d73/d89/f12f 2992141 0 2024-03-18T14:51:16.981 INFO:tasks.workunit.client.1.smithi100.stdout:6/933: mknod d6/d7/df/d26/c12e 0 2024-03-18T14:51:16.981 INFO:tasks.workunit.client.1.smithi100.stdout:6/934: creat d6/d7/df/d72/f12f x:0 0 0 2024-03-18T14:51:16.981 INFO:tasks.workunit.client.1.smithi100.stdout:6/935: fdatasync d6/d7/df/d26/d4e/d53/d110/d112/d36/f12d 0 2024-03-18T14:51:16.984 INFO:tasks.workunit.client.1.smithi100.stdout:7/978: mkdir d5/d169 0 2024-03-18T14:51:16.990 INFO:tasks.workunit.client.1.smithi100.stdout:3/956: rename d3/dd/d21/dcd/fd9 to d3/dd/d21/d3e/df8/db9/df0/f143 0 2024-03-18T14:51:16.990 INFO:tasks.workunit.client.1.smithi100.stdout:1/972: dwrite d8/d30/f40 [0,4194304] 0 2024-03-18T14:51:16.992 INFO:tasks.workunit.client.1.smithi100.stdout:4/947: symlink d9/d13/d39/dfd/l11b 0 2024-03-18T14:51:16.992 INFO:tasks.workunit.client.1.smithi100.stdout:4/948: fsync d9/d27/d42/f6a 0 2024-03-18T14:51:16.992 INFO:tasks.workunit.client.1.smithi100.stdout:4/949: chown d9/d27/f2e 2 1 2024-03-18T14:51:16.993 INFO:tasks.workunit.client.1.smithi100.stdout:2/977: getdents d3/dd4 0 2024-03-18T14:51:16.998 INFO:tasks.workunit.client.1.smithi100.stdout:6/936: creat d6/d7/df/d26/d4e/db5/f130 x:0 0 0 2024-03-18T14:51:17.006 INFO:tasks.workunit.client.1.smithi100.stdout:6/937: mknod d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/de8/c131 0 2024-03-18T14:51:17.006 INFO:tasks.workunit.client.1.smithi100.stdout:6/938: truncate d6/d9f/fe9 271378 0 2024-03-18T14:51:17.006 INFO:tasks.workunit.client.1.smithi100.stdout:6/939: creat d6/d7/df/d26/d4e/db5/f132 x:0 0 0 2024-03-18T14:51:17.008 INFO:tasks.workunit.client.1.smithi100.stdout:2/978: link d3/d6/c128 d3/d1a/d6c/c144 0 2024-03-18T14:51:17.012 INFO:tasks.workunit.client.1.smithi100.stdout:7/979: getdents d5 0 2024-03-18T14:51:17.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/979: symlink d3/da/de/d20/d82/dad/l145 0 2024-03-18T14:51:17.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/980: chown d3/da/de/d20/dea/d40/dac/l132 343 1 2024-03-18T14:51:17.015 INFO:tasks.workunit.client.1.smithi100.stdout:2/981: rename d3/da/de to d3/da/de/d20/dea/d4f/d146 22 2024-03-18T14:51:17.015 INFO:tasks.workunit.client.1.smithi100.stdout:3/957: dwrite d3/fd2 [0,4194304] 0 2024-03-18T14:51:17.016 INFO:tasks.workunit.client.1.smithi100.stdout:1/973: dread d8/d1e/d11/d2d/f123 [0,4194304] 0 2024-03-18T14:51:17.016 INFO:tasks.workunit.client.1.smithi100.stdout:2/982: creat d3/da/de/d20/dea/d40/d7d/f147 x:0 0 0 2024-03-18T14:51:17.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/940: rmdir d6/d7/df/d72/d65/d7a 39 2024-03-18T14:51:17.018 INFO:tasks.workunit.client.1.smithi100.stdout:6/941: fdatasync d6/d12/fdb 0 2024-03-18T14:51:17.020 INFO:tasks.workunit.client.1.smithi100.stdout:7/980: unlink d5/d14/d16/d63/d124/l93 0 2024-03-18T14:51:17.021 INFO:tasks.workunit.client.1.smithi100.stdout:5/921: sync 2024-03-18T14:51:17.022 INFO:tasks.workunit.client.1.smithi100.stdout:0/990: sync 2024-03-18T14:51:17.022 INFO:tasks.workunit.client.1.smithi100.stdout:5/922: truncate d29/d2d/d39/d71/d96/fa3 673169 0 2024-03-18T14:51:17.022 INFO:tasks.workunit.client.1.smithi100.stdout:5/923: creat d29/d2d/d39/d61/f134 x:0 0 0 2024-03-18T14:51:17.022 INFO:tasks.workunit.client.1.smithi100.stdout:5/924: write d29/d2d/d32/f7c [1148962,91526] 0 2024-03-18T14:51:17.031 INFO:tasks.workunit.client.1.smithi100.stdout:1/974: creat d8/d133/d71/db7/f13a x:0 0 0 2024-03-18T14:51:17.032 INFO:tasks.workunit.client.1.smithi100.stdout:3/958: dwrite d3/ff [0,4194304] 0 2024-03-18T14:51:17.033 INFO:tasks.workunit.client.1.smithi100.stdout:3/959: creat d3/d16/d13/d1f/d37/f144 x:0 0 0 2024-03-18T14:51:17.035 INFO:tasks.workunit.client.1.smithi100.stdout:2/983: creat d3/da/f148 x:0 0 0 2024-03-18T14:51:17.037 INFO:tasks.workunit.client.1.smithi100.stdout:5/925: write d29/d2d/dbe/dc2/fe7 [3335650,108414] 0 2024-03-18T14:51:17.037 INFO:tasks.workunit.client.1.smithi100.stdout:6/942: mkdir d6/d7/df/d26/d4e/db5/dba/d133 0 2024-03-18T14:51:17.038 INFO:tasks.workunit.client.1.smithi100.stdout:6/943: fdatasync d6/d7/df/d72/f85 0 2024-03-18T14:51:17.038 INFO:tasks.workunit.client.1.smithi100.stdout:7/981: creat d5/d90/f16a x:0 0 0 2024-03-18T14:51:17.038 INFO:tasks.workunit.client.1.smithi100.stdout:7/982: write d5/d14/d16/d63/d124/d66/d4f/fd8 [2470477,35586] 0 2024-03-18T14:51:17.044 INFO:tasks.workunit.client.1.smithi100.stdout:0/991: rename dd/d31/d91/d56/d62/l134 to dd/d31/d91/d56/d71/dcc/l165 0 2024-03-18T14:51:17.044 INFO:tasks.workunit.client.1.smithi100.stdout:0/992: creat dd/d31/d91/d56/de0/d113/f166 x:0 0 0 2024-03-18T14:51:17.050 INFO:tasks.workunit.client.1.smithi100.stdout:1/975: creat d8/d2a/d37/d50/d11d/f13b x:0 0 0 2024-03-18T14:51:17.050 INFO:tasks.workunit.client.1.smithi100.stdout:1/976: stat d8/d2a/d37/d50/f100 0 2024-03-18T14:51:17.050 INFO:tasks.workunit.client.1.smithi100.stdout:1/977: creat d8/d133/d71/db7/f13c x:0 0 0 2024-03-18T14:51:17.052 INFO:tasks.workunit.client.1.smithi100.stdout:7/983: read d5/fd [946907,99437] 0 2024-03-18T14:51:17.057 INFO:tasks.workunit.client.1.smithi100.stdout:2/984: truncate d3/dd4/ff9 3138635 0 2024-03-18T14:51:17.062 INFO:tasks.workunit.client.1.smithi100.stdout:6/944: symlink d6/d7/df/d72/d65/d7a/da5/l134 0 2024-03-18T14:51:17.063 INFO:tasks.workunit.client.1.smithi100.stdout:0/993: dwrite dd/d31/d91/d13b/da9/dac/f14a [0,4194304] 0 2024-03-18T14:51:17.064 INFO:tasks.workunit.client.1.smithi100.stdout:1/978: write d8/d1e/d11/d27/d132/fd4 [2448959,36101] 0 2024-03-18T14:51:17.065 INFO:tasks.workunit.client.1.smithi100.stdout:7/984: dread d5/d30/d4d/d5a/f78 [0,4194304] 0 2024-03-18T14:51:17.065 INFO:tasks.workunit.client.1.smithi100.stdout:7/985: chown d5/d14/d16/d63/d124/dc3/l15a 16 1 2024-03-18T14:51:17.065 INFO:tasks.workunit.client.1.smithi100.stdout:2/985: write d3/d6/d5f/fe5 [3288683,12575] 0 2024-03-18T14:51:17.070 INFO:tasks.workunit.client.1.smithi100.stdout:5/926: rmdir d29/d2d/d8f 39 2024-03-18T14:51:17.077 INFO:tasks.workunit.client.1.smithi100.stdout:0/994: creat dd/d31/d91/d65/d143/f167 x:0 0 0 2024-03-18T14:51:17.082 INFO:tasks.workunit.client.1.smithi100.stdout:6/945: link d6/d7/df/d26/d4e/l9e d6/d7/df/d72/d8b/l135 0 2024-03-18T14:51:17.084 INFO:tasks.workunit.client.1.smithi100.stdout:0/995: mknod dd/d31/d91/d56/d71/dcc/dcf/c168 0 2024-03-18T14:51:17.086 INFO:tasks.workunit.client.1.smithi100.stdout:2/986: dread d3/dd4/ff9 [0,4194304] 0 2024-03-18T14:51:17.086 INFO:tasks.workunit.client.1.smithi100.stdout:2/987: truncate d3/da/de/fa5 4985779 0 2024-03-18T14:51:17.087 INFO:tasks.workunit.client.1.smithi100.stdout:7/986: dwrite d5/d14/d16/d63/d124/dd1/d86/de6/d122/f131 [0,4194304] 0 2024-03-18T14:51:17.087 INFO:tasks.workunit.client.1.smithi100.stdout:7/987: getdents d5/d169 0 2024-03-18T14:51:17.093 INFO:tasks.workunit.client.1.smithi100.stdout:3/960: dwrite d3/f57 [0,4194304] 0 2024-03-18T14:51:17.096 INFO:tasks.workunit.client.1.smithi100.stdout:0/996: truncate dd/d2b/d95/d79/fa4 832996 0 2024-03-18T14:51:17.096 INFO:tasks.workunit.client.1.smithi100.stdout:0/997: read - dd/d31/d91/d56/d62/d6e/f119 zero size 2024-03-18T14:51:17.107 INFO:tasks.workunit.client.1.smithi100.stdout:0/998: rename dd/d61/dcd/dff/c138 to dd/d31/d91/d56/d71/dcc/dcf/d120/d13a/c169 0 2024-03-18T14:51:17.107 INFO:tasks.workunit.client.1.smithi100.stdout:0/999: write dd/d31/d91/d13b/da9/dac/f14a [4794544,42754] 0 2024-03-18T14:51:17.107 INFO:tasks.workunit.client.1.smithi100.stdout:6/946: dwrite d6/d7/df/d72/d65/fa9 [0,4194304] 0 2024-03-18T14:51:17.108 INFO:tasks.workunit.client.1.smithi100.stdout:3/961: write d3/fb6 [1145013,79713] 0 2024-03-18T14:51:17.110 INFO:tasks.workunit.client.1.smithi100.stdout:2/988: dwrite d3/da/de/d6e/f115 [0,4194304] 0 2024-03-18T14:51:17.118 INFO:tasks.workunit.client.1.smithi100.stdout:6/947: creat d6/d7/f136 x:0 0 0 2024-03-18T14:51:17.118 INFO:tasks.workunit.client.1.smithi100.stdout:3/962: dread d3/d16/d13/d1f/d3f/d4f/d65/feb [0,4194304] 0 2024-03-18T14:51:17.121 INFO:tasks.workunit.client.1.smithi100.stdout:7/988: dwrite d5/d90/da1/f13c [0,4194304] 0 2024-03-18T14:51:17.123 INFO:tasks.workunit.client.1.smithi100.stdout:2/989: getdents d3/d1a/d6c 0 2024-03-18T14:51:17.128 INFO:tasks.workunit.client.1.smithi100.stdout:7/989: mknod d5/d22/d3b/c16b 0 2024-03-18T14:51:17.128 INFO:tasks.workunit.client.1.smithi100.stdout:7/990: dread - d5/d14/d16/d63/d124/dd1/d107/f152 zero size 2024-03-18T14:51:17.130 INFO:tasks.workunit.client.1.smithi100.stdout:7/991: mknod d5/d14/d16/d63/d124/d66/d4f/dcb/c16c 0 2024-03-18T14:51:17.130 INFO:tasks.workunit.client.1.smithi100.stdout:3/963: dread d3/d16/d13/d1f/d3f/d4f/de0/ffd [0,4194304] 0 2024-03-18T14:51:17.130 INFO:tasks.workunit.client.1.smithi100.stdout:3/964: fdatasync d3/dd/d21/d3e/f79 0 2024-03-18T14:51:17.131 INFO:tasks.workunit.client.1.smithi100.stdout:7/992: symlink d5/d14/d16/d63/d124/dd1/d86/de6/d122/l16d 0 2024-03-18T14:51:17.132 INFO:tasks.workunit.client.1.smithi100.stdout:6/948: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d27/f9c [0,4194304] 0 2024-03-18T14:51:17.132 INFO:tasks.workunit.client.1.smithi100.stdout:6/949: chown d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d8a/cc6 3213705 1 2024-03-18T14:51:17.132 INFO:tasks.workunit.client.1.smithi100.stdout:3/965: creat d3/dd/d21/d3e/df8/db9/df0/f145 x:0 0 0 2024-03-18T14:51:17.139 INFO:tasks.workunit.client.1.smithi100.stdout:7/993: getdents d5/d22/d3b/d47 0 2024-03-18T14:51:17.139 INFO:tasks.workunit.client.1.smithi100.stdout:7/994: read - d5/d30/d142/f162 zero size 2024-03-18T14:51:17.139 INFO:tasks.workunit.client.1.smithi100.stdout:7/995: fsync d5/d14/d16/d63/d124/d66/d4f/dcb/f14d 0 2024-03-18T14:51:17.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/966: dread d3/fd5 [0,4194304] 0 2024-03-18T14:51:17.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/967: truncate d3/d16/d13/d1f/d3f/d4f/de0/de6/f137 970686 0 2024-03-18T14:51:17.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/968: chown d3/d16/d13/f39 209369223 1 2024-03-18T14:51:17.141 INFO:tasks.workunit.client.1.smithi100.stdout:3/969: chown d3/d114 92908371 1 2024-03-18T14:51:17.141 INFO:tasks.workunit.client.1.smithi100.stdout:6/950: dread d6/d12/da2/f104 [0,4194304] 0 2024-03-18T14:51:17.142 INFO:tasks.workunit.client.1.smithi100.stdout:6/951: readlink d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/l109 0 2024-03-18T14:51:17.142 INFO:tasks.workunit.client.1.smithi100.stdout:6/952: write d6/d7/df/d26/d4e/d53/f113 [873470,110878] 0 2024-03-18T14:51:17.143 INFO:tasks.workunit.client.1.smithi100.stdout:3/970: dread d3/d16/d13/d17/f7a [0,4194304] 0 2024-03-18T14:51:17.144 INFO:tasks.workunit.client.1.smithi100.stdout:7/996: symlink d5/d14/d16/d63/d124/dd1/l16e 0 2024-03-18T14:51:17.147 INFO:tasks.workunit.client.1.smithi100.stdout:6/953: mknod d6/d7/df/d26/d4e/d53/d110/d112/c137 0 2024-03-18T14:51:17.147 INFO:tasks.workunit.client.1.smithi100.stdout:6/954: stat d6/d9f/fca 0 2024-03-18T14:51:17.148 INFO:tasks.workunit.client.1.smithi100.stdout:3/971: unlink d3/da5/f106 0 2024-03-18T14:51:17.150 INFO:tasks.workunit.client.1.smithi100.stdout:7/997: symlink d5/d14/d16/d63/d124/dd1/d86/d13e/l16f 0 2024-03-18T14:51:17.152 INFO:tasks.workunit.client.1.smithi100.stdout:6/955: creat d6/d7/df/d72/d65/f138 x:0 0 0 2024-03-18T14:51:17.153 INFO:tasks.workunit.client.1.smithi100.stdout:3/972: rename d3/dd/d21/c12d to d3/d16/d13/d1f/d3f/d4f/d65/de9/c146 0 2024-03-18T14:51:17.154 INFO:tasks.workunit.client.1.smithi100.stdout:6/956: creat d6/d7/df/d26/d4e/db5/de1/d107/f139 x:0 0 0 2024-03-18T14:51:17.156 INFO:tasks.workunit.client.1.smithi100.stdout:3/973: symlink d3/d16/d13/d1f/l147 0 2024-03-18T14:51:17.158 INFO:tasks.workunit.client.1.smithi100.stdout:3/974: readlink d3/dd/d21/lb8 0 2024-03-18T14:51:17.159 INFO:tasks.workunit.client.1.smithi100.stdout:3/975: mkdir d3/dd/d21/d3e/df8/db9/df0/d148 0 2024-03-18T14:51:17.160 INFO:tasks.workunit.client.1.smithi100.stdout:7/998: write d5/d14/d16/d63/d124/dd1/f43 [2714198,26903] 0 2024-03-18T14:51:17.160 INFO:tasks.workunit.client.1.smithi100.stdout:7/999: read d5/d30/f10e [466779,50074] 0 2024-03-18T14:51:17.162 INFO:tasks.workunit.client.1.smithi100.stdout:3/976: mkdir d3/dd/d149 0 2024-03-18T14:51:17.164 INFO:tasks.workunit.client.1.smithi100.stdout:3/977: creat d3/d16/d13/d1f/d3f/d4f/f14a x:0 0 0 2024-03-18T14:51:17.164 INFO:tasks.workunit.client.1.smithi100.stdout:3/978: dread - d3/dd/d21/d3e/df8/db9/df0/f145 zero size 2024-03-18T14:51:17.169 INFO:tasks.workunit.client.1.smithi100.stdout:3/979: symlink d3/dd/l14b 0 2024-03-18T14:51:17.169 INFO:tasks.workunit.client.1.smithi100.stdout:3/980: stat d3/dd/d21/d3e/fcc 0 2024-03-18T14:51:17.171 INFO:tasks.workunit.client.1.smithi100.stdout:3/981: rename d3/d16/d13/d1f/d37/fb3 to d3/dd/dd0/f14c 0 2024-03-18T14:51:17.171 INFO:tasks.workunit.client.1.smithi100.stdout:6/957: dwrite d6/d7/df/d26/d4e/db5/dba/dfe/fbf [0,4194304] 0 2024-03-18T14:51:17.173 INFO:tasks.workunit.client.1.smithi100.stdout:3/982: unlink d3/dd/d49/f70 0 2024-03-18T14:51:17.173 INFO:tasks.workunit.client.1.smithi100.stdout:3/983: write d3/da5/ff9 [1425720,21308] 0 2024-03-18T14:51:17.177 INFO:tasks.workunit.client.1.smithi100.stdout:6/958: link d6/d7/df/d72/d48/f4a d6/d7/df/d26/d4e/db5/de1/f13a 0 2024-03-18T14:51:17.178 INFO:tasks.workunit.client.1.smithi100.stdout:3/984: creat d3/dd/d49/f14d x:0 0 0 2024-03-18T14:51:17.180 INFO:tasks.workunit.client.1.smithi100.stdout:6/959: mknod d6/c13b 0 2024-03-18T14:51:17.181 INFO:tasks.workunit.client.1.smithi100.stdout:3/985: mkdir d3/d16/d13/d1f/d3f/d4f/d65/d140/d14e 0 2024-03-18T14:51:17.182 INFO:tasks.workunit.client.1.smithi100.stdout:6/960: mknod d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d125/c13c 0 2024-03-18T14:51:17.184 INFO:tasks.workunit.client.1.smithi100.stdout:3/986: mknod d3/d16/d13/d1f/d3f/d4f/d65/de9/c14f 0 2024-03-18T14:51:17.185 INFO:tasks.workunit.client.1.smithi100.stdout:6/961: link d6/cd d6/d7/df/d72/d65/d7a/c13d 0 2024-03-18T14:51:17.188 INFO:tasks.workunit.client.1.smithi100.stdout:3/987: mknod d3/d16/d13/d1f/d3f/d4f/d65/de9/c150 0 2024-03-18T14:51:17.190 INFO:tasks.workunit.client.1.smithi100.stdout:6/962: symlink d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/l13e 0 2024-03-18T14:51:17.191 INFO:tasks.workunit.client.1.smithi100.stdout:6/963: stat d6/d12/d78/f114 0 2024-03-18T14:51:17.191 INFO:tasks.workunit.client.1.smithi100.stdout:3/988: mknod d3/dd/d21/d3e/df8/db9/df0/d148/c151 0 2024-03-18T14:51:17.192 INFO:tasks.workunit.client.1.smithi100.stdout:6/964: mknod d6/d12/d2c/c13f 0 2024-03-18T14:51:17.192 INFO:tasks.workunit.client.1.smithi100.stdout:6/965: truncate d6/d7/df/d26/d4e/d53/d110/d112/d36/f12d 980629 0 2024-03-18T14:51:17.193 INFO:tasks.workunit.client.1.smithi100.stdout:3/989: mkdir d3/d16/d13/d1f/d3f/d4f/d152 0 2024-03-18T14:51:17.194 INFO:tasks.workunit.client.1.smithi100.stdout:3/990: symlink d3/dd/d10d/l153 0 2024-03-18T14:51:17.194 INFO:tasks.workunit.client.1.smithi100.stdout:3/991: truncate d3/dd/d21/d3e/df8/db9/df0/f143 1694939 0 2024-03-18T14:51:17.196 INFO:tasks.workunit.client.1.smithi100.stdout:3/992: creat d3/d16/d13/d17/dab/f154 x:0 0 0 2024-03-18T14:51:17.196 INFO:tasks.workunit.client.1.smithi100.stdout:3/993: readlink d3/dd/d21/d3e/df8/db9/df0/l138 0 2024-03-18T14:51:17.197 INFO:tasks.workunit.client.1.smithi100.stdout:3/994: creat d3/d16/d13/d1f/d3f/d4f/d65/de9/f155 x:0 0 0 2024-03-18T14:51:17.204 INFO:tasks.workunit.client.1.smithi100.stdout:3/995: mknod d3/dd/d21/d3e/df8/db9/df0/d148/c156 0 2024-03-18T14:51:17.210 INFO:tasks.workunit.client.1.smithi100.stdout:3/996: symlink d3/d16/d13/d17/dab/l157 0 2024-03-18T14:51:17.217 INFO:tasks.workunit.client.1.smithi100.stdout:3/997: dread d3/fc1 [0,4194304] 0 2024-03-18T14:51:17.217 INFO:tasks.workunit.client.1.smithi100.stdout:3/998: stat d3/d16/dca/d10e 0 2024-03-18T14:51:17.218 INFO:tasks.workunit.client.1.smithi100.stdout:3/999: stat d3/d16/d13/d1f/d3f/d4f/d65/de9/c14f 0 2024-03-18T14:51:17.235 INFO:tasks.workunit.client.1.smithi100.stdout:4/950: sync 2024-03-18T14:51:17.235 INFO:tasks.workunit.client.1.smithi100.stdout:4/951: write d9/d27/fa9 [2106373,77786] 0 2024-03-18T14:51:17.235 INFO:tasks.workunit.client.1.smithi100.stdout:4/952: truncate d9/d13/d25/fa5 890277 0 2024-03-18T14:51:17.243 INFO:tasks.workunit.client.1.smithi100.stdout:4/953: write d9/d3a/fb3 [2141586,106376] 0 2024-03-18T14:51:17.245 INFO:tasks.workunit.client.1.smithi100.stdout:4/954: mknod d9/d27/d62/d109/c11c 0 2024-03-18T14:51:17.247 INFO:tasks.workunit.client.1.smithi100.stdout:4/955: creat d9/d27/d62/f11d x:0 0 0 2024-03-18T14:51:17.248 INFO:tasks.workunit.client.1.smithi100.stdout:4/956: symlink d9/d27/d62/d7b/l11e 0 2024-03-18T14:51:17.248 INFO:tasks.workunit.client.1.smithi100.stdout:4/957: dread - d9/d13/d39/d79/f97 zero size 2024-03-18T14:51:17.318 INFO:tasks.workunit.client.1.smithi100.stdout:5/927: sync 2024-03-18T14:51:17.319 INFO:tasks.workunit.client.1.smithi100.stdout:2/990: sync 2024-03-18T14:51:17.319 INFO:tasks.workunit.client.1.smithi100.stdout:1/979: sync 2024-03-18T14:51:17.319 INFO:tasks.workunit.client.1.smithi100.stdout:2/991: truncate d3/ffe 396833 0 2024-03-18T14:51:17.319 INFO:tasks.workunit.client.1.smithi100.stdout:5/928: creat d29/d2d/d39/d61/d126/f135 x:0 0 0 2024-03-18T14:51:17.320 INFO:tasks.workunit.client.1.smithi100.stdout:1/980: mkdir d8/d1e/d11/d8a/d13d 0 2024-03-18T14:51:17.321 INFO:tasks.workunit.client.1.smithi100.stdout:2/992: symlink d3/da/de/d20/d82/dad/de2/l149 0 2024-03-18T14:51:17.322 INFO:tasks.workunit.client.1.smithi100.stdout:5/929: mknod d29/dd0/d130/c136 0 2024-03-18T14:51:17.333 INFO:tasks.workunit.client.1.smithi100.stdout:5/930: dread d29/f2b [0,4194304] 0 2024-03-18T14:51:17.333 INFO:tasks.workunit.client.1.smithi100.stdout:5/931: fdatasync d29/f2c 0 2024-03-18T14:51:17.334 INFO:tasks.workunit.client.1.smithi100.stdout:5/932: dread - d29/d2d/d39/d61/f134 zero size 2024-03-18T14:51:17.334 INFO:tasks.workunit.client.1.smithi100.stdout:5/933: readlink l3 0 2024-03-18T14:51:17.334 INFO:tasks.workunit.client.1.smithi100.stdout:5/934: read - d29/d2d/d39/d83/dc1/dbb/d9e/dd7/df9/f124 zero size 2024-03-18T14:51:17.334 INFO:tasks.workunit.client.1.smithi100.stdout:5/935: write d29/d4a/f54 [1448757,55409] 0 2024-03-18T14:51:17.335 INFO:tasks.workunit.client.1.smithi100.stdout:5/936: rename d29/d2d/d32/c3c to d29/d2d/d8f/c137 0 2024-03-18T14:51:17.346 INFO:tasks.workunit.client.1.smithi100.stdout:1/981: dwrite d8/d1e/d11/d27/d73/f11c [0,4194304] 0 2024-03-18T14:51:17.349 INFO:tasks.workunit.client.1.smithi100.stdout:1/982: getdents d8/d2a/d37/d50/dc5 0 2024-03-18T14:51:17.350 INFO:tasks.workunit.client.1.smithi100.stdout:1/983: getdents d8/d1e/d11/d6d/d59 0 2024-03-18T14:51:17.350 INFO:tasks.workunit.client.1.smithi100.stdout:1/984: stat d8/d30/d61/da2 0 2024-03-18T14:51:17.352 INFO:tasks.workunit.client.1.smithi100.stdout:1/985: dread d8/d30/d109/f135 [0,4194304] 0 2024-03-18T14:51:17.353 INFO:tasks.workunit.client.1.smithi100.stdout:2/993: dwrite d3/d6/d29/d36/d49/f54 [0,4194304] 0 2024-03-18T14:51:17.354 INFO:tasks.workunit.client.1.smithi100.stdout:5/937: dwrite d29/d2d/d39/d83/dc1/dbb/fa7 [0,4194304] 0 2024-03-18T14:51:17.354 INFO:tasks.workunit.client.1.smithi100.stdout:1/986: rename d8/d1e/d11/d27/d132/fd4 to d8/d1e/d11/d27/d3f/daa/dd1/f13e 0 2024-03-18T14:51:17.355 INFO:tasks.workunit.client.1.smithi100.stdout:2/994: symlink d3/d6/d29/d36/d4a/l14a 0 2024-03-18T14:51:17.360 INFO:tasks.workunit.client.1.smithi100.stdout:2/995: creat d3/da/de/d20/d82/dad/dc5/de3/f14b x:0 0 0 2024-03-18T14:51:17.361 INFO:tasks.workunit.client.1.smithi100.stdout:1/987: getdents d8/d1e/d11/d8a/dda 0 2024-03-18T14:51:17.361 INFO:tasks.workunit.client.1.smithi100.stdout:1/988: write d8/d30/d61/d8d/f106 [728865,80440] 0 2024-03-18T14:51:17.365 INFO:tasks.workunit.client.1.smithi100.stdout:2/996: link d3/da/f148 d3/d6/d29/f14c 0 2024-03-18T14:51:17.365 INFO:tasks.workunit.client.1.smithi100.stdout:2/997: write d3/da/de/d20/dea/d40/f83 [1658223,91127] 0 2024-03-18T14:51:17.366 INFO:tasks.workunit.client.1.smithi100.stdout:1/989: unlink d8/d1e/d11/d27/d3f/f116 0 2024-03-18T14:51:17.368 INFO:tasks.workunit.client.1.smithi100.stdout:1/990: mkdir d8/d1e/d11/d27/d73/d89/d13f 0 2024-03-18T14:51:17.372 INFO:tasks.workunit.client.1.smithi100.stdout:1/991: mkdir d8/d133/d140 0 2024-03-18T14:51:17.372 INFO:tasks.workunit.client.1.smithi100.stdout:1/992: fdatasync d8/d1e/d11/d27/d3f/daa/dd5/f120 0 2024-03-18T14:51:17.375 INFO:tasks.workunit.client.1.smithi100.stdout:1/993: mknod d8/d30/d61/d8d/dde/de6/c141 0 2024-03-18T14:51:17.375 INFO:tasks.workunit.client.1.smithi100.stdout:1/994: chown d8/d1e/d11/d2d 101268 1 2024-03-18T14:51:17.376 INFO:tasks.workunit.client.1.smithi100.stdout:1/995: mkdir d8/d133/d142 0 2024-03-18T14:51:17.377 INFO:tasks.workunit.client.1.smithi100.stdout:6/966: sync 2024-03-18T14:51:17.378 INFO:tasks.workunit.client.1.smithi100.stdout:2/998: dwrite d3/d1a/d76/f11c [0,4194304] 0 2024-03-18T14:51:17.378 INFO:tasks.workunit.client.1.smithi100.stdout:2/999: dread - d3/da/de/d20/dea/fa6 zero size 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:1/996: creat d8/d2a/d37/d50/f143 x:0 0 0 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:1/997: truncate d8/f7e 5018396 0 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:1/998: stat d8/d1e/d11/d8a/d13d 0 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:1/999: readlink d8/d133/d71/ldf 0 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:6/967: symlink d6/d7/df/df1/l140 0 2024-03-18T14:51:17.379 INFO:tasks.workunit.client.1.smithi100.stdout:6/968: truncate d6/d7/df/d72/d48/d76/fad 1666329 0 2024-03-18T14:51:17.396 INFO:tasks.workunit.client.1.smithi100.stdout:6/969: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/fa0 [0,4194304] 0 2024-03-18T14:51:17.396 INFO:tasks.workunit.client.1.smithi100.stdout:6/970: write d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/de4/f10a [470368,114510] 0 2024-03-18T14:51:17.411 INFO:tasks.workunit.client.1.smithi100.stdout:6/971: dwrite d6/d12/d78/fb4 [0,4194304] 0 2024-03-18T14:51:17.422 INFO:tasks.workunit.client.1.smithi100.stdout:6/972: unlink d6/d12/d2c/cb1 0 2024-03-18T14:51:17.431 INFO:tasks.workunit.client.1.smithi100.stdout:4/958: sync 2024-03-18T14:51:17.431 INFO:tasks.workunit.client.1.smithi100.stdout:4/959: chown d9/d13/d25 64888661 1 2024-03-18T14:51:17.432 INFO:tasks.workunit.client.1.smithi100.stdout:4/960: dread - d9/d27/d62/d7b/d82/dac/ffa zero size 2024-03-18T14:51:17.434 INFO:tasks.workunit.client.1.smithi100.stdout:5/938: sync 2024-03-18T14:51:17.434 INFO:tasks.workunit.client.1.smithi100.stdout:5/939: chown d29/d2d/d39/d71/d96/dc8/l103 25347350 1 2024-03-18T14:51:17.436 INFO:tasks.workunit.client.1.smithi100.stdout:5/940: unlink d29/d2d/d39/d65/f6a 0 2024-03-18T14:51:17.436 INFO:tasks.workunit.client.1.smithi100.stdout:5/941: write d29/d2d/d39/f4e [1951606,128753] 0 2024-03-18T14:51:17.437 INFO:tasks.workunit.client.1.smithi100.stdout:4/961: mknod d9/d13/d25/c11f 0 2024-03-18T14:51:17.438 INFO:tasks.workunit.client.1.smithi100.stdout:5/942: mknod d29/d2d/d8f/df4/c138 0 2024-03-18T14:51:17.438 INFO:tasks.workunit.client.1.smithi100.stdout:5/943: readlink d29/d2d/d39/d83/dc1/dbb/d9e/dd7/df9/lfd 0 2024-03-18T14:51:17.439 INFO:tasks.workunit.client.1.smithi100.stdout:4/962: creat d9/d13/d39/d76/f120 x:0 0 0 2024-03-18T14:51:17.439 INFO:tasks.workunit.client.1.smithi100.stdout:6/973: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/fab [4194304,4194304] 0 2024-03-18T14:51:17.441 INFO:tasks.workunit.client.1.smithi100.stdout:6/974: symlink d6/d7/df/d72/d48/l141 0 2024-03-18T14:51:17.445 INFO:tasks.workunit.client.1.smithi100.stdout:6/975: creat d6/d12/f142 x:0 0 0 2024-03-18T14:51:17.470 INFO:tasks.workunit.client.1.smithi100.stdout:4/963: dwrite d9/d27/d42/f72 [0,4194304] 0 2024-03-18T14:51:17.470 INFO:tasks.workunit.client.1.smithi100.stdout:5/944: dwrite d29/d2d/d39/d61/f6b [0,4194304] 0 2024-03-18T14:51:17.470 INFO:tasks.workunit.client.1.smithi100.stdout:6/976: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d27/d64/fa3 [0,4194304] 0 2024-03-18T14:51:17.470 INFO:tasks.workunit.client.1.smithi100.stdout:5/945: write d29/d2d/d39/d61/d126/f135 [892185,77146] 0 2024-03-18T14:51:17.472 INFO:tasks.workunit.client.1.smithi100.stdout:4/964: link d9/d27/d62/f11d d9/d27/d42/f121 0 2024-03-18T14:51:17.472 INFO:tasks.workunit.client.1.smithi100.stdout:4/965: write d9/f2b [792435,39373] 0 2024-03-18T14:51:17.472 INFO:tasks.workunit.client.1.smithi100.stdout:4/966: truncate d9/d13/d5a/d5b/da0/dcd/f103 963481 0 2024-03-18T14:51:17.475 INFO:tasks.workunit.client.1.smithi100.stdout:6/977: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/f143 x:0 0 0 2024-03-18T14:51:17.476 INFO:tasks.workunit.client.1.smithi100.stdout:5/946: symlink d29/d2d/l139 0 2024-03-18T14:51:17.476 INFO:tasks.workunit.client.1.smithi100.stdout:5/947: write d29/d2d/f6f [4615475,29615] 0 2024-03-18T14:51:17.482 INFO:tasks.workunit.client.1.smithi100.stdout:4/967: rename d9/da6 to d9/d13/d5a/d122 0 2024-03-18T14:51:17.489 INFO:tasks.workunit.client.1.smithi100.stdout:5/948: rename d29/d2d/d39/d61/ddd to d29/d2d/d39/d65/d13a 0 2024-03-18T14:51:17.489 INFO:tasks.workunit.client.1.smithi100.stdout:5/949: fdatasync d29/d2d/d8f/df4/f10f 0 2024-03-18T14:51:17.491 INFO:tasks.workunit.client.1.smithi100.stdout:5/950: mknod d29/d2d/d39/d83/dc1/dbb/c13b 0 2024-03-18T14:51:17.491 INFO:tasks.workunit.client.1.smithi100.stdout:5/951: chown f2 29299367 1 2024-03-18T14:51:17.493 INFO:tasks.workunit.client.1.smithi100.stdout:5/952: link d29/d2d/d39/l51 d29/d2d/d39/dfe/l13c 0 2024-03-18T14:51:17.494 INFO:tasks.workunit.client.1.smithi100.stdout:5/953: mkdir d29/d2d/d39/d13d 0 2024-03-18T14:51:17.496 INFO:tasks.workunit.client.1.smithi100.stdout:6/978: dwrite d6/d7/df/d26/d4e/d53/d110/d112/f68 [0,4194304] 0 2024-03-18T14:51:17.496 INFO:tasks.workunit.client.1.smithi100.stdout:5/954: mkdir d29/d2d/d13e 0 2024-03-18T14:51:17.496 INFO:tasks.workunit.client.1.smithi100.stdout:5/955: chown d29/d2d/d39/d83/dc1/dbb/f6e 751 1 2024-03-18T14:51:17.502 INFO:tasks.workunit.client.1.smithi100.stdout:4/968: dwrite d9/d17/f45 [0,4194304] 0 2024-03-18T14:51:17.509 INFO:tasks.workunit.client.1.smithi100.stdout:4/969: getdents d9/d27/d62/d7b/d82/dac 0 2024-03-18T14:51:17.517 INFO:tasks.workunit.client.1.smithi100.stdout:5/956: dwrite d29/d2d/d39/d83/f12b [0,4194304] 0 2024-03-18T14:51:17.517 INFO:tasks.workunit.client.1.smithi100.stdout:6/979: dwrite d6/d7/f8d [0,4194304] 0 2024-03-18T14:51:17.523 INFO:tasks.workunit.client.1.smithi100.stdout:6/980: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d7f/f144 x:0 0 0 2024-03-18T14:51:17.530 INFO:tasks.workunit.client.1.smithi100.stdout:4/970: dwrite d9/d27/d62/d7b/d82/dac/ffa [0,4194304] 0 2024-03-18T14:51:17.535 INFO:tasks.workunit.client.1.smithi100.stdout:5/957: dwrite d29/d2d/dbe/dc2/fe7 [0,4194304] 0 2024-03-18T14:51:17.543 INFO:tasks.workunit.client.1.smithi100.stdout:5/958: rmdir d29/d2d/d39/d71 39 2024-03-18T14:51:17.548 INFO:tasks.workunit.client.1.smithi100.stdout:5/959: dread f25 [0,4194304] 0 2024-03-18T14:51:17.548 INFO:tasks.workunit.client.1.smithi100.stdout:4/971: dwrite d9/d13/d39/d76/f94 [0,4194304] 0 2024-03-18T14:51:17.548 INFO:tasks.workunit.client.1.smithi100.stdout:5/960: stat d29/d2d/d39/d83/dc1/dbb/d9e/dd7/f107 0 2024-03-18T14:51:17.548 INFO:tasks.workunit.client.1.smithi100.stdout:5/961: readlink d29/d2d/d32/d92/lc6 0 2024-03-18T14:51:17.549 INFO:tasks.workunit.client.1.smithi100.stdout:4/972: read d9/f32 [500005,63231] 0 2024-03-18T14:51:17.552 INFO:tasks.workunit.client.1.smithi100.stdout:4/973: mknod d9/d13/d5a/da4/c123 0 2024-03-18T14:51:17.555 INFO:tasks.workunit.client.1.smithi100.stdout:5/962: creat d29/d2d/d39/d83/dc1/dbb/d9e/f13f x:0 0 0 2024-03-18T14:51:17.555 INFO:tasks.workunit.client.1.smithi100.stdout:5/963: fsync d29/d2d/d39/d61/f6b 0 2024-03-18T14:51:17.555 INFO:tasks.workunit.client.1.smithi100.stdout:5/964: write d29/d4a/f128 [864870,122293] 0 2024-03-18T14:51:17.556 INFO:tasks.workunit.client.1.smithi100.stdout:4/974: symlink d9/d13/d39/d4c/df5/l124 0 2024-03-18T14:51:17.558 INFO:tasks.workunit.client.1.smithi100.stdout:4/975: rename d9/d13/d5a/d5b/df9 to d9/d27/d62/d7b/d82/d125 0 2024-03-18T14:51:17.561 INFO:tasks.workunit.client.1.smithi100.stdout:4/976: rename d9/d13/d25/l2c to d9/d13/d5a/l126 0 2024-03-18T14:51:17.561 INFO:tasks.workunit.client.1.smithi100.stdout:4/977: truncate d9/d13/d5a/d5b/f116 107962 0 2024-03-18T14:51:17.563 INFO:tasks.workunit.client.1.smithi100.stdout:4/978: symlink d9/d13/d5a/d5b/da0/dcd/l127 0 2024-03-18T14:51:17.564 INFO:tasks.workunit.client.1.smithi100.stdout:4/979: creat d9/d13/d25/d51/f128 x:0 0 0 2024-03-18T14:51:17.569 INFO:tasks.workunit.client.1.smithi100.stdout:5/965: dwrite d29/d2d/d39/d61/f6b [0,4194304] 0 2024-03-18T14:51:17.577 INFO:tasks.workunit.client.1.smithi100.stdout:5/966: write d29/d2d/d39/d71/fd3 [1157595,72816] 0 2024-03-18T14:51:17.579 INFO:tasks.workunit.client.1.smithi100.stdout:5/967: creat d29/d2d/d39/d83/dc1/d10b/f140 x:0 0 0 2024-03-18T14:51:17.579 INFO:tasks.workunit.client.1.smithi100.stdout:5/968: dread - d29/d2d/d39/d61/ff3 zero size 2024-03-18T14:51:17.579 INFO:tasks.workunit.client.1.smithi100.stdout:4/980: dwrite d9/d13/d39/d76/fcc [0,4194304] 0 2024-03-18T14:51:17.588 INFO:tasks.workunit.client.1.smithi100.stdout:4/981: rename d9/d13/d5a/c5c to d9/d13/d5a/d5b/da0/dd8/c129 0 2024-03-18T14:51:17.597 INFO:tasks.workunit.client.1.smithi100.stdout:4/982: dread d9/d27/d42/f72 [0,4194304] 0 2024-03-18T14:51:17.603 INFO:tasks.workunit.client.1.smithi100.stdout:4/983: truncate d9/d13/d5a/d122/fb6 3898328 0 2024-03-18T14:51:17.603 INFO:tasks.workunit.client.1.smithi100.stdout:4/984: chown d9/f11 184137 1 2024-03-18T14:51:17.603 INFO:tasks.workunit.client.1.smithi100.stdout:4/985: chown d9/d27/d62/d7b 8877 1 2024-03-18T14:51:17.603 INFO:tasks.workunit.client.1.smithi100.stdout:5/969: dwrite d29/d2d/d39/d83/dc1/dbb/fa4 [0,4194304] 0 2024-03-18T14:51:17.603 INFO:tasks.workunit.client.1.smithi100.stdout:5/970: chown d29/d2d/d39/d65/d13a/f111 1455378 1 2024-03-18T14:51:17.605 INFO:tasks.workunit.client.1.smithi100.stdout:4/986: symlink d9/d13/d39/d76/l12a 0 2024-03-18T14:51:17.605 INFO:tasks.workunit.client.1.smithi100.stdout:4/987: fdatasync d9/d27/d62/f73 0 2024-03-18T14:51:17.606 INFO:tasks.workunit.client.1.smithi100.stdout:4/988: chown d9/d27/d42/f121 12 1 2024-03-18T14:51:17.606 INFO:tasks.workunit.client.1.smithi100.stdout:6/981: sync 2024-03-18T14:51:17.610 INFO:tasks.workunit.client.1.smithi100.stdout:5/971: mkdir d29/d2d/d39/d60/d141 0 2024-03-18T14:51:17.611 INFO:tasks.workunit.client.1.smithi100.stdout:5/972: write d29/f8c [1553987,119661] 0 2024-03-18T14:51:17.611 INFO:tasks.workunit.client.1.smithi100.stdout:5/973: truncate d29/fa6 931421 0 2024-03-18T14:51:17.611 INFO:tasks.workunit.client.1.smithi100.stdout:5/974: fsync d29/d2d/d39/f46 0 2024-03-18T14:51:17.615 INFO:tasks.workunit.client.1.smithi100.stdout:6/982: link d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/f143 d6/d12/d78/f145 0 2024-03-18T14:51:17.618 INFO:tasks.workunit.client.1.smithi100.stdout:5/975: creat d29/d2d/d39/d61/d129/f142 x:0 0 0 2024-03-18T14:51:17.618 INFO:tasks.workunit.client.1.smithi100.stdout:5/976: dread - d29/d2d/dbe/dc2/f127 zero size 2024-03-18T14:51:17.620 INFO:tasks.workunit.client.1.smithi100.stdout:4/989: dwrite d9/dc8/fd6 [0,4194304] 0 2024-03-18T14:51:17.620 INFO:tasks.workunit.client.1.smithi100.stdout:4/990: creat d9/d13/d5a/d5b/f12b x:0 0 0 2024-03-18T14:51:17.626 INFO:tasks.workunit.client.1.smithi100.stdout:5/977: dread d29/d2d/d32/d92/fed [0,4194304] 0 2024-03-18T14:51:17.627 INFO:tasks.workunit.client.1.smithi100.stdout:5/978: chown d29/f50 45 1 2024-03-18T14:51:17.628 INFO:tasks.workunit.client.1.smithi100.stdout:4/991: dread d9/dc8/fd6 [0,4194304] 0 2024-03-18T14:51:17.629 INFO:tasks.workunit.client.1.smithi100.stdout:5/979: symlink d29/d2d/d8f/l143 0 2024-03-18T14:51:17.630 INFO:tasks.workunit.client.1.smithi100.stdout:5/980: stat f15 0 2024-03-18T14:51:17.631 INFO:tasks.workunit.client.1.smithi100.stdout:4/992: symlink d9/d13/d39/d76/da3/l12c 0 2024-03-18T14:51:17.633 INFO:tasks.workunit.client.1.smithi100.stdout:4/993: unlink d9/d27/da2/ffe 0 2024-03-18T14:51:17.634 INFO:tasks.workunit.client.1.smithi100.stdout:4/994: dread d9/d3a/f86 [0,4194304] 0 2024-03-18T14:51:17.638 INFO:tasks.workunit.client.1.smithi100.stdout:4/995: symlink d9/d13/d39/d4c/l12d 0 2024-03-18T14:51:17.639 INFO:tasks.workunit.client.1.smithi100.stdout:4/996: truncate d9/d13/d39/f10b 950790 0 2024-03-18T14:51:17.652 INFO:tasks.workunit.client.1.smithi100.stdout:6/983: dwrite d6/d7/f5f [0,4194304] 0 2024-03-18T14:51:17.653 INFO:tasks.workunit.client.1.smithi100.stdout:6/984: link d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/l100 d6/d7/df/d72/dc9/l146 0 2024-03-18T14:51:17.654 INFO:tasks.workunit.client.1.smithi100.stdout:6/985: stat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/l100 0 2024-03-18T14:51:17.655 INFO:tasks.workunit.client.1.smithi100.stdout:6/986: truncate d6/d12/f43 4247741 0 2024-03-18T14:51:17.676 INFO:tasks.workunit.client.1.smithi100.stdout:6/987: dwrite d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/fae [0,4194304] 0 2024-03-18T14:51:17.678 INFO:tasks.workunit.client.1.smithi100.stdout:6/988: mknod d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/c147 0 2024-03-18T14:51:17.678 INFO:tasks.workunit.client.1.smithi100.stdout:6/989: write d6/d7/df/fed [4281587,14199] 0 2024-03-18T14:51:17.681 INFO:tasks.workunit.client.1.smithi100.stdout:6/990: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/f148 x:0 0 0 2024-03-18T14:51:17.681 INFO:tasks.workunit.client.1.smithi100.stdout:6/991: creat d6/d9f/f149 x:0 0 0 2024-03-18T14:51:17.683 INFO:tasks.workunit.client.1.smithi100.stdout:6/992: read d6/d7/df/d72/d48/d76/fad [1125326,73335] 0 2024-03-18T14:51:17.683 INFO:tasks.workunit.client.1.smithi100.stdout:6/993: dread - d6/d7/df/d72/d65/d7a/da5/ff9 zero size 2024-03-18T14:51:17.692 INFO:tasks.workunit.client.1.smithi100.stdout:6/994: write d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/dcf/fab [7610881,125109] 0 2024-03-18T14:51:17.694 INFO:tasks.workunit.client.1.smithi100.stdout:6/995: creat d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/f14a x:0 0 0 2024-03-18T14:51:17.696 INFO:tasks.workunit.client.1.smithi100.stdout:6/996: unlink d6/d7/df/d26/d4e/d53/d110/d112/c39 0 2024-03-18T14:51:17.697 INFO:tasks.workunit.client.1.smithi100.stdout:6/997: write d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/db8/fe0 [401652,778] 0 2024-03-18T14:51:17.700 INFO:tasks.workunit.client.1.smithi100.stdout:6/998: rename d6/d7/df/d26/d4e/d53/d110/d112/d36/d4f/d71/d27/f5d to d6/d9f/f14b 0 2024-03-18T14:51:17.700 INFO:tasks.workunit.client.1.smithi100.stdout:6/999: write d6/d7/df/d26/d4e/d53/d110/d112/d36/f12d [904150,81164] 0 2024-03-18T14:51:17.741 INFO:tasks.workunit.client.1.smithi100.stdout:5/981: sync 2024-03-18T14:51:17.742 INFO:tasks.workunit.client.1.smithi100.stdout:5/982: mkdir d29/d2d/d39/d83/dc1/dbb/d144 0 2024-03-18T14:51:17.743 INFO:tasks.workunit.client.1.smithi100.stdout:5/983: write d29/d2d/f11f [4958897,3050] 0 2024-03-18T14:51:17.744 INFO:tasks.workunit.client.1.smithi100.stdout:5/984: dread - d29/d2d/d39/d83/dc1/dbb/f112 zero size 2024-03-18T14:51:17.746 INFO:tasks.workunit.client.1.smithi100.stdout:5/985: unlink d29/d2d/d39/c58 0 2024-03-18T14:51:17.749 INFO:tasks.workunit.client.1.smithi100.stdout:5/986: getdents d29 0 2024-03-18T14:51:17.749 INFO:tasks.workunit.client.1.smithi100.stdout:5/987: fsync d29/d2d/d39/d61/d126/f135 0 2024-03-18T14:51:17.757 INFO:tasks.workunit.client.1.smithi100.stdout:5/988: dread d29/d2d/d39/f46 [0,4194304] 0 2024-03-18T14:51:17.758 INFO:tasks.workunit.client.1.smithi100.stdout:5/989: dread - d29/d2d/d39/d83/dc1/d10b/f140 zero size 2024-03-18T14:51:17.760 INFO:tasks.workunit.client.1.smithi100.stdout:5/990: truncate d29/dd0/ff6 3338693 0 2024-03-18T14:51:17.763 INFO:tasks.workunit.client.1.smithi100.stdout:5/991: rmdir d29/d2d/d39/d71 39 2024-03-18T14:51:17.763 INFO:tasks.workunit.client.1.smithi100.stdout:5/992: fdatasync d29/d2d/d39/d65/d13a/f111 0 2024-03-18T14:51:17.766 INFO:tasks.workunit.client.1.smithi100.stdout:5/993: creat d29/dd0/d10d/f145 x:0 0 0 2024-03-18T14:51:17.769 INFO:tasks.workunit.client.1.smithi100.stdout:5/994: link d29/d2d/d39/d71/f77 d29/d4a/f146 0 2024-03-18T14:51:17.769 INFO:tasks.workunit.client.1.smithi100.stdout:5/995: write d29/d2d/d39/d83/dc1/dbb/f5d [1922938,58350] 0 2024-03-18T14:51:17.769 INFO:tasks.workunit.client.1.smithi100.stdout:5/996: fdatasync d29/d2d/f10e 0 2024-03-18T14:51:17.771 INFO:tasks.workunit.client.1.smithi100.stdout:5/997: unlink d29/f9b 0 2024-03-18T14:51:17.771 INFO:tasks.workunit.client.1.smithi100.stdout:5/998: fsync d29/d2d/d39/d83/f122 0 2024-03-18T14:51:17.772 INFO:tasks.workunit.client.1.smithi100.stdout:5/999: mkdir d29/d2d/d13e/d147 0 2024-03-18T14:51:17.784 INFO:tasks.workunit.client.1.smithi100.stdout:4/997: sync 2024-03-18T14:51:17.797 INFO:tasks.workunit.client.1.smithi100.stdout:4/998: dwrite d9/d3a/f64 [4194304,4194304] 0 2024-03-18T14:51:17.805 INFO:tasks.workunit.client.1.smithi100.stdout:4/999: rename d9/d27/d62/d7b/d82/dac/c100 to d9/d17/de1/c12e 0 2024-03-18T14:51:17.809 INFO:tasks.workunit.client.1.smithi100.stderr:+ rm -rf -- ./tmp.uLlJmv3TXW 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi012 to 1559M 2024-03-18T14:51:18.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:17 smithi100 ceph-mon[37807]: pgmap v13: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 194 MiB/s rd, 233 MiB/s wr, 353 op/s 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:18.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi012 to 1559M 2024-03-18T14:51:18.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:18.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:17 smithi012 ceph-mon[29182]: pgmap v13: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 194 MiB/s rd, 233 MiB/s wr, 353 op/s 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Adjusting osd_memory_target on smithi100 to 2581M 2024-03-18T14:51:19.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: pgmap v14: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 244 MiB/s rd, 306 MiB/s wr, 491 op/s 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:51:19.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:19 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:51:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Adjusting osd_memory_target on smithi100 to 2581M 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: pgmap v14: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 244 MiB/s rd, 306 MiB/s wr, 491 op/s 2024-03-18T14:51:20.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:51:20.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:19 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:20.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: Reconfiguring node-exporter.smithi012 (dependencies changed)... 2024-03-18T14:51:20.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:20 smithi012 ceph-mon[29182]: Reconfiguring daemon node-exporter.smithi012 on smithi012 2024-03-18T14:51:21.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: Reconfiguring node-exporter.smithi012 (dependencies changed)... 2024-03-18T14:51:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:20 smithi100 ceph-mon[37807]: Reconfiguring daemon node-exporter.smithi012 on smithi012 2024-03-18T14:51:21.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:21 smithi012 ceph-mon[29182]: pgmap v15: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 321 op/s 2024-03-18T14:51:22.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:21 smithi100 ceph-mon[37807]: pgmap v15: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 321 op/s 2024-03-18T14:51:23.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:23 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:23.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:23 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:24.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:23 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:23 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:24.773 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:24 smithi012 ceph-mon[29182]: Reconfiguring alertmanager.smithi012 (dependencies changed)... 2024-03-18T14:51:24.773 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:24 smithi012 ceph-mon[29182]: Reconfiguring daemon alertmanager.smithi012 on smithi012 2024-03-18T14:51:24.774 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:24 smithi012 ceph-mon[29182]: pgmap v16: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 321 op/s 2024-03-18T14:51:25.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:24 smithi100 ceph-mon[37807]: Reconfiguring alertmanager.smithi012 (dependencies changed)... 2024-03-18T14:51:25.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:24 smithi100 ceph-mon[37807]: Reconfiguring daemon alertmanager.smithi012 on smithi012 2024-03-18T14:51:25.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:24 smithi100 ceph-mon[37807]: pgmap v16: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 321 op/s 2024-03-18T14:51:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:25 smithi012 ceph-mon[29182]: pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 409 op/s 2024-03-18T14:51:26.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:25 smithi100 ceph-mon[37807]: pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 150 MiB/s rd, 192 MiB/s wr, 409 op/s 2024-03-18T14:51:28.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:28 smithi012 ceph-mon[29182]: pgmap v18: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 50 MiB/s rd, 73 MiB/s wr, 226 op/s 2024-03-18T14:51:28.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:28 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:28.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:28 smithi100 ceph-mon[37807]: pgmap v18: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 50 MiB/s rd, 73 MiB/s wr, 226 op/s 2024-03-18T14:51:28.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:28 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:29 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:29 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:29 smithi012 ceph-mon[29182]: Reconfiguring grafana.smithi012 (dependencies changed)... 2024-03-18T14:51:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:29 smithi012 ceph-mon[29182]: Reconfiguring daemon grafana.smithi012 on smithi012 2024-03-18T14:51:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:29 smithi012 ceph-mon[29182]: pgmap v19: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 51 MiB/s rd, 74 MiB/s wr, 313 op/s 2024-03-18T14:51:30.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:29 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:30.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:29 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:51:30.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:29 smithi100 ceph-mon[37807]: Reconfiguring grafana.smithi012 (dependencies changed)... 2024-03-18T14:51:30.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:29 smithi100 ceph-mon[37807]: Reconfiguring daemon grafana.smithi012 on smithi012 2024-03-18T14:51:30.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:29 smithi100 ceph-mon[37807]: pgmap v19: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 51 MiB/s rd, 74 MiB/s wr, 313 op/s 2024-03-18T14:51:31.174 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-18T14:51:31.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-18T14:51:32.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:32 smithi012 ceph-mon[29182]: pgmap v20: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-03-18T14:51:32.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:32 smithi100 ceph-mon[37807]: pgmap v20: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-03-18T14:51:34.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:33 smithi012 ceph-mon[29182]: pgmap v21: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-03-18T14:51:34.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:33 smithi100 ceph-mon[37807]: pgmap v21: 65 pgs: 65 active+clean; 1.6 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-03-18T14:51:34.767 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-18T14:51:34.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-18T14:51:36.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:36 smithi012 ceph-mon[29182]: pgmap v22: 65 pgs: 65 active+clean; 658 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-03-18T14:51:36.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:36 smithi100 ceph-mon[37807]: pgmap v22: 65 pgs: 65 active+clean; 658 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-03-18T14:51:38.724 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:38 smithi012 ceph-mon[29182]: pgmap v23: 65 pgs: 65 active+clean; 658 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-03-18T14:51:38.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:38 smithi100 ceph-mon[37807]: pgmap v23: 65 pgs: 65 active+clean; 658 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-03-18T14:51:40.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:40 smithi012 ceph-mon[29182]: pgmap v24: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-03-18T14:51:40.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:40 smithi100 ceph-mon[37807]: pgmap v24: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-03-18T14:51:42.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 -- 172.21.15.12:0/3375464018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 msgr2=0x7fc75c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 --2- 172.21.15.12:0/3375464018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c0fd3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc750004940 tx=0x7fc750009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 -- 172.21.15.12:0/3375464018 shutdown_connections 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 --2- 172.21.15.12:0/3375464018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c0fd3f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 --2- 172.21.15.12:0/3375464018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 -- 172.21.15.12:0/3375464018 >> 172.21.15.12:0/3375464018 conn(0x7fc75c0f7430 msgr2=0x7fc75c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.199+0000 7fc761395700 1 -- 172.21.15.12:0/3375464018 shutdown_connections 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.200+0000 7fc761395700 1 -- 172.21.15.12:0/3375464018 wait complete. 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.200+0000 7fc761395700 1 Processor -- start 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.200+0000 7fc761395700 1 -- start start 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc761395700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc761395700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc761395700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c067140 con 0x7fc75c0fcf80 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc761395700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c110b40 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc75affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc75affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:38734/0 (socket says 172.21.15.12:38734) 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc75affd700 1 -- 172.21.15.12:0/1411835683 learned_addr learned my addr 172.21.15.12:0/1411835683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:42.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.201+0000 7fc75a7fc700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc75affd700 1 -- 172.21.15.12:0/1411835683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 msgr2=0x7fc75c110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc75affd700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc75affd700 1 -- 172.21.15.12:0/1411835683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750005040 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc75affd700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00eb70 tx=0x7fc74c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74c007cc0 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc74c00f040 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74c00ca90 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c110dc0 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.202+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c1111f0 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.204+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c04c710 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.204+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7fc74c00cbf0 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.205+0000 7fc7437fe700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 0x7fc74406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.205+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc74c089d90 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.205+0000 7fc75a7fc700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 0x7fc74406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.205+0000 7fc75a7fc700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 0x7fc74406db20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc750009b30 tx=0x7fc750009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.210+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fc74c057510 con 0x7fc75c0fbcd0 2024-03-18T14:51:42.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:42 smithi012 ceph-mon[29182]: pgmap v25: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-03-18T14:51:42.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.522+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc75c101220 con 0x7fc74406b670 2024-03-18T14:51:42.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.523+0000 7fc7437fe700 1 -- 172.21.15.12:0/1411835683 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc75c101220 con 0x7fc74406b670 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 msgr2=0x7fc74406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 0x7fc74406db20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc750009b30 tx=0x7fc750009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 msgr2=0x7fc75c10e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00eb70 tx=0x7fc74c00b040 comp rx=0 tx=0).stop 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 shutdown_connections 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc74406b670 0x7fc74406db20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc75c0fbcd0 0x7fc75c10e0b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 --2- 172.21.15.12:0/1411835683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc75c0fcf80 0x7fc75c110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.526+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 >> 172.21.15.12:0/1411835683 conn(0x7fc75c0f7430 msgr2=0x7fc75c0f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.527+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 shutdown_connections 2024-03-18T14:51:42.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.527+0000 7fc761395700 1 -- 172.21.15.12:0/1411835683 wait complete. 2024-03-18T14:51:42.542 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:51:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 -- 172.21.15.12:0/771139540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 msgr2=0x7fd6ec0fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 --2- 172.21.15.12:0/771139540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec0fd410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0004940 tx=0x7fd6e0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 -- 172.21.15.12:0/771139540 shutdown_connections 2024-03-18T14:51:42.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 --2- 172.21.15.12:0/771139540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec0fd410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 --2- 172.21.15.12:0/771139540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec0fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 -- 172.21.15.12:0/771139540 >> 172.21.15.12:0/771139540 conn(0x7fd6ec0f7480 msgr2=0x7fd6ec0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 -- 172.21.15.12:0/771139540 shutdown_connections 2024-03-18T14:51:42.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.671+0000 7fd6f1909700 1 -- 172.21.15.12:0/771139540 wait complete. 2024-03-18T14:51:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 Processor -- start 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 -- start start 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec111610 con 0x7fd6ec0fcfa0 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.672+0000 7fd6f1909700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec111780 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:38756/0 (socket says 172.21.15.12:38756) 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 -- 172.21.15.12:0/2721658867 learned_addr learned my addr 172.21.15.12:0/2721658867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6ea7fc700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 -- 172.21.15.12:0/2721658867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 msgr2=0x7fd6ec102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 -- 172.21.15.12:0/2721658867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0005040 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6eaffd700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00eb40 tx=0x7fd6dc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc007cc0 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec102900 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6dc00f040 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.673+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec102e70 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.674+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc00c860 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.675+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec05f5f0 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.676+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7fd6dc00c9c0 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.677+0000 7fd6d37fe700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 0x7fd6d406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.677+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd6dc089b60 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.677+0000 7fd6ea7fc700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 0x7fd6d406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.678+0000 7fd6ea7fc700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 0x7fd6d406dbc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00043d0 tx=0x7fd6e0009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:42.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:42.681+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fd6dc057340 con 0x7fd6ec0fbe30 2024-03-18T14:51:42.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:42 smithi100 ceph-mon[37807]: pgmap v25: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-03-18T14:51:43.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.056+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ec101240 con 0x7fd6d406b710 2024-03-18T14:51:43.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.057+0000 7fd6d37fe700 1 -- 172.21.15.12:0/2721658867 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fd6ec101240 con 0x7fd6d406b710 2024-03-18T14:51:43.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 msgr2=0x7fd6d406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 0x7fd6d406dbc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00043d0 tx=0x7fd6e0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:43.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 msgr2=0x7fd6ec103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00eb40 tx=0x7fd6dc00b040 comp rx=0 tx=0).stop 2024-03-18T14:51:43.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 shutdown_connections 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fd6d406b710 0x7fd6d406dbc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6ec0fbe30 0x7fd6ec103e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 --2- 172.21.15.12:0/2721658867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6ec0fcfa0 0x7fd6ec102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 >> 172.21.15.12:0/2721658867 conn(0x7fd6ec0f7480 msgr2=0x7fd6ec0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 shutdown_connections 2024-03-18T14:51:43.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.060+0000 7fd6f1909700 1 -- 172.21.15.12:0/2721658867 wait complete. 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 -- 172.21.15.12:0/1099532539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 msgr2=0x7f12f40fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/1099532539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f40fd250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f12e4004880 tx=0x7f12e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 -- 172.21.15.12:0/1099532539 shutdown_connections 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/1099532539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 0x7f12f40fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/1099532539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f40fd250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 -- 172.21.15.12:0/1099532539 >> 172.21.15.12:0/1099532539 conn(0x7f12f40f8550 msgr2=0x7f12f40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 -- 172.21.15.12:0/1099532539 shutdown_connections 2024-03-18T14:51:43.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.190+0000 7f12fa5ac700 1 -- 172.21.15.12:0/1099532539 wait complete. 2024-03-18T14:51:43.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 Processor -- start 2024-03-18T14:51:43.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 -- start start 2024-03-18T14:51:43.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 0x7f12f410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410e590 con 0x7f12f40fe040 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12fa5ac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410ede0 con 0x7f12f40fce40 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12f37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 0x7f12f410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.191+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:38776/0 (socket says 172.21.15.12:38776) 2024-03-18T14:51:43.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f3fff700 1 -- 172.21.15.12:0/3949146666 learned_addr learned my addr 172.21.15.12:0/3949146666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f3fff700 1 -- 172.21.15.12:0/3949146666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 msgr2=0x7f12f410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f3fff700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 0x7f12f410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f3fff700 1 -- 172.21.15.12:0/3949146666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8005040 con 0x7f12f40fce40 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f3fff700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12e4004fe0 tx=0x7f12e4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e400e030 con 0x7f12f40fce40 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12e4013e60 con 0x7f12f40fce40 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4005040 con 0x7f12f40fce40 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e401ce40 con 0x7f12f40fce40 2024-03-18T14:51:43.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.192+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f410f430 con 0x7f12f40fce40 2024-03-18T14:51:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.194+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f405f5f0 con 0x7f12f40fce40 2024-03-18T14:51:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.194+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f12e4013990 con 0x7f12f40fce40 2024-03-18T14:51:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.195+0000 7f12f0ff9700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 0x7f12dc06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.195+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f12e408ba30 con 0x7f12f40fce40 2024-03-18T14:51:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.195+0000 7f12f37fe700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 0x7f12dc06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.195+0000 7f12f37fe700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 0x7f12dc06db10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009fd0 tx=0x7f12e800b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:43.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.199+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f12e4059370 con 0x7f12f40fce40 2024-03-18T14:51:43.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.480+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f12f410f760 con 0x7f12dc06b660 2024-03-18T14:51:43.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.489+0000 7f12f0ff9700 1 -- 172.21.15.12:0/3949146666 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f12f410f760 con 0x7f12dc06b660 2024-03-18T14:51:43.490 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:51:43.490 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 starting - - - - 2024-03-18T14:51:43.490 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (8m) 28s ago 8m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (7m) 33s ago 7m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (6m) 28s ago 8m 82.1M - 9.4.7 954c08fa6188 a14a00878205 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (2m) 28s ago 2m 133M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (3m) 28s ago 3m 211M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (3m) 33s ago 3m 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (2m) 33s ago 2m 11.6M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (10m) 28s ago 10m 335M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (65s) 33s ago 7m 488M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 148db09fb780 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (10m) 28s ago 10m 47.8M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (7m) 33s ago 7m 39.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 starting - - - - 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (7m) 33s ago 7m 17.3M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (6m) 28s ago 6m 240M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:51:43.491 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (6m) 28s ago 6m 235M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:51:43.492 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (5m) 28s ago 5m 195M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:51:43.492 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (5m) 33s ago 5m 274M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:51:43.492 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (4m) 33s ago 4m 221M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:51:43.492 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (4m) 33s ago 4m 266M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:51:43.492 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (6m) 28s ago 8m 58.3M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:51:43.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 msgr2=0x7f12dc06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 0x7f12dc06db10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009fd0 tx=0x7f12e800b040 comp rx=0 tx=0).stop 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 msgr2=0x7f12f410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12e4004fe0 tx=0x7f12e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 shutdown_connections 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f12dc06b660 0x7f12dc06db10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12f40fce40 0x7f12f410da50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 --2- 172.21.15.12:0/3949146666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12f40fe040 0x7f12f410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 >> 172.21.15.12:0/3949146666 conn(0x7f12f40f8550 msgr2=0x7f12f40f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.492+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 shutdown_connections 2024-03-18T14:51:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.493+0000 7f12fa5ac700 1 -- 172.21.15.12:0/3949146666 wait complete. 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 -- 172.21.15.12:0/698747874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fb210 msgr2=0x7fc4480fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/698747874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fb210 0x7fc4480fb620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc438004940 tx=0x7fc438009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 -- 172.21.15.12:0/698747874 shutdown_connections 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/698747874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fbb60 0x7fc4480fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/698747874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fb210 0x7fc4480fb620 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 -- 172.21.15.12:0/698747874 >> 172.21.15.12:0/698747874 conn(0x7fc4480f6c10 msgr2=0x7fc4480f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 -- 172.21.15.12:0/698747874 shutdown_connections 2024-03-18T14:51:43.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.630+0000 7fc44cd4e700 1 -- 172.21.15.12:0/698747874 wait complete. 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 Processor -- start 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 -- start start 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fb210 0x7fc44810db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44810e6b0 con 0x7fc4480fbb60 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44cd4e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44810e820 con 0x7fc4480fb210 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38426/0 (socket says 172.21.15.12:38426) 2024-03-18T14:51:43.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 -- 172.21.15.12:0/2052250989 learned_addr learned my addr 172.21.15.12:0/2052250989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc44659c700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fb210 0x7fc44810db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 -- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fb210 msgr2=0x7fc44810db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fb210 0x7fc44810db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 -- 172.21.15.12:0/2052250989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438005040 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc445d9b700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc43c002a50 tx=0x7fc43c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.631+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc43c007cc0 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.632+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc448113280 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.632+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc43c00f040 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.632+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc448113840 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.632+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc43c00d9f0 con 0x7fc4480fbb60 2024-03-18T14:51:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.632+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44804c710 con 0x7fc4480fbb60 2024-03-18T14:51:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.633+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7fc43c00db90 con 0x7fc4480fbb60 2024-03-18T14:51:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.633+0000 7fc436ffd700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 0x7fc43006db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.633+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc43c089f70 con 0x7fc4480fbb60 2024-03-18T14:51:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.634+0000 7fc44659c700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 0x7fc43006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.634+0000 7fc44659c700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 0x7fc43006db20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b3b0 tx=0x7fc438009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:43.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:43.638+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fc43c0576f0 con 0x7fc4480fbb60 2024-03-18T14:51:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:43 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:43 smithi012 ceph-mon[29182]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:43.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:43 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:43.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:43 smithi100 ceph-mon[37807]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:44.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.082+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc448066090 con 0x7fc4480fbb60 2024-03-18T14:51:44.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.083+0000 7fc436ffd700 1 -- 172.21.15.12:0/2052250989 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fc43c057280 con 0x7fc4480fbb60 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 1, 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:51:44.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 13, 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:51:44.087 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 msgr2=0x7fc43006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 0x7fc43006db20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b3b0 tx=0x7fc438009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 msgr2=0x7fc44810e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc43c002a50 tx=0x7fc43c0046b0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 shutdown_connections 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fc43006b670 0x7fc43006db20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4480fb210 0x7fc44810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 --2- 172.21.15.12:0/2052250989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4480fbb60 0x7fc44810e0b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 >> 172.21.15.12:0/2052250989 conn(0x7fc4480f6c10 msgr2=0x7fc4480f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 shutdown_connections 2024-03-18T14:51:44.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.086+0000 7fc44cd4e700 1 -- 172.21.15.12:0/2052250989 wait complete. 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 -- 172.21.15.12:0/838959793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fc130 msgr2=0x7f11c00fc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 --2- 172.21.15.12:0/838959793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fc130 0x7f11c00fc540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f11b0004940 tx=0x7f11b0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 -- 172.21.15.12:0/838959793 shutdown_connections 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 --2- 172.21.15.12:0/838959793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fca80 0x7f11c00fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 --2- 172.21.15.12:0/838959793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fc130 0x7f11c00fc540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 -- 172.21.15.12:0/838959793 >> 172.21.15.12:0/838959793 conn(0x7f11c00f6ba0 msgr2=0x7f11c00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 -- 172.21.15.12:0/838959793 shutdown_connections 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.211+0000 7f11c4862700 1 -- 172.21.15.12:0/838959793 wait complete. 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 Processor -- start 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 -- start start 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 0x7f11c0113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:44.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0114af0 con 0x7f11c00fca80 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11c4862700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0115340 con 0x7f11c00fc130 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.212+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38432/0 (socket says 172.21.15.12:38432) 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 -- 172.21.15.12:0/905358081 learned_addr learned my addr 172.21.15.12:0/905358081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11be59c700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 0x7f11c0113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 -- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 msgr2=0x7f11c0113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 0x7f11c0113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 -- 172.21.15.12:0/905358081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4005040 con 0x7f11c00fca80 2024-03-18T14:51:44.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11be59c700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 0x7f11c0113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.213+0000 7f11bdd9b700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11b40096d0 tx=0x7f11b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.214+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b400e840 con 0x7f11c00fca80 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.214+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11b400f040 con 0x7f11c00fca80 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.214+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b4013dc0 con 0x7f11c00fca80 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.214+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b0005040 con 0x7f11c00fca80 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.214+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c01158b0 con 0x7f11c00fca80 2024-03-18T14:51:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.215+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c005f5f0 con 0x7f11c00fca80 2024-03-18T14:51:44.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.216+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f11b400e9a0 con 0x7f11c00fca80 2024-03-18T14:51:44.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.217+0000 7f11aeffd700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 0x7f11a806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:51:44.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.217+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f11b4089f80 con 0x7f11c00fca80 2024-03-18T14:51:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.217+0000 7f11be59c700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 0x7f11a806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:51:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.218+0000 7f11be59c700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 0x7f11a806dbe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009b30 tx=0x7f11b0009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:51:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.223+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f11b40576e0 con 0x7f11c00fca80 2024-03-18T14:51:44.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.593+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c0115c90 con 0x7f11a806b730 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.594+0000 7f11aeffd700 1 -- 172.21.15.12:0/905358081 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f11c0115c90 con 0x7f11a806b730 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "1/2 daemons upgraded", 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "", 2024-03-18T14:51:44.595 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:51:44.596 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:51:44.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 msgr2=0x7f11a806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 0x7f11a806dbe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009b30 tx=0x7f11b0009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 msgr2=0x7f11c01144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11b40096d0 tx=0x7f11b4009e30 comp rx=0 tx=0).stop 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 shutdown_connections 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f11a806b730 0x7f11a806dbe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.596+0000 7f11c4862700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11c00fc130 0x7f11c0113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.597+0000 7f11c4862700 1 --2- 172.21.15.12:0/905358081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11c00fca80 0x7f11c01144f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.597+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 >> 172.21.15.12:0/905358081 conn(0x7f11c00f6ba0 msgr2=0x7f11c00f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.597+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 shutdown_connections 2024-03-18T14:51:44.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:51:44.597+0000 7f11c4862700 1 -- 172.21.15.12:0/905358081 wait complete. 2024-03-18T14:51:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:44 smithi012 ceph-mon[29182]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:44 smithi012 ceph-mon[29182]: pgmap v26: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-03-18T14:51:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:44 smithi012 ceph-mon[29182]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:44 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2052250989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:51:44.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:44 smithi100 ceph-mon[37807]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:44.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:44 smithi100 ceph-mon[37807]: pgmap v26: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-03-18T14:51:44.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:44 smithi100 ceph-mon[37807]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:44.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:44 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2052250989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:51:45.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:45 smithi012 ceph-mon[29182]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:45.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:45 smithi100 ceph-mon[37807]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:51:46.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:46 smithi012 ceph-mon[29182]: pgmap v27: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-03-18T14:51:46.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:46 smithi100 ceph-mon[37807]: pgmap v27: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-03-18T14:51:48.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:48 smithi012 ceph-mon[29182]: pgmap v28: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-03-18T14:51:48.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:48 smithi100 ceph-mon[37807]: pgmap v28: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-03-18T14:51:50.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:50 smithi012 ceph-mon[29182]: pgmap v29: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 310 op/s 2024-03-18T14:51:50.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:50 smithi100 ceph-mon[37807]: pgmap v29: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 310 op/s 2024-03-18T14:51:52.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:52 smithi012 ceph-mon[29182]: pgmap v30: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T14:51:52.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:52 smithi100 ceph-mon[37807]: pgmap v30: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T14:51:54.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:53 smithi012 ceph-mon[29182]: pgmap v31: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T14:51:54.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:53 smithi100 ceph-mon[37807]: pgmap v31: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T14:51:56.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:56 smithi012 ceph-mon[29182]: pgmap v32: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 270 op/s 2024-03-18T14:51:56.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:56 smithi100 ceph-mon[37807]: pgmap v32: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 270 op/s 2024-03-18T14:51:58.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:58 smithi012 ceph-mon[29182]: pgmap v33: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-03-18T14:51:58.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:51:58 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:51:58.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:58 smithi100 ceph-mon[37807]: pgmap v33: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-03-18T14:51:58.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:51:58 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:00.699 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:00 smithi012 ceph-mon[29182]: pgmap v34: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-03-18T14:52:00.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:00 smithi100 ceph-mon[37807]: pgmap v34: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-03-18T14:52:02.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:02 smithi012 ceph-mon[29182]: pgmap v35: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-18T14:52:02.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:02 smithi100 ceph-mon[37807]: pgmap v35: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-18T14:52:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:03 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:03 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:03 smithi012 ceph-mon[29182]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:52:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:03 smithi012 ceph-mon[29182]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:52:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:03 smithi012 ceph-mon[29182]: pgmap v36: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-18T14:52:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:03 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:03 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:03 smithi100 ceph-mon[37807]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:52:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:03 smithi100 ceph-mon[37807]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:52:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:03 smithi100 ceph-mon[37807]: pgmap v36: 65 pgs: 65 active+clean; 311 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-03-18T14:52:06.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:06 smithi012 ceph-mon[29182]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-03-18T14:52:06.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:06 smithi100 ceph-mon[37807]: pgmap v37: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-03-18T14:52:08.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:08 smithi012 ceph-mon[29182]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-03-18T14:52:08.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:08 smithi100 ceph-mon[37807]: pgmap v38: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-03-18T14:52:10.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:10 smithi012 ceph-mon[29182]: pgmap v39: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 398 op/s 2024-03-18T14:52:10.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:10 smithi100 ceph-mon[37807]: pgmap v39: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 398 op/s 2024-03-18T14:52:12.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:12 smithi012 ceph-mon[29182]: pgmap v40: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 288 op/s 2024-03-18T14:52:12.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:12 smithi100 ceph-mon[37807]: pgmap v40: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 288 op/s 2024-03-18T14:52:13.703 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:13 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:13.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:13 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:14.718 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:14 smithi012 ceph-mon[29182]: pgmap v41: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 288 op/s 2024-03-18T14:52:14.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.717+0000 7f704a1b0700 1 -- 172.21.15.12:0/3330054027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 msgr2=0x7f70440fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:14.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.717+0000 7f704a1b0700 1 --2- 172.21.15.12:0/3330054027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f70440fd3f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7038004940 tx=0x7f7038009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:14.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 -- 172.21.15.12:0/3330054027 shutdown_connections 2024-03-18T14:52:14.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 --2- 172.21.15.12:0/3330054027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f70440fd3f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:14.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 --2- 172.21.15.12:0/3330054027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f70440fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 -- 172.21.15.12:0/3330054027 >> 172.21.15.12:0/3330054027 conn(0x7f70440f7430 msgr2=0x7f70440f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 -- 172.21.15.12:0/3330054027 shutdown_connections 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 -- 172.21.15.12:0/3330054027 wait complete. 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 Processor -- start 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 -- start start 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.718+0000 7f704a1b0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f7044110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f704a1b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f7044112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f704a1b0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044066370 con 0x7f70440fcf80 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f704a1b0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440664b0 con 0x7f70440fbcd0 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f7042ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f7044112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f70437fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f7044110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f7042ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f7044112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37630/0 (socket says 172.21.15.12:37630) 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f7042ffd700 1 -- 172.21.15.12:0/1570641442 learned_addr learned my addr 172.21.15.12:0/1570641442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:14.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f70437fe700 1 -- 172.21.15.12:0/1570641442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 msgr2=0x7f7044112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f70437fe700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f7044112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f70437fe700 1 -- 172.21.15.12:0/1570641442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038005040 con 0x7f70440fbcd0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f70437fe700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f7044110700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7034002820 tx=0x7f7034004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7034007cc0 con 0x7f70440fbcd0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.719+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7044113250 con 0x7f70440fbcd0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.720+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70441137e0 con 0x7f70440fbcd0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.720+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f703400f040 con 0x7f70440fbcd0 2024-03-18T14:52:14.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.720+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7034013920 con 0x7f70440fbcd0 2024-03-18T14:52:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.721+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704405f5f0 con 0x7f70440fbcd0 2024-03-18T14:52:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.721+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f703401c400 con 0x7f70440fbcd0 2024-03-18T14:52:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.722+0000 7f702bfff700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 0x7f702c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.722+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7034089ff0 con 0x7f70440fbcd0 2024-03-18T14:52:14.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.722+0000 7f7042ffd700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 0x7f702c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:14.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.723+0000 7f7042ffd700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 0x7f702c06db10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7038004f90 tx=0x7f7038009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:14.727+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f7034053ba0 con 0x7f70440fbcd0 2024-03-18T14:52:14.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:14 smithi100 ceph-mon[37807]: pgmap v41: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 288 op/s 2024-03-18T14:52:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.016+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7044101360 con 0x7f702c06b660 2024-03-18T14:52:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.018+0000 7f702bfff700 1 -- 172.21.15.12:0/1570641442 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7044101360 con 0x7f702c06b660 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 msgr2=0x7f702c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 0x7f702c06db10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7038004f90 tx=0x7f7038009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 msgr2=0x7f7044110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f7044110700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7034002820 tx=0x7f7034004570 comp rx=0 tx=0).stop 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 shutdown_connections 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f702c06b660 0x7f702c06db10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70440fbcd0 0x7f7044110700 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 --2- 172.21.15.12:0/1570641442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70440fcf80 0x7f7044112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 >> 172.21.15.12:0/1570641442 conn(0x7f70440f7430 msgr2=0x7f70440f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 shutdown_connections 2024-03-18T14:52:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.021+0000 7f704a1b0700 1 -- 172.21.15.12:0/1570641442 wait complete. 2024-03-18T14:52:15.032 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 -- 172.21.15.12:0/2857815765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fcee0 msgr2=0x7f58480fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 --2- 172.21.15.12:0/2857815765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fcee0 0x7f58480fd310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5844004940 tx=0x7f5844009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 -- 172.21.15.12:0/2857815765 shutdown_connections 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 --2- 172.21.15.12:0/2857815765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fe0d0 0x7f58480fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 --2- 172.21.15.12:0/2857815765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fcee0 0x7f58480fd310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 -- 172.21.15.12:0/2857815765 >> 172.21.15.12:0/2857815765 conn(0x7f58480f8560 msgr2=0x7f58480fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 -- 172.21.15.12:0/2857815765 shutdown_connections 2024-03-18T14:52:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.135+0000 7f584fb17700 1 -- 172.21.15.12:0/2857815765 wait complete. 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 Processor -- start 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 -- start start 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fcee0 0x7f5848071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848111660 con 0x7f58480fe0d0 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584fb17700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481117d0 con 0x7f58480fcee0 2024-03-18T14:52:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584d0b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584d0b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37642/0 (socket says 172.21.15.12:37642) 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584d0b2700 1 -- 172.21.15.12:0/1808240911 learned_addr learned my addr 172.21.15.12:0/1808240911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.136+0000 7f584d8b3700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fcee0 0x7f5848071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584d0b2700 1 -- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fcee0 msgr2=0x7f5848071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584d0b2700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fcee0 0x7f5848071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584d0b2700 1 -- 172.21.15.12:0/1808240911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838005040 con 0x7f58480fe0d0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584d0b2700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f58380096d0 tx=0x7f5838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583800e840 con 0x7f58480fe0d0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5844005040 con 0x7f58480fe0d0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848070cf0 con 0x7f58480fe0d0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f583800f040 con 0x7f58480fe0d0 2024-03-18T14:52:15.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.137+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5838013dc0 con 0x7f58480fe0d0 2024-03-18T14:52:15.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.138+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584805f5f0 con 0x7f58480fe0d0 2024-03-18T14:52:15.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.139+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f583800e9a0 con 0x7f58480fe0d0 2024-03-18T14:52:15.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.140+0000 7f583e7fc700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 0x7f583406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.140+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f5838089ec0 con 0x7f58480fe0d0 2024-03-18T14:52:15.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.140+0000 7f584d8b3700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 0x7f583406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.140+0000 7f584d8b3700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 0x7f583406db10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5848071bd0 tx=0x7f5844009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.149+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f5838053b90 con 0x7f58480fe0d0 2024-03-18T14:52:15.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.537+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58480710e0 con 0x7f583406b660 2024-03-18T14:52:15.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.538+0000 7f583e7fc700 1 -- 172.21.15.12:0/1808240911 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f58480710e0 con 0x7f583406b660 2024-03-18T14:52:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 msgr2=0x7f583406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 0x7f583406db10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5848071bd0 tx=0x7f5844009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 msgr2=0x7f5848070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f58380096d0 tx=0x7f5838009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 shutdown_connections 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f583406b660 0x7f583406db10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58480fcee0 0x7f5848071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 --2- 172.21.15.12:0/1808240911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58480fe0d0 0x7f5848070310 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 >> 172.21.15.12:0/1808240911 conn(0x7f58480f8560 msgr2=0x7f58480f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 shutdown_connections 2024-03-18T14:52:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.540+0000 7f584fb17700 1 -- 172.21.15.12:0/1808240911 wait complete. 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 -- 172.21.15.12:0/3412243348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fcf80 msgr2=0x7f46340fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 --2- 172.21.15.12:0/3412243348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fcf80 0x7f46340fd3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4628004940 tx=0x7f4628009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 -- 172.21.15.12:0/3412243348 shutdown_connections 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 --2- 172.21.15.12:0/3412243348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fcf80 0x7f46340fd3f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 --2- 172.21.15.12:0/3412243348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fbcd0 0x7f46340fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 -- 172.21.15.12:0/3412243348 >> 172.21.15.12:0/3412243348 conn(0x7f46340f7430 msgr2=0x7f46340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 -- 172.21.15.12:0/3412243348 shutdown_connections 2024-03-18T14:52:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.688+0000 7f463a550700 1 -- 172.21.15.12:0/3412243348 wait complete. 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 Processor -- start 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 -- start start 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 0x7f463410fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 0x7f46341100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341106f0 con 0x7f46340fbcd0 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f463a550700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4634110f40 con 0x7f46340fcf80 2024-03-18T14:52:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f4633fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 0x7f463410fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f4633fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 0x7f463410fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37658/0 (socket says 172.21.15.12:37658) 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f4633fff700 1 -- 172.21.15.12:0/1493376748 learned_addr learned my addr 172.21.15.12:0/1493376748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.689+0000 7f46337fe700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 0x7f46341100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f46337fe700 1 -- 172.21.15.12:0/1493376748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 msgr2=0x7f463410fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f46337fe700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 0x7f463410fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f46337fe700 1 -- 172.21.15.12:0/1493376748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624005040 con 0x7f46340fcf80 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f46337fe700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 0x7f46341100f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462800e070 con 0x7f46340fcf80 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.690+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4628005040 con 0x7f46340fcf80 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.691+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4634111510 con 0x7f46340fcf80 2024-03-18T14:52:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.691+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f462801c400 con 0x7f46340fcf80 2024-03-18T14:52:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.691+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462800b670 con 0x7f46340fcf80 2024-03-18T14:52:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.692+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46341029d0 con 0x7f46340fcf80 2024-03-18T14:52:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.692+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f4628013af0 con 0x7f46340fcf80 2024-03-18T14:52:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.693+0000 7f4630ff9700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 0x7f461c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.693+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f462808b540 con 0x7f46340fcf80 2024-03-18T14:52:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.693+0000 7f4633fff700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 0x7f461c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.694+0000 7f4633fff700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 0x7f461c06db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46240044e0 tx=0x7f462400b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:15.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.697+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f4628058e40 con 0x7f46340fcf80 2024-03-18T14:52:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:15.993+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4634105a40 con 0x7f461c06b660 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.002+0000 7f4630ff9700 1 -- 172.21.15.12:0/1493376748 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f4634105a40 con 0x7f461c06b660 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 starting - - - - 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (9m) 60s ago 9m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (8m) 65s ago 8m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 starting - - - - 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (3m) 60s ago 3m 133M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (3m) 60s ago 3m 211M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:52:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (3m) 65s ago 3m 14.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (3m) 65s ago 3m 11.6M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:9283 running (10m) 60s ago 10m 335M - 17.2.7-684-g6716c9f3 19857db3290c 9ea17c10c6c5 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (97s) 65s ago 8m 488M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 148db09fb780 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (10m) 60s ago 10m 47.8M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (7m) 65s ago 7m 39.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 starting - - - - 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (8m) 65s ago 8m 17.3M - 1.5.0 0da6a335fe13 5edf31d3c567 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (7m) 60s ago 7m 240M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (6m) 60s ago 6m 235M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (6m) 60s ago 6m 195M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (5m) 65s ago 5m 274M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (5m) 65s ago 5m 221M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (5m) 65s ago 5m 266M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:52:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (7m) 60s ago 8m 58.3M - 2.43.0 a07b618ecd1d f4a40863e789 2024-03-18T14:52:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 msgr2=0x7f461c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 0x7f461c06db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46240044e0 tx=0x7f462400b040 comp rx=0 tx=0).stop 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 msgr2=0x7f46341100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 0x7f46341100f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 shutdown_connections 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f461c06b660 0x7f461c06db10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46340fbcd0 0x7f463410fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 --2- 172.21.15.12:0/1493376748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46340fcf80 0x7f46341100f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 >> 172.21.15.12:0/1493376748 conn(0x7f46340f7430 msgr2=0x7f46340f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 shutdown_connections 2024-03-18T14:52:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.006+0000 7f463a550700 1 -- 172.21.15.12:0/1493376748 wait complete. 2024-03-18T14:52:16.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 -- 172.21.15.12:0/2370441052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fc6f0 msgr2=0x7f83e40fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 --2- 172.21.15.12:0/2370441052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fc6f0 0x7f83e40fcb60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83d0004a50 tx=0x7f83d000af40 comp rx=0 tx=0).stop 2024-03-18T14:52:16.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 -- 172.21.15.12:0/2370441052 shutdown_connections 2024-03-18T14:52:16.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 --2- 172.21.15.12:0/2370441052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fc6f0 0x7f83e40fcb60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 --2- 172.21.15.12:0/2370441052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fb440 0x7f83e40fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 -- 172.21.15.12:0/2370441052 >> 172.21.15.12:0/2370441052 conn(0x7f83e40f6ba0 msgr2=0x7f83e40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 -- 172.21.15.12:0/2370441052 shutdown_connections 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.128+0000 7f83eb147700 1 -- 172.21.15.12:0/2370441052 wait complete. 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 Processor -- start 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 -- start start 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fc6f0 0x7f83e410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e410e4f0 con 0x7f83e40fb440 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83eb147700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e410ed40 con 0x7f83e40fc6f0 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.129+0000 7f83e8ee3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37686/0 (socket says 172.21.15.12:37686) 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 -- 172.21.15.12:0/3856775265 learned_addr learned my addr 172.21.15.12:0/3856775265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:16.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e3fff700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fc6f0 0x7f83e410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 -- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fc6f0 msgr2=0x7f83e410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fc6f0 0x7f83e410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 -- 172.21.15.12:0/3856775265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0005040 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e8ee3700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f83d8002a80 tx=0x7f83d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d8007cc0 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e410f020 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e410f560 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83d800f040 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.130+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d800da80 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.132+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c80052e0 con 0x7f83e40fb440 2024-03-18T14:52:16.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.132+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7f83d80097a0 con 0x7f83e40fb440 2024-03-18T14:52:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.132+0000 7f83e17fa700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 0x7f83d406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.133+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f83d8089e10 con 0x7f83e40fb440 2024-03-18T14:52:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.133+0000 7f83e3fff700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 0x7f83d406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.133+0000 7f83e3fff700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 0x7f83d406db10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c3b0 tx=0x7f83d000af40 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.137+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f83d8057590 con 0x7f83e40fb440 2024-03-18T14:52:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.582+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83c8005860 con 0x7f83e40fb440 2024-03-18T14:52:16.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.584+0000 7f83e17fa700 1 -- 172.21.15.12:0/3856775265 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f83d8057120 con 0x7f83e40fb440 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 1, 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:16.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 13, 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:52:16.586 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 msgr2=0x7f83d406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 0x7f83d406db10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c3b0 tx=0x7f83d000af40 comp rx=0 tx=0).stop 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 msgr2=0x7f83e410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f83d8002a80 tx=0x7f83d80046b0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 shutdown_connections 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7f83d406b660 0x7f83d406db10 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c3b0 tx=0x7f83d000af40 comp rx=0 tx=0).stop 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e40fb440 0x7f83e410d9b0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f83d8002a80 tx=0x7f83d80046b0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 --2- 172.21.15.12:0/3856775265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e40fc6f0 0x7f83e410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 >> 172.21.15.12:0/3856775265 conn(0x7f83e40f6ba0 msgr2=0x7f83e40f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 shutdown_connections 2024-03-18T14:52:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.589+0000 7f83eb147700 1 -- 172.21.15.12:0/3856775265 wait complete. 2024-03-18T14:52:16.684 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:16 smithi012 ceph-mon[29182]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:16.684 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:16 smithi012 ceph-mon[29182]: pgmap v42: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 874 op/s 2024-03-18T14:52:16.684 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:16 smithi012 ceph-mon[29182]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2072117919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 msgr2=0x7fbd980fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2072117919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd980fe810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbd94004940 tx=0x7fbd94009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2072117919 shutdown_connections 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2072117919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd980fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2072117919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd980fe810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2072117919 >> 172.21.15.12:0/2072117919 conn(0x7fbd980f6ba0 msgr2=0x7fbd980f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2072117919 shutdown_connections 2024-03-18T14:52:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.716+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2072117919 wait complete. 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 Processor -- start 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 -- start start 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd9810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd9810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9810e4c0 con 0x7fbd980fe400 2024-03-18T14:52:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9fde8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9810ed10 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9db84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd9810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9db84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd9810d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37698/0 (socket says 172.21.15.12:37698) 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9db84700 1 -- 172.21.15.12:0/2745482317 learned_addr learned my addr 172.21.15.12:0/2745482317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.717+0000 7fbd9d383700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd9810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9d383700 1 -- 172.21.15.12:0/2745482317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 msgr2=0x7fbd9810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9d383700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd9810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9d383700 1 -- 172.21.15.12:0/2745482317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94005040 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9d383700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd9810dec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009fd0 tx=0x7fbd8800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd88004e90 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd8800f040 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd8800ca50 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9810ef90 con 0x7fbd980fed50 2024-03-18T14:52:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.718+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9810f480 con 0x7fbd980fed50 2024-03-18T14:52:16.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.719+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9805f5f0 con 0x7fbd980fed50 2024-03-18T14:52:16.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.720+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 89192+0+0 (secure 0 0 0) 0x7fbd88007cc0 con 0x7fbd980fed50 2024-03-18T14:52:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.721+0000 7fbd8e7fc700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 0x7fbd8406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.721+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fbd88089c20 con 0x7fbd980fed50 2024-03-18T14:52:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.721+0000 7fbd9db84700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 0x7fbd8406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.721+0000 7fbd9db84700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 0x7fbd8406db10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009b30 tx=0x7fbd94009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:16.725+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fbd88050040 con 0x7fbd980fed50 2024-03-18T14:52:16.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:16 smithi100 ceph-mon[37807]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:16.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:16 smithi100 ceph-mon[37807]: pgmap v42: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 874 op/s 2024-03-18T14:52:16.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:16 smithi100 ceph-mon[37807]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:17.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.086+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 --> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd9810f6f0 con 0x7fbd8406b660 2024-03-18T14:52:17.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.088+0000 7fbd8e7fc700 1 -- 172.21.15.12:0/2745482317 <== mgr.24359 v2:172.21.15.100:6828/3399432197 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fbd9810f6f0 con 0x7fbd8406b660 2024-03-18T14:52:17.088 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "1/2 daemons upgraded", 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "", 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:52:17.089 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:52:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 msgr2=0x7fbd8406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 0x7fbd8406db10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009b30 tx=0x7fbd94009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 msgr2=0x7fbd9810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd9810dec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009fd0 tx=0x7fbd8800b040 comp rx=0 tx=0).stop 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 shutdown_connections 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:6828/3399432197,v1:172.21.15.100:6829/3399432197] conn(0x7fbd8406b660 0x7fbd8406db10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd980fe400 0x7fbd9810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 --2- 172.21.15.12:0/2745482317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd980fed50 0x7fbd9810dec0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 >> 172.21.15.12:0/2745482317 conn(0x7fbd980f6ba0 msgr2=0x7fbd980f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 shutdown_connections 2024-03-18T14:52:17.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:17.091+0000 7fbd9fde8700 1 -- 172.21.15.12:0/2745482317 wait complete. 2024-03-18T14:52:17.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:17 smithi012 ceph-mon[29182]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:17.691 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:17 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3856775265' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:52:17.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:17 smithi100 ceph-mon[37807]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:17.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:17 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3856775265' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:52:18.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:18 smithi012 ceph-mon[29182]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:18.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:18 smithi012 ceph-mon[29182]: pgmap v43: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 721 op/s 2024-03-18T14:52:18.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:18 smithi100 ceph-mon[37807]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:18.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:18 smithi100 ceph-mon[37807]: pgmap v43: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 721 op/s 2024-03-18T14:52:20.682 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:20 smithi100 ceph-mon[37807]: pgmap v44: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 879 op/s 2024-03-18T14:52:20.683 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:20.683 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:20 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:20.683 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:20 smithi100 ceph-mon[37807]: Reconfiguring node-exporter.smithi100 (dependencies changed)... 2024-03-18T14:52:20.683 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:20 smithi100 ceph-mon[37807]: Reconfiguring daemon node-exporter.smithi100 on smithi100 2024-03-18T14:52:20.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:20 smithi012 ceph-mon[29182]: pgmap v44: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 879 op/s 2024-03-18T14:52:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:20 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:20 smithi012 ceph-mon[29182]: Reconfiguring node-exporter.smithi100 (dependencies changed)... 2024-03-18T14:52:20.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:20 smithi012 ceph-mon[29182]: Reconfiguring daemon node-exporter.smithi100 on smithi100 2024-03-18T14:52:22.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: pgmap v45: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 743 op/s 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:52:22.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:22 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: pgmap v45: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 743 op/s 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:52:22.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:52:22.949 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:22 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:23.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:23 smithi100 ceph-mon[37807]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:52:23.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:23 smithi100 ceph-mon[37807]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:52:23.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:23 smithi100 ceph-mon[37807]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:52:23.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:23 smithi100 ceph-mon[37807]: Upgrade: Need to upgrade myself (mgr.smithi100.rwnrcn) 2024-03-18T14:52:23.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:23 smithi100 ceph-mon[37807]: Upgrade: Need to upgrade myself (mgr.smithi100.rwnrcn) 2024-03-18T14:52:23.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:23 smithi012 ceph-mon[29182]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:52:23.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:23 smithi012 ceph-mon[29182]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:52:23.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:23 smithi012 ceph-mon[29182]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:52:23.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:23 smithi012 ceph-mon[29182]: Upgrade: Need to upgrade myself (mgr.smithi100.rwnrcn) 2024-03-18T14:52:23.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:23 smithi012 ceph-mon[29182]: Upgrade: Need to upgrade myself (mgr.smithi100.rwnrcn) 2024-03-18T14:52:24.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 743 op/s 2024-03-18T14:52:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:52:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:52:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:52:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:24 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 743 op/s 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:52:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:24 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:52:25.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:25 smithi100 ceph-mon[37807]: Upgrade: Updating mgr.smithi012.tmscgt 2024-03-18T14:52:25.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:25 smithi100 ceph-mon[37807]: Deploying daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:52:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:25 smithi012 ceph-mon[29182]: Upgrade: Updating mgr.smithi012.tmscgt 2024-03-18T14:52:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:25 smithi012 ceph-mon[29182]: Deploying daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:52:26.646 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:26 smithi012 ceph-mon[29182]: pgmap v47: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.47k op/s 2024-03-18T14:52:26.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:26 smithi100 ceph-mon[37807]: pgmap v47: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.47k op/s 2024-03-18T14:52:27.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:27 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:27.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:27 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:28.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:28 smithi100 ceph-mon[37807]: pgmap v48: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 884 op/s 2024-03-18T14:52:28.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:28 smithi012 ceph-mon[29182]: pgmap v48: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 884 op/s 2024-03-18T14:52:30.808 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:30 smithi012 ceph-mon[29182]: pgmap v49: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 1.25k op/s 2024-03-18T14:52:30.808 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:30 smithi012 ceph-mon[29182]: mgrmap e26: smithi100.rwnrcn(active, since 92s), standbys: smithi012.tmscgt 2024-03-18T14:52:31.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:30 smithi100 ceph-mon[37807]: pgmap v49: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 1.25k op/s 2024-03-18T14:52:31.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:30 smithi100 ceph-mon[37807]: mgrmap e26: smithi100.rwnrcn(active, since 92s), standbys: smithi012.tmscgt 2024-03-18T14:52:32.712 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:32 smithi012 ceph-mon[29182]: pgmap v50: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 1.09k op/s 2024-03-18T14:52:33.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:32 smithi100 ceph-mon[37807]: pgmap v50: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 1.09k op/s 2024-03-18T14:52:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: pgmap v51: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 1.09k op/s 2024-03-18T14:52:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:34 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: pgmap v51: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 1.09k op/s 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:34.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:34 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:36.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:36 smithi012 ceph-mon[29182]: pgmap v52: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.23k op/s 2024-03-18T14:52:36.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:36 smithi100 ceph-mon[37807]: pgmap v52: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.23k op/s 2024-03-18T14:52:37.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:37 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:37.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:37 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:37.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:37 smithi012 ceph-mon[29182]: pgmap v53: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 500 op/s 2024-03-18T14:52:38.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:37 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:38.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:37 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:38.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:37 smithi100 ceph-mon[37807]: pgmap v53: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 500 op/s 2024-03-18T14:52:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:40 smithi012 ceph-mon[29182]: pgmap v54: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 799 op/s 2024-03-18T14:52:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:40 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:40 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:40.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:40 smithi100 ceph-mon[37807]: pgmap v54: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 799 op/s 2024-03-18T14:52:40.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:40 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:40.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:40 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:42.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:42 smithi012 ceph-mon[29182]: pgmap v55: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-03-18T14:52:42.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:42 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:42.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:42 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:42.945 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:42 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:43.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:42 smithi100 ceph-mon[37807]: pgmap v55: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-03-18T14:52:43.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:42 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:43.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:42 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:43.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:42 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:43.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: Standby manager daemon smithi012.tmscgt restarted 2024-03-18T14:52:43.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: Standby manager daemon smithi012.tmscgt started 2024-03-18T14:52:43.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/crt"}]: dispatch 2024-03-18T14:52:43.872 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:52:43.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/key"}]: dispatch 2024-03-18T14:52:43.873 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:43 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: Standby manager daemon smithi012.tmscgt restarted 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: Standby manager daemon smithi012.tmscgt started 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/crt"}]: dispatch 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi012.tmscgt/key"}]: dispatch 2024-03-18T14:52:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:43 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:52:44.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:44 smithi012 ceph-mon[29182]: pgmap v56: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-03-18T14:52:44.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:44 smithi012 ceph-mon[29182]: mgrmap e27: smithi100.rwnrcn(active, since 106s), standbys: smithi012.tmscgt 2024-03-18T14:52:45.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:44 smithi100 ceph-mon[37807]: pgmap v56: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-03-18T14:52:45.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:44 smithi100 ceph-mon[37807]: mgrmap e27: smithi100.rwnrcn(active, since 106s), standbys: smithi012.tmscgt 2024-03-18T14:52:45.924 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:45 smithi012 ceph-mon[29182]: pgmap v57: 65 pgs: 65 active+clean; 249 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 640 op/s 2024-03-18T14:52:46.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:45 smithi100 ceph-mon[37807]: pgmap v57: 65 pgs: 65 active+clean; 249 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 640 op/s 2024-03-18T14:52:47.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 -- 172.21.15.12:0/119431220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 msgr2=0x7ff124102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 --2- 172.21.15.12:0/119431220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff124102320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff118004940 tx=0x7ff118009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 -- 172.21.15.12:0/119431220 shutdown_connections 2024-03-18T14:52:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 --2- 172.21.15.12:0/119431220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff124102320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 --2- 172.21.15.12:0/119431220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 0x7ff1240ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.258+0000 7ff128d51700 1 -- 172.21.15.12:0/119431220 >> 172.21.15.12:0/119431220 conn(0x7ff1240f7430 msgr2=0x7ff1240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.259+0000 7ff128d51700 1 -- 172.21.15.12:0/119431220 shutdown_connections 2024-03-18T14:52:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.259+0000 7ff128d51700 1 -- 172.21.15.12:0/119431220 wait complete. 2024-03-18T14:52:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 Processor -- start 2024-03-18T14:52:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 -- start start 2024-03-18T14:52:47.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 0x7ff1240fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240fbe70 con 0x7ff1240fff00 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff128d51700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240fbfe0 con 0x7ff1240662e0 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff121d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff121d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59110/0 (socket says 172.21.15.12:59110) 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff121d9b700 1 -- 172.21.15.12:0/1385935939 learned_addr learned my addr 172.21.15.12:0/1385935939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.260+0000 7ff12259c700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 0x7ff1240fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff121d9b700 1 -- 172.21.15.12:0/1385935939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 msgr2=0x7ff1240fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff121d9b700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 0x7ff1240fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff121d9b700 1 -- 172.21.15.12:0/1385935939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118005040 con 0x7ff1240fff00 2024-03-18T14:52:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff121d9b700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff118009150 tx=0x7ff118009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11800e040 con 0x7ff1240fff00 2024-03-18T14:52:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1240fc260 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff118004290 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11801c630 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.261+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff124115a90 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.263+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 28) v1 ==== 45629+0+0 (secure 0 0 0) 0x7ff118013420 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.263+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12405f5f0 con 0x7ff1240fff00 2024-03-18T14:52:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.263+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff11804cdb0 con 0x7ff1240fff00 2024-03-18T14:52:47.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.272+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff118011550 con 0x7ff1240fff00 2024-03-18T14:52:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.438+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 29) v1 ==== 46050+0+0 (secure 0 0 0) 0x7ff118026440 con 0x7ff1240fff00 2024-03-18T14:52:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.438+0000 7ff112ffd700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 0x7ff10c040200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.447+0000 7ff12259c700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 0x7ff10c040200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.449+0000 7ff12259c700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 0x7ff10c040200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1240fd180 tx=0x7ff11400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:52:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: osdmap e47: 6 total, 6 up, 6 in 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd='[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]': finished 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: mgrmap e28: smithi012.tmscgt(active, starting, since 0.00698007s) 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:52:47.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:47.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:52:47.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:52:47.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.654+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1240f8f70 con 0x7ff10c03de20 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 172.21.15.100:0/797324674' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: osdmap e47: 6 total, 6 up, 6 in 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.24359 ' entity='mgr.smithi100.rwnrcn' cmd='[{"prefix": "mgr fail", "who": "smithi100.rwnrcn"}]': finished 2024-03-18T14:52:47.656 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: mgrmap e28: smithi012.tmscgt(active, starting, since 0.00698007s) 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:52:47.657 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:52:47.658 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:52:47.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.656+0000 7ff112ffd700 1 -- 172.21.15.12:0/1385935939 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff1240f8f70 con 0x7ff10c03de20 2024-03-18T14:52:47.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 msgr2=0x7ff10c040200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 0x7ff10c040200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1240fd180 tx=0x7ff11400b040 comp rx=0 tx=0).stop 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 msgr2=0x7ff1240fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff118009150 tx=0x7ff118009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 shutdown_connections 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff10c03de20 0x7ff10c040200 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.659+0000 7ff128d51700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1240662e0 0x7ff1240fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.660+0000 7ff128d51700 1 --2- 172.21.15.12:0/1385935939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1240fff00 0x7ff1240fb8c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.660+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 >> 172.21.15.12:0/1385935939 conn(0x7ff1240f7430 msgr2=0x7ff1240f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.660+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 shutdown_connections 2024-03-18T14:52:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.660+0000 7ff128d51700 1 -- 172.21.15.12:0/1385935939 wait complete. 2024-03-18T14:52:47.674 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:52:47.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.825+0000 7f47af7fe700 1 -- 172.21.15.12:0/2134134733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a400bcb0 con 0x7f47b00662e0 2024-03-18T14:52:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 -- 172.21.15.12:0/2134134733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00662e0 msgr2=0x7f47b00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/2134134733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00662e0 0x7f47b00ff9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47a4004940 tx=0x7f47a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 -- 172.21.15.12:0/2134134733 shutdown_connections 2024-03-18T14:52:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/2134134733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00fff00 0x7f47b0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/2134134733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00662e0 0x7f47b00ff9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 -- 172.21.15.12:0/2134134733 >> 172.21.15.12:0/2134134733 conn(0x7f47b00f7430 msgr2=0x7f47b00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 -- 172.21.15.12:0/2134134733 shutdown_connections 2024-03-18T14:52:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.826+0000 7f47b6bfd700 1 -- 172.21.15.12:0/2134134733 wait complete. 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.827+0000 7f47b6bfd700 1 Processor -- start 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.827+0000 7f47b6bfd700 1 -- start start 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.827+0000 7f47b6bfd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.827+0000 7f47b6bfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00fff00 0x7f47b0110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b6bfd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0110760 con 0x7f47b00fff00 2024-03-18T14:52:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b6bfd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0110fb0 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47affff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00fff00 0x7f47b0110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:49292/0 (socket says 172.21.15.12:49292) 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 -- 172.21.15.12:0/5249050 learned_addr learned my addr 172.21.15.12:0/5249050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 -- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00fff00 msgr2=0x7f47b0110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00fff00 0x7f47b0110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 -- 172.21.15.12:0/5249050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4005040 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47b4999700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b3b0 tx=0x7f47a4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a400e040 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47a400bcc0 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.828+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a401c630 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.829+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b0111230 con 0x7f47b00662e0 2024-03-18T14:52:47.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.829+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b0111770 con 0x7f47b00662e0 2024-03-18T14:52:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.829+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) v1 ==== 46050+0+0 (secure 0 0 0) 0x7f47a4026440 con 0x7f47b00662e0 2024-03-18T14:52:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.829+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b005f5f0 con 0x7f47b00662e0 2024-03-18T14:52:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.830+0000 7f47ad7fa700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 0x7f47a003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.830+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f47a400be30 con 0x7f47b00662e0 2024-03-18T14:52:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.830+0000 7f47affff700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 0x7f47a003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.830+0000 7f47affff700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 0x7f47a003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f479c004990 tx=0x7f479c009ee0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:47.835+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f47a4011780 con 0x7f47b00662e0 2024-03-18T14:52:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.116+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b0111bb0 con 0x7f47a0038d60 2024-03-18T14:52:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.118+0000 7f47ad7fa700 1 -- 172.21.15.12:0/5249050 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f47b0111bb0 con 0x7f47a0038d60 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.120+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 msgr2=0x7f47a003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.120+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 0x7f47a003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f479c004990 tx=0x7f479c009ee0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.120+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 msgr2=0x7f47b010fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.120+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b3b0 tx=0x7f47a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 shutdown_connections 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f47a0038d60 0x7f47a003b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47b00662e0 0x7f47b010fc20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 --2- 172.21.15.12:0/5249050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b00fff00 0x7f47b0110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 >> 172.21.15.12:0/5249050 conn(0x7f47b00f7430 msgr2=0x7f47b00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 shutdown_connections 2024-03-18T14:52:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.121+0000 7f47b6bfd700 1 -- 172.21.15.12:0/5249050 wait complete. 2024-03-18T14:52:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 -- 172.21.15.12:0/2462780508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fcf90 msgr2=0x7f67280fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2462780508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fcf90 0x7f67280fd400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f671c004940 tx=0x7f671c009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 -- 172.21.15.12:0/2462780508 shutdown_connections 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2462780508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fcf90 0x7f67280fd400 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2462780508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fbce0 0x7f67280fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 -- 172.21.15.12:0/2462780508 >> 172.21.15.12:0/2462780508 conn(0x7f67280f7480 msgr2=0x7f67280f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 -- 172.21.15.12:0/2462780508 shutdown_connections 2024-03-18T14:52:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.251+0000 7f672c9b2700 1 -- 172.21.15.12:0/2462780508 wait complete. 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 Processor -- start 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 -- start start 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fcf90 0x7f67280682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59156/0 (socket says 172.21.15.12:59156) 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728068920 con 0x7f67280fbce0 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672c9b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728069170 con 0x7f67280fcf90 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.252+0000 7f672659c700 1 -- 172.21.15.12:0/2000013933 learned_addr learned my addr 172.21.15.12:0/2000013933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f6725d9b700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fcf90 0x7f67280682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f672659c700 1 -- 172.21.15.12:0/2000013933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fcf90 msgr2=0x7f67280682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f672659c700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fcf90 0x7f67280682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f672659c700 1 -- 172.21.15.12:0/2000013933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c005040 con 0x7f67280fbce0 2024-03-18T14:52:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f672659c700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f671800eb40 tx=0x7f671800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.253+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6718007cc0 con 0x7f67280fbce0 2024-03-18T14:52:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.254+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67280693f0 con 0x7f67280fbce0 2024-03-18T14:52:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.254+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6728069930 con 0x7f67280fbce0 2024-03-18T14:52:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.254+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f671800f040 con 0x7f67280fbce0 2024-03-18T14:52:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.254+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67180094e0 con 0x7f67280fbce0 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.255+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67080052b0 con 0x7f67280fbce0 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.255+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 29) v1 ==== 46050+0+0 (secure 0 0 0) 0x7f67180096a0 con 0x7f67280fbce0 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.255+0000 7f6716ffd700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 0x7f671003b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.255+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f671804c700 con 0x7f67280fbce0 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.256+0000 7f6725d9b700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 0x7f671003b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.256+0000 7f6725d9b700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 0x7f671003b260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f671c009b30 tx=0x7f671c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:48.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.261+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f67180118b0 con 0x7f67280fbce0 2024-03-18T14:52:48.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:48 smithi100 ceph-mon[37807]: [18/Mar/2024:14:52:47] ENGINE Bus STARTING 2024-03-18T14:52:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:48 smithi100 ceph-mon[37807]: [18/Mar/2024:14:52:47] ENGINE Serving on http://172.21.15.12:8765 2024-03-18T14:52:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:48 smithi100 ceph-mon[37807]: [18/Mar/2024:14:52:47] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:52:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:48 smithi100 ceph-mon[37807]: [18/Mar/2024:14:52:47] ENGINE Bus STARTED 2024-03-18T14:52:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:48 smithi100 ceph-mon[37807]: mgrmap e29: smithi012.tmscgt(active, since 1.00833s) 2024-03-18T14:52:48.557 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:48 smithi012 ceph-mon[29182]: [18/Mar/2024:14:52:47] ENGINE Bus STARTING 2024-03-18T14:52:48.557 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:48 smithi012 ceph-mon[29182]: [18/Mar/2024:14:52:47] ENGINE Serving on http://172.21.15.12:8765 2024-03-18T14:52:48.557 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:48 smithi012 ceph-mon[29182]: [18/Mar/2024:14:52:47] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:52:48.557 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:48 smithi012 ceph-mon[29182]: [18/Mar/2024:14:52:47] ENGINE Bus STARTED 2024-03-18T14:52:48.558 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:48 smithi012 ceph-mon[29182]: mgrmap e29: smithi012.tmscgt(active, since 1.00833s) 2024-03-18T14:52:48.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.556+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6708000bc0 con 0x7f6710038db0 2024-03-18T14:52:48.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.575+0000 7f6716ffd700 1 -- 172.21.15.12:0/2000013933 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6708000bc0 con 0x7f6710038db0 2024-03-18T14:52:48.575 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (80s) 8s ago 9m 24.5M - 0.25.0 c8568f914cd2 086e7618caee 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (10m) 8s ago 10m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (8m) 11s ago 8m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (50s) 8s ago 9m 77.4M - 9.4.7 954c08fa6188 e6a6e1198813 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (4m) 8s ago 4m 138M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (4m) 8s ago 4m 207M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (4m) 11s ago 4m 15.0M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (4m) 11s ago 3m 12.1M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (15s) 8s ago 11m 90.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283 running (2m) 11s ago 8m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 148db09fb780 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (11m) 8s ago 11m 49.8M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (8m) 11s ago 8m 40.5M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:52:48.576 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (86s) 8s ago 9m 16.6M - 1.5.0 0da6a335fe13 a5d56b3ba88f 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (26s) 11s ago 8m 8044k - 1.5.0 0da6a335fe13 21fdc81b0f67 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (7m) 8s ago 7m 236M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (7m) 8s ago 7m 249M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (6m) 8s ago 6m 207M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (6m) 11s ago 6m 272M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (6m) 11s ago 6m 242M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (5m) 11s ago 5m 220M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:52:48.577 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (29s) 8s ago 9m 44.8M - 2.43.0 a07b618ecd1d 5d9688049cc3 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 msgr2=0x7f671003b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 0x7f671003b260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f671c009b30 tx=0x7f671c009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 msgr2=0x7f6728067d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f671800eb40 tx=0x7f671800b040 comp rx=0 tx=0).stop 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 shutdown_connections 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6710038db0 0x7f671003b260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67280fbce0 0x7f6728067d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 --2- 172.21.15.12:0/2000013933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67280fcf90 0x7f67280682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 >> 172.21.15.12:0/2000013933 conn(0x7f67280f7480 msgr2=0x7f67280663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 shutdown_connections 2024-03-18T14:52:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.578+0000 7f672c9b2700 1 -- 172.21.15.12:0/2000013933 wait complete. 2024-03-18T14:52:48.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.710+0000 7f95c5c7f700 1 -- 172.21.15.12:0/3849848429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 msgr2=0x7f95c00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.710+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/3849848429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c00fd400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f95b4004940 tx=0x7f95b4009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 -- 172.21.15.12:0/3849848429 shutdown_connections 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/3849848429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c00fd400 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/3849848429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 -- 172.21.15.12:0/3849848429 >> 172.21.15.12:0/3849848429 conn(0x7f95c00f7440 msgr2=0x7f95c00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 -- 172.21.15.12:0/3849848429 shutdown_connections 2024-03-18T14:52:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 -- 172.21.15.12:0/3849848429 wait complete. 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.711+0000 7f95c5c7f700 1 Processor -- start 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95c5c7f700 1 -- start start 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95c5c7f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c01106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95c5c7f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95c5c7f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0066be0 con 0x7f95c00fcf90 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95c5c7f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0066d20 con 0x7f95c00fbce0 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95beffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95beffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53332/0 (socket says 172.21.15.12:53332) 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95beffd700 1 -- 172.21.15.12:0/563730936 learned_addr learned my addr 172.21.15.12:0/563730936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95bf7fe700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c01106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95bf7fe700 1 -- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 msgr2=0x7f95c0112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95bf7fe700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95bf7fe700 1 -- 172.21.15.12:0/563730936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b4005040 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.712+0000 7f95beffd700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95bf7fe700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c01106d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009fd0 tx=0x7f95b000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b0004e90 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c0113160 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95b000f040 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b000ca50 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.713+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c01136a0 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.714+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 29) v1 ==== 46050+0+0 (secure 0 0 0) 0x7f95b000cbb0 con 0x7f95c00fbce0 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.714+0000 7f95a7fff700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 0x7f95a803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.714+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f95b004c4a0 con 0x7f95c00fbce0 2024-03-18T14:52:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.714+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c005f5f0 con 0x7f95c00fbce0 2024-03-18T14:52:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.714+0000 7f95beffd700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 0x7f95a803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.715+0000 7f95beffd700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 0x7f95a803b210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95b400b0d0 tx=0x7f95b4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:48.719+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f95b001b2c0 con 0x7f95c00fbce0 2024-03-18T14:52:49.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.178+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95c01047d0 con 0x7f95c00fbce0 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.179+0000 7f95a7fff700 1 -- 172.21.15.12:0/563730936 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f95b0019070 con 0x7f95c00fbce0 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:52:49.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 12, 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:52:49.181 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 msgr2=0x7f95a803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 0x7f95a803b210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f95b400b0d0 tx=0x7f95b4009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 msgr2=0x7f95c01106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c01106d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009fd0 tx=0x7f95b000b040 comp rx=0 tx=0).stop 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 shutdown_connections 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f95a8038d60 0x7f95a803b210 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c00fbce0 0x7f95c01106d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 --2- 172.21.15.12:0/563730936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95c00fcf90 0x7f95c0112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 >> 172.21.15.12:0/563730936 conn(0x7f95c00f7440 msgr2=0x7f95c00f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:49.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 shutdown_connections 2024-03-18T14:52:49.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.182+0000 7f95c5c7f700 1 -- 172.21.15.12:0/563730936 wait complete. 2024-03-18T14:52:49.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:49 smithi100 ceph-mon[37807]: pgmap v3: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:49.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:49 smithi100 ceph-mon[37807]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:49.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:49 smithi100 ceph-mon[37807]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:49 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/563730936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/1239082258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 msgr2=0x7fdd180faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/1239082258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180faee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c004940 tx=0x7fdd0c009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/1239082258 shutdown_connections 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/1239082258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180faee0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/1239082258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd180fe450 0x7fdd180fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.306+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/1239082258 >> 172.21.15.12:0/1239082258 conn(0x7fdd180f6c10 msgr2=0x7fdd180f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.307+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/1239082258 shutdown_connections 2024-03-18T14:52:49.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.307+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/1239082258 wait complete. 2024-03-18T14:52:49.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.308+0000 7fdd1ceb5700 1 Processor -- start 2024-03-18T14:52:49.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd1ceb5700 1 -- start start 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd1ceb5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd180fe450 0x7fdd180fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd1ceb5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd15d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd15d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53360/0 (socket says 172.21.15.12:53360) 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd15d9b700 1 -- 172.21.15.12:0/2060231613 learned_addr learned my addr 172.21.15.12:0/2060231613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180fe060 con 0x7fdd180feda0 2024-03-18T14:52:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.309+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd181115c0 con 0x7fdd180fe450 2024-03-18T14:52:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd15d9b700 1 -- 172.21.15.12:0/2060231613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd180fe450 msgr2=0x7fdd180fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd15d9b700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd180fe450 0x7fdd180fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd15d9b700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c005040 con 0x7fdd180feda0 2024-03-18T14:52:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd15d9b700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b0d0 tx=0x7fdd0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c00e030 con 0x7fdd180feda0 2024-03-18T14:52:49.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd0c01c400 con 0x7fdd180feda0 2024-03-18T14:52:49.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.310+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c00b650 con 0x7fdd180feda0 2024-03-18T14:52:49.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.311+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd18111810 con 0x7fdd180feda0 2024-03-18T14:52:49.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.311+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd180fca80 con 0x7fdd180feda0 2024-03-18T14:52:49.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.313+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 29) v1 ==== 46050+0+0 (secure 0 0 0) 0x7fdd0c01c570 con 0x7fdd180feda0 2024-03-18T14:52:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.313+0000 7fdd06ffd700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 0x7fdd0003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:52:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.313+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1804c710 con 0x7fdd180feda0 2024-03-18T14:52:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.313+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fdd0c04db00 con 0x7fdd180feda0 2024-03-18T14:52:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.313+0000 7fdd1659c700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 0x7fdd0003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:52:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.314+0000 7fdd1659c700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 0x7fdd0003b250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009fd0 tx=0x7fdd0800b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:52:49.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.322+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fdd0c020080 con 0x7fdd180feda0 2024-03-18T14:52:49.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.447+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 30) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fdd0c018ad0 con 0x7fdd180feda0 2024-03-18T14:52:49.659 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:49 smithi012 ceph-mon[29182]: pgmap v3: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:49.659 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:49 smithi012 ceph-mon[29182]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:49.659 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:49 smithi012 ceph-mon[29182]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:49.659 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:49 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/563730936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:52:49.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.698+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd18111750 con 0x7fdd00038da0 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.700+0000 7fdd06ffd700 1 -- 172.21.15.12:0/2060231613 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fdd18111750 con 0x7fdd00038da0 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr" 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:52:49.701 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "2/2 daemons upgraded", 2024-03-18T14:52:49.702 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "", 2024-03-18T14:52:49.702 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:52:49.702 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:52:49.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 msgr2=0x7fdd0003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 0x7fdd0003b250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009fd0 tx=0x7fdd0800b040 comp rx=0 tx=0).stop 2024-03-18T14:52:49.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 msgr2=0x7fdd180fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:52:49.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00b0d0 tx=0x7fdd0c009e30 comp rx=0 tx=0).stop 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 shutdown_connections 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fdd00038da0 0x7fdd0003b250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd180fe450 0x7fdd180fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 --2- 172.21.15.12:0/2060231613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd180feda0 0x7fdd180fc110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.702+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 >> 172.21.15.12:0/2060231613 conn(0x7fdd180f6c10 msgr2=0x7fdd180f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.703+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 shutdown_connections 2024-03-18T14:52:49.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:52:49.703+0000 7fdd1ceb5700 1 -- 172.21.15.12:0/2060231613 wait complete. 2024-03-18T14:52:50.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:50 smithi100 ceph-mon[37807]: pgmap v4: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:50.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:50 smithi100 ceph-mon[37807]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:50.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:50 smithi100 ceph-mon[37807]: mgrmap e30: smithi012.tmscgt(active, since 3s) 2024-03-18T14:52:50.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:50 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:50.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:50 smithi012 ceph-mon[29182]: pgmap v4: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:50.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:50 smithi012 ceph-mon[29182]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:50.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:50 smithi012 ceph-mon[29182]: mgrmap e30: smithi012.tmscgt(active, since 3s) 2024-03-18T14:52:50.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:50 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:51.501 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:51 smithi100 ceph-mon[37807]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:51.501 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:51 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:51.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:51 smithi012 ceph-mon[29182]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:52:51.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:51 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:52.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:52 smithi100 ceph-mon[37807]: pgmap v5: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:52.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:52 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:52.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:52 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:52.633 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:52 smithi012 ceph-mon[29182]: pgmap v5: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:52.634 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:52 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:52.634 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:52 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:54.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:53 smithi012 ceph-mon[29182]: pgmap v6: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:54.293 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:53 smithi100 ceph-mon[37807]: pgmap v6: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:52:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:54 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:52:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:54 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:52:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:54 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:52:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:54 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:52:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:54 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:52:55.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:54 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:52:55.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:54 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:52:55.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:54 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:52:55.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:54 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:52:55.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:54 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/177993737' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:52:56.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:55 smithi012 ceph-mon[29182]: pgmap v7: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 846 KiB/s rd, 1.0 MiB/s wr, 746 op/s 2024-03-18T14:52:56.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:55 smithi012 ceph-mon[29182]: mgrmap e31: smithi012.tmscgt(active, since 8s), standbys: smithi100.rwnrcn 2024-03-18T14:52:56.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:55 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:56.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:55 smithi100 ceph-mon[37807]: pgmap v7: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 846 KiB/s rd, 1.0 MiB/s wr, 746 op/s 2024-03-18T14:52:56.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:55 smithi100 ceph-mon[37807]: mgrmap e31: smithi012.tmscgt(active, since 8s), standbys: smithi100.rwnrcn 2024-03-18T14:52:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:55 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:52:57.708 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-18T14:52:57.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-18T14:52:57.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:57 smithi012 ceph-mon[29182]: pgmap v8: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 657 KiB/s rd, 825 KiB/s wr, 580 op/s 2024-03-18T14:52:57.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:57 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:57.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:57 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:58.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:57 smithi100 ceph-mon[37807]: pgmap v8: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 657 KiB/s rd, 825 KiB/s wr, 580 op/s 2024-03-18T14:52:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:57 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:57 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:58.162 DEBUG:teuthology.parallel:result is None 2024-03-18T14:52:59.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:59 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:59.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:52:59 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:59.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:59 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:52:59.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:52:59 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:00.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:00 smithi012 ceph-mon[29182]: pgmap v9: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 642 op/s 2024-03-18T14:53:00.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:00 smithi100 ceph-mon[37807]: pgmap v9: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 642 op/s 2024-03-18T14:53:00.599 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-18T14:53:00.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-18T14:53:00.902 DEBUG:teuthology.parallel:result is None 2024-03-18T14:53:00.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T14:53:00.933 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T14:53:00.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-18T14:53:00.968 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-18T14:53:00.968 DEBUG:teuthology.parallel:result is None 2024-03-18T14:53:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:02 smithi012 ceph-mon[29182]: pgmap v10: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 588 op/s 2024-03-18T14:53:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:02 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:02 smithi100 ceph-mon[37807]: pgmap v10: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 588 op/s 2024-03-18T14:53:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:02 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:04.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:04 smithi012 ceph-mon[29182]: pgmap v11: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 588 op/s 2024-03-18T14:53:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:04 smithi100 ceph-mon[37807]: pgmap v11: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 588 op/s 2024-03-18T14:53:06.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:06 smithi012 ceph-mon[29182]: pgmap v12: 65 pgs: 65 active+clean; 202 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 701 op/s 2024-03-18T14:53:06.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:06 smithi100 ceph-mon[37807]: pgmap v12: 65 pgs: 65 active+clean; 202 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 701 op/s 2024-03-18T14:53:08.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:08 smithi012 ceph-mon[29182]: pgmap v13: 65 pgs: 65 active+clean; 202 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 266 op/s 2024-03-18T14:53:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:08 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:08 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:08 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:08.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:08 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:08.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:08 smithi100 ceph-mon[37807]: pgmap v13: 65 pgs: 65 active+clean; 202 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 985 KiB/s wr, 266 op/s 2024-03-18T14:53:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:08 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:08 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:08 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:08 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:09.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:09 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:09.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:09 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:10 smithi012 ceph-mon[29182]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 994 KiB/s wr, 281 op/s 2024-03-18T14:53:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:10 smithi012 ceph-mon[29182]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:53:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:10 smithi012 ceph-mon[29182]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:53:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:10 smithi012 ceph-mon[29182]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:53:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:10 smithi012 ceph-mon[29182]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:53:10.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:10 smithi100 ceph-mon[37807]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 994 KiB/s wr, 281 op/s 2024-03-18T14:53:10.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:10 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:53:10.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:10 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:53:10.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:10 smithi100 ceph-mon[37807]: Reconfiguring prometheus.smithi012 (dependencies changed)... 2024-03-18T14:53:10.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:10 smithi100 ceph-mon[37807]: Reconfiguring daemon prometheus.smithi012 on smithi012 2024-03-18T14:53:12.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:12 smithi012 ceph-mon[29182]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 386 KiB/s rd, 401 KiB/s wr, 127 op/s 2024-03-18T14:53:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:12 smithi100 ceph-mon[37807]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 386 KiB/s rd, 401 KiB/s wr, 127 op/s 2024-03-18T14:53:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:14 smithi012 ceph-mon[29182]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 386 KiB/s rd, 401 KiB/s wr, 127 op/s 2024-03-18T14:53:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:14 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:14 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:14 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:53:14.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:14 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:14.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:14 smithi100 ceph-mon[37807]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 386 KiB/s rd, 401 KiB/s wr, 127 op/s 2024-03-18T14:53:14.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:14 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:14.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:14 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:14.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:14 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:53:14.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:14 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:15.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:15 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:53:15.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:15 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:53:16.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:16 smithi100 ceph-mon[37807]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 387 KiB/s rd, 401 KiB/s wr, 128 op/s 2024-03-18T14:53:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:53:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:53:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:16.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:16 smithi012 ceph-mon[29182]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 387 KiB/s rd, 401 KiB/s wr, 128 op/s 2024-03-18T14:53:16.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:53:16.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:53:16.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:17.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:17 smithi100 ceph-mon[37807]: Upgrade: Updating mgr.smithi100.rwnrcn 2024-03-18T14:53:17.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:17 smithi100 ceph-mon[37807]: Deploying daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:53:17.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:17 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:17.701 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:17 smithi012 ceph-mon[29182]: Upgrade: Updating mgr.smithi100.rwnrcn 2024-03-18T14:53:17.701 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:17 smithi012 ceph-mon[29182]: Deploying daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:53:17.701 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:17 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:18.290 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:18 smithi100 ceph-mon[37807]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.4 KiB/s rd, 9.0 KiB/s wr, 15 op/s 2024-03-18T14:53:18.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:18 smithi012 ceph-mon[29182]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.4 KiB/s rd, 9.0 KiB/s wr, 15 op/s 2024-03-18T14:53:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 -- 172.21.15.12:0/330590837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc066260 msgr2=0x7f2bbc0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/330590837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc066260 0x7f2bbc0ffa40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2bac004940 tx=0x7f2bac009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 -- 172.21.15.12:0/330590837 shutdown_connections 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/330590837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/330590837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc066260 0x7f2bbc0ffa40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 -- 172.21.15.12:0/330590837 >> 172.21.15.12:0/330590837 conn(0x7f2bbc0f7430 msgr2=0x7f2bbc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.825+0000 7f2bc0c91700 1 -- 172.21.15.12:0/330590837 shutdown_connections 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 -- 172.21.15.12:0/330590837 wait complete. 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 Processor -- start 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 -- start start 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc066260 0x7f2bbc0fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0fbef0 con 0x7f2bbc0fff80 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bc0c91700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0fc060 con 0x7f2bbc066260 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bb9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44512/0 (socket says 172.21.15.12:44512) 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.826+0000 7f2bb9d9b700 1 -- 172.21.15.12:0/799892099 learned_addr learned my addr 172.21.15.12:0/799892099 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bba59c700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc066260 0x7f2bbc0fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bb9d9b700 1 -- 172.21.15.12:0/799892099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc066260 msgr2=0x7f2bbc0fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bb9d9b700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc066260 0x7f2bbc0fd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bb9d9b700 1 -- 172.21.15.12:0/799892099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac005040 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bb9d9b700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0002a50 tx=0x7f2bb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb0007cc0 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb000f040 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0fc340 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb000da30 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.827+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc115aa0 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.829+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc05f5f0 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.829+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 31) v1 ==== 91657+0+0 (secure 0 0 0) 0x7f2bb000db90 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.829+0000 7f2baaffd700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 0x7f2ba406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.830+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f2bb008cc90 con 0x7f2bbc0fff80 2024-03-18T14:53:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.830+0000 7f2bba59c700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 0x7f2ba406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.830+0000 7f2bba59c700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 0x7f2ba406fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0fd200 tx=0x7f2bac009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:19.834+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f2bb0056860 con 0x7f2bbc0fff80 2024-03-18T14:53:20.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.085+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bbc0f8f70 con 0x7f2ba406d900 2024-03-18T14:53:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.087+0000 7f2baaffd700 1 -- 172.21.15.12:0/799892099 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2bbc0f8f70 con 0x7f2ba406d900 2024-03-18T14:53:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.090+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 msgr2=0x7f2ba406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.090+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 0x7f2ba406fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0fd200 tx=0x7f2bac009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.090+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 msgr2=0x7f2bbc0fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.090+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0002a50 tx=0x7f2bb00046b0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 shutdown_connections 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f2ba406d900 0x7f2ba406fdb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bbc066260 0x7f2bbc0fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 --2- 172.21.15.12:0/799892099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc0fff80 0x7f2bbc0fb940 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 >> 172.21.15.12:0/799892099 conn(0x7f2bbc0f7430 msgr2=0x7f2bbc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 shutdown_connections 2024-03-18T14:53:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.091+0000 7f2bc0c91700 1 -- 172.21.15.12:0/799892099 wait complete. 2024-03-18T14:53:20.107 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:53:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 -- 172.21.15.12:0/36577313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fe4c0 msgr2=0x7fa8940fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/36577313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fe4c0 0x7fa8940fe8d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa890004940 tx=0x7fa890009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 -- 172.21.15.12:0/36577313 shutdown_connections 2024-03-18T14:53:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/36577313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fee10 0x7fa8940faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/36577313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fe4c0 0x7fa8940fe8d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 -- 172.21.15.12:0/36577313 >> 172.21.15.12:0/36577313 conn(0x7fa8940f6ba0 msgr2=0x7fa8940f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 -- 172.21.15.12:0/36577313 shutdown_connections 2024-03-18T14:53:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.205+0000 7fa89bbcc700 1 -- 172.21.15.12:0/36577313 wait complete. 2024-03-18T14:53:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 Processor -- start 2024-03-18T14:53:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 -- start start 2024-03-18T14:53:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fe4c0 0x7fa89410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89410e590 con 0x7fa8940fee10 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa89bbcc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89410ede0 con 0x7fa8940fe4c0 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa899167700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa899167700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44532/0 (socket says 172.21.15.12:44532) 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.206+0000 7fa899167700 1 -- 172.21.15.12:0/3476321310 learned_addr learned my addr 172.21.15.12:0/3476321310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa899968700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fe4c0 0x7fa89410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa899167700 1 -- 172.21.15.12:0/3476321310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fe4c0 msgr2=0x7fa89410da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa899167700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fe4c0 0x7fa89410da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa899167700 1 -- 172.21.15.12:0/3476321310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890005040 con 0x7fa8940fee10 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa899167700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa884002a50 tx=0x7fa8840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa884007cc0 con 0x7fa8940fee10 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa88400f040 con 0x7fa8940fee10 2024-03-18T14:53:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89410f0c0 con 0x7fa8940fee10 2024-03-18T14:53:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa88400da30 con 0x7fa8940fee10 2024-03-18T14:53:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.207+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89410f600 con 0x7fa8940fee10 2024-03-18T14:53:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.208+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89405f5f0 con 0x7fa8940fee10 2024-03-18T14:53:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.210+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 31) v1 ==== 91657+0+0 (secure 0 0 0) 0x7fa88400dbb0 con 0x7fa8940fee10 2024-03-18T14:53:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.210+0000 7fa88a7fc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 0x7fa88006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.210+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fa88408ccb0 con 0x7fa8940fee10 2024-03-18T14:53:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.210+0000 7fa899968700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 0x7fa88006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.211+0000 7fa899968700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 0x7fa88006fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa890009fd0 tx=0x7fa890009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.215+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fa88405a3b0 con 0x7fa8940fee10 2024-03-18T14:53:20.477 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:20 smithi012 ceph-mon[29182]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.8 KiB/s rd, 9.0 KiB/s wr, 16 op/s 2024-03-18T14:53:20.477 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:20 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:20.477 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:20 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:20.477 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:20 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.476+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa89410fa40 con 0x7fa88006d900 2024-03-18T14:53:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.477+0000 7fa88a7fc700 1 -- 172.21.15.12:0/3476321310 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fa89410fa40 con 0x7fa88006d900 2024-03-18T14:53:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 msgr2=0x7fa88006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 0x7fa88006fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa890009fd0 tx=0x7fa890009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 msgr2=0x7fa89410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa884002a50 tx=0x7fa8840046b0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 shutdown_connections 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fa88006d900 0x7fa88006fdb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8940fe4c0 0x7fa89410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 --2- 172.21.15.12:0/3476321310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8940fee10 0x7fa89410df90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.479+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 >> 172.21.15.12:0/3476321310 conn(0x7fa8940f6ba0 msgr2=0x7fa8940f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.480+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 shutdown_connections 2024-03-18T14:53:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.480+0000 7fa89bbcc700 1 -- 172.21.15.12:0/3476321310 wait complete. 2024-03-18T14:53:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:20 smithi100 ceph-mon[37807]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 9.8 KiB/s rd, 9.0 KiB/s wr, 16 op/s 2024-03-18T14:53:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:20 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:20 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:20 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3365042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 msgr2=0x7fe6f40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3365042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f40ffa40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8004940 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3365042056 shutdown_connections 2024-03-18T14:53:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3365042056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 0x7fe6f4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3365042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f40ffa40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3365042056 >> 172.21.15.12:0/3365042056 conn(0x7fe6f40f7430 msgr2=0x7fe6f40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3365042056 shutdown_connections 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3365042056 wait complete. 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 Processor -- start 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.590+0000 7fe6fa2d2700 1 -- start start 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6fa2d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6fa2d2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 0x7fe6f410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6fa2d2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f410e580 con 0x7fe6f4066260 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6fa2d2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f410ed80 con 0x7fe6f40fff80 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44550/0 (socket says 172.21.15.12:44550) 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 -- 172.21.15.12:0/3063622156 learned_addr learned my addr 172.21.15.12:0/3063622156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f37fe700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 0x7fe6f410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 -- 172.21.15.12:0/3063622156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 msgr2=0x7fe6f410df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 0x7fe6f410df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 -- 172.21.15.12:0/3063622156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8005040 con 0x7fe6f4066260 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f3fff700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8000c00 tx=0x7fe6e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.591+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e800e030 con 0x7fe6f4066260 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.592+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6e8013de0 con 0x7fe6f4066260 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.592+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f410f000 con 0x7fe6f4066260 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.592+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e801ce50 con 0x7fe6f4066260 2024-03-18T14:53:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.592+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f410f540 con 0x7fe6f4066260 2024-03-18T14:53:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.593+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f405f5f0 con 0x7fe6f4066260 2024-03-18T14:53:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.593+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 31) v1 ==== 91657+0+0 (secure 0 0 0) 0x7fe6e801c400 con 0x7fe6f4066260 2024-03-18T14:53:20.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.594+0000 7fe6f0ff9700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 0x7fe6dc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.594+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fe6e808e8c0 con 0x7fe6f4066260 2024-03-18T14:53:20.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.594+0000 7fe6f37fe700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 0x7fe6dc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.594+0000 7fe6f37fe700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 0x7fe6dc06fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b80 tx=0x7fe6e400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.602+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe6e805bfc0 con 0x7fe6f4066260 2024-03-18T14:53:20.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.820+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6f410f7d0 con 0x7fe6dc06d900 2024-03-18T14:53:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.830+0000 7fe6f0ff9700 1 -- 172.21.15.12:0/3063622156 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe6f410f7d0 con 0x7fe6dc06d900 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (113s) 24s ago 10m 26.6M - 0.25.0 c8568f914cd2 086e7618caee 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (10m) 24s ago 10m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (9m) 30s ago 9m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (82s) 24s ago 9m 77.5M - 9.4.7 954c08fa6188 e6a6e1198813 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (4m) 24s ago 4m 125M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (4m) 24s ago 4m 189M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (4m) 30s ago 4m 15.3M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:53:20.831 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (4m) 30s ago 4m 12.3M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (47s) 24s ago 11m 477M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 starting - - - - 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (11m) 24s ago 11m 53.6M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (9m) 30s ago 9m 42.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (118s) 24s ago 10m 16.0M - 1.5.0 0da6a335fe13 a5d56b3ba88f 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (59s) 30s ago 9m 7936k - 1.5.0 0da6a335fe13 21fdc81b0f67 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (8m) 24s ago 8m 245M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (7m) 24s ago 7m 256M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (7m) 24s ago 7m 213M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (7m) 30s ago 7m 282M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (6m) 30s ago 6m 251M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (6m) 30s ago 6m 228M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:53:20.832 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 starting - - - - 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 msgr2=0x7fe6dc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 0x7fe6dc06fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b80 tx=0x7fe6e400b040 comp rx=0 tx=0).stop 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 msgr2=0x7fe6f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8000c00 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 shutdown_connections 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe6dc06d900 0x7fe6dc06fdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6f4066260 0x7fe6f410da40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 --2- 172.21.15.12:0/3063622156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6f40fff80 0x7fe6f410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 >> 172.21.15.12:0/3063622156 conn(0x7fe6f40f7430 msgr2=0x7fe6f40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 shutdown_connections 2024-03-18T14:53:20.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.833+0000 7fe6fa2d2700 1 -- 172.21.15.12:0/3063622156 wait complete. 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 -- 172.21.15.12:0/2446777788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 msgr2=0x7f07880fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2446777788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f07880fd250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0778004940 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 -- 172.21.15.12:0/2446777788 shutdown_connections 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2446777788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 0x7f07880fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2446777788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f07880fd250 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 -- 172.21.15.12:0/2446777788 >> 172.21.15.12:0/2446777788 conn(0x7f07880f8550 msgr2=0x7f07880fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 -- 172.21.15.12:0/2446777788 shutdown_connections 2024-03-18T14:53:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.943+0000 7f078d9a0700 1 -- 172.21.15.12:0/2446777788 wait complete. 2024-03-18T14:53:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 Processor -- start 2024-03-18T14:53:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 -- start start 2024-03-18T14:53:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 0x7f078810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078810e4f0 con 0x7f07880fce40 2024-03-18T14:53:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f078d9a0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078810ed40 con 0x7f07880fe040 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44558/0 (socket says 172.21.15.12:44558) 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 -- 172.21.15.12:0/2610216836 learned_addr learned my addr 172.21.15.12:0/2610216836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f07867fc700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 0x7f078810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 -- 172.21.15.12:0/2610216836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 msgr2=0x7f078810def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 0x7f078810def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.944+0000 7f0786ffd700 1 -- 172.21.15.12:0/2610216836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778005040 con 0x7f07880fce40 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f0786ffd700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0778000c00 tx=0x7f0778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077800e030 con 0x7f07880fce40 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0778013de0 con 0x7f07880fce40 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078810efc0 con 0x7f07880fce40 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077801ce50 con 0x7f07880fce40 2024-03-18T14:53:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.945+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078810f500 con 0x7f07880fce40 2024-03-18T14:53:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.946+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078805f5f0 con 0x7f07880fce40 2024-03-18T14:53:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.946+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 31) v1 ==== 91657+0+0 (secure 0 0 0) 0x7f077801c400 con 0x7f07880fce40 2024-03-18T14:53:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.947+0000 7f077f7fe700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 0x7f077406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.947+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f077808e2d0 con 0x7f07880fce40 2024-03-18T14:53:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.947+0000 7f07867fc700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 0x7f077406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.947+0000 7f07867fc700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 0x7f077406fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0770009b80 tx=0x7f077000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:20.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:20.955+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f077805b9d0 con 0x7f07880fce40 2024-03-18T14:53:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.266+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f07881058d0 con 0x7f07880fce40 2024-03-18T14:53:21.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.267+0000 7f077f7fe700 1 -- 172.21.15.12:0/2610216836 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f077805b560 con 0x7f07880fce40 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:53:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 12, 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:53:21.269 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 msgr2=0x7f077406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 0x7f077406fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0770009b80 tx=0x7f077000b040 comp rx=0 tx=0).stop 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 msgr2=0x7f078810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0778000c00 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 shutdown_connections 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f077406d900 0x7f077406fdb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f07880fce40 0x7f078810d9b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 --2- 172.21.15.12:0/2610216836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07880fe040 0x7f078810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 >> 172.21.15.12:0/2610216836 conn(0x7f07880f8550 msgr2=0x7f07880f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 shutdown_connections 2024-03-18T14:53:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.270+0000 7f078d9a0700 1 -- 172.21.15.12:0/2610216836 wait complete. 2024-03-18T14:53:21.283 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:21 smithi100 ceph-mon[37807]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.375+0000 7f6faa018700 1 -- 172.21.15.12:0/2342795867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40fff00 msgr2=0x7f6fa4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.375+0000 7f6faa018700 1 --2- 172.21.15.12:0/2342795867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40fff00 0x7f6fa4102320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6f94004940 tx=0x7f6f94009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.375+0000 7f6faa018700 1 -- 172.21.15.12:0/2342795867 shutdown_connections 2024-03-18T14:53:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.375+0000 7f6faa018700 1 --2- 172.21.15.12:0/2342795867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40fff00 0x7f6fa4102320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 --2- 172.21.15.12:0/2342795867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40662e0 0x7f6fa40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 -- 172.21.15.12:0/2342795867 >> 172.21.15.12:0/2342795867 conn(0x7f6fa40f7430 msgr2=0x7f6fa40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:21.377 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:21 smithi012 ceph-mon[29182]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 -- 172.21.15.12:0/2342795867 shutdown_connections 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 -- 172.21.15.12:0/2342795867 wait complete. 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 Processor -- start 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 -- start start 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.376+0000 7f6faa018700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6faa018700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40fff00 0x7f6fa4110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6faa018700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4110790 con 0x7f6fa40662e0 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6faa018700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4110f90 con 0x7f6fa40fff00 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44564/0 (socket says 172.21.15.12:44564) 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 -- 172.21.15.12:0/4060310180 learned_addr learned my addr 172.21.15.12:0/4060310180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa2ffd700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40fff00 0x7f6fa4110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 -- 172.21.15.12:0/4060310180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40fff00 msgr2=0x7f6fa4110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40fff00 0x7f6fa4110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 -- 172.21.15.12:0/4060310180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94005040 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6fa37fe700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6f98002a50 tx=0x7f6f980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f98007cc0 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa4111270 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f9800f040 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f9800db30 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.377+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa41117b0 con 0x7f6fa40662e0 2024-03-18T14:53:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.378+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa405f5f0 con 0x7f6fa40662e0 2024-03-18T14:53:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.379+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 31) v1 ==== 91657+0+0 (secure 0 0 0) 0x7f6f9800dc90 con 0x7f6fa40662e0 2024-03-18T14:53:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.379+0000 7f6f8bfff700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 0x7f6f8c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.379+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f6f9808cc20 con 0x7f6fa40662e0 2024-03-18T14:53:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.379+0000 7f6fa2ffd700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 0x7f6f8c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.380+0000 7f6fa2ffd700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 0x7f6f8c06fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400b3b0 tx=0x7f6f94009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:21.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.384+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f6f9805a2a0 con 0x7f6fa40662e0 2024-03-18T14:53:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.615+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fa4111b30 con 0x7f6f8c06d900 2024-03-18T14:53:21.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.616+0000 7f6f8bfff700 1 -- 172.21.15.12:0/4060310180 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f6fa4111b30 con 0x7f6f8c06d900 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "", 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mgr daemons", 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:53:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:53:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 msgr2=0x7f6f8c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 0x7f6f8c06fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400b3b0 tx=0x7f6f94009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 msgr2=0x7f6fa410fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6f98002a50 tx=0x7f6f980046b0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 shutdown_connections 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6f8c06d900 0x7f6f8c06fdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fa40662e0 0x7f6fa410fc50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 --2- 172.21.15.12:0/4060310180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40fff00 0x7f6fa4110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.619+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 >> 172.21.15.12:0/4060310180 conn(0x7f6fa40f7430 msgr2=0x7f6fa40fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.620+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 shutdown_connections 2024-03-18T14:53:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:21.620+0000 7f6faa018700 1 -- 172.21.15.12:0/4060310180 wait complete. 2024-03-18T14:53:22.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:22 smithi100 ceph-mon[37807]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:22.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:22 smithi100 ceph-mon[37807]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:22.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:22 smithi100 ceph-mon[37807]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:22.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:22 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/2610216836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:22.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:22 smithi012 ceph-mon[29182]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:22.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:22 smithi012 ceph-mon[29182]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:22.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:22 smithi012 ceph-mon[29182]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:22.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:22 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/2610216836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:23.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:23 smithi100 ceph-mon[37807]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:23 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:23.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:23 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:23.663 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:23 smithi012 ceph-mon[29182]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:23.663 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:23 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:23.663 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:23 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:24.515 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:24 smithi012 ceph-mon[29182]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:24.515 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:24 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:24.516 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:24 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:24 smithi100 ceph-mon[37807]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:24 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:24.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:24 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:26 smithi100 ceph-mon[37807]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:26 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:26 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:26.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:26 smithi012 ceph-mon[29182]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:26.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:26 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:26.376 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:26 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:27.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:53:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:53:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:53:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:53:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:53:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:27 smithi012 ceph-mon[29182]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:53:27.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:53:27.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:53:27.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:53:27.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:53:27.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:53:27.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:27 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/3839975292' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:53:28.396 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: mgrmap e32: smithi012.tmscgt(active, since 40s), standbys: smithi100.rwnrcn 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi012.tmscgt"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi012.tmscgt"}]': finished 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi100.rwnrcn"}]: dispatch 2024-03-18T14:53:28.397 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi100.rwnrcn"}]': finished 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.398 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:28 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: mgrmap e32: smithi012.tmscgt(active, since 40s), standbys: smithi100.rwnrcn 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi012.tmscgt"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi012.tmscgt"}]': finished 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi100.rwnrcn"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi100.rwnrcn"}]': finished 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:28.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:28 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:29.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all mgr 2024-03-18T14:53:29.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all rgw 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all rbd-mirror 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all ceph-exporter 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all iscsi 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all nfs 2024-03-18T14:53:29.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:29 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all nvmeof 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all mgr 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all rgw 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all rbd-mirror 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all ceph-exporter 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all iscsi 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all nfs 2024-03-18T14:53:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:29 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all nvmeof 2024-03-18T14:53:30.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:30 smithi012 ceph-mon[29182]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:30.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:30 smithi100 ceph-mon[37807]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:31.431 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:31 smithi012 ceph-mon[29182]: Upgrade: Updating node-exporter.smithi012 (1/2) 2024-03-18T14:53:31.431 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:31 smithi012 ceph-mon[29182]: Deploying daemon node-exporter.smithi012 on smithi012 2024-03-18T14:53:31.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:31 smithi100 ceph-mon[37807]: Upgrade: Updating node-exporter.smithi012 (1/2) 2024-03-18T14:53:31.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:31 smithi100 ceph-mon[37807]: Deploying daemon node-exporter.smithi012 on smithi012 2024-03-18T14:53:32.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:32 smithi012 ceph-mon[29182]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:32.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:32 smithi100 ceph-mon[37807]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:32.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:34.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:34 smithi012 ceph-mon[29182]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:34.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:34 smithi100 ceph-mon[37807]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:35.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:35 smithi012 ceph-mon[29182]: Upgrade: Updating node-exporter.smithi100 (2/2) 2024-03-18T14:53:35.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:35 smithi012 ceph-mon[29182]: Deploying daemon node-exporter.smithi100 on smithi100 2024-03-18T14:53:35.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:35 smithi100 ceph-mon[37807]: Upgrade: Updating node-exporter.smithi100 (2/2) 2024-03-18T14:53:35.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:35 smithi100 ceph-mon[37807]: Deploying daemon node-exporter.smithi100 on smithi100 2024-03-18T14:53:36.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:36 smithi100 ceph-mon[37807]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:36.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:36 smithi012 ceph-mon[29182]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:37.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:37.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:37.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:38 smithi100 ceph-mon[37807]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:38.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:38 smithi012 ceph-mon[29182]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:40.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:40 smithi012 ceph-mon[29182]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:40.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:40 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:40.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:40 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:40.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:40 smithi100 ceph-mon[37807]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:40.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:40 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:40.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:40 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:41.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:41 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:41.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:41 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:41.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:41 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:41.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:41 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:42.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:42 smithi012 ceph-mon[29182]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:42 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:42.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:42 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:42.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:42 smithi100 ceph-mon[37807]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:42.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:42 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:42.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:42 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:43.542 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:43 smithi100 ceph-mon[37807]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:43.542 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:43 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:43.543 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:43 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:43 smithi012 ceph-mon[29182]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:43.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:43 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:43.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:43 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:44 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:44.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:44 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:44.974 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:44 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:44.975 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:44 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:45.899 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:45 smithi100 ceph-mon[37807]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:45.971 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:45 smithi012 ceph-mon[29182]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:47.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:47 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:47.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:47 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:53:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:48 smithi012 ceph-mon[29182]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:48 smithi100 ceph-mon[37807]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:49.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:49 smithi100 ceph-mon[37807]: Upgrade: Updating prometheus.smithi012 2024-03-18T14:53:49.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:49 smithi100 ceph-mon[37807]: Deploying daemon prometheus.smithi012 on smithi012 2024-03-18T14:53:49.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:49 smithi012 ceph-mon[29182]: Upgrade: Updating prometheus.smithi012 2024-03-18T14:53:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:49 smithi012 ceph-mon[29182]: Deploying daemon prometheus.smithi012 on smithi012 2024-03-18T14:53:50.187 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:50 smithi012 ceph-mon[29182]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:50.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:50 smithi100 ceph-mon[37807]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:51.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 -- 172.21.15.12:0/4259169235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fce70 msgr2=0x7fe7840fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 --2- 172.21.15.12:0/4259169235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fce70 0x7fe7840fd280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009ce0 tx=0x7fe77c004d10 comp rx=0 tx=0).stop 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 -- 172.21.15.12:0/4259169235 shutdown_connections 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 --2- 172.21.15.12:0/4259169235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fdfe0 0x7fe7840fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 --2- 172.21.15.12:0/4259169235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fce70 0x7fe7840fd280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 -- 172.21.15.12:0/4259169235 >> 172.21.15.12:0/4259169235 conn(0x7fe7840f8550 msgr2=0x7fe7840fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 -- 172.21.15.12:0/4259169235 shutdown_connections 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.741+0000 7fe788e1f700 1 -- 172.21.15.12:0/4259169235 wait complete. 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 Processor -- start 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 -- start start 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 0x7fe784071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784072260 con 0x7fe7840fdfe0 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe788e1f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784070850 con 0x7fe7840fce70 2024-03-18T14:53:51.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36672/0 (socket says 172.21.15.12:36672) 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 -- 172.21.15.12:0/22212709 learned_addr learned my addr 172.21.15.12:0/22212709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe78259c700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 0x7fe784071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 -- 172.21.15.12:0/22212709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 msgr2=0x7fe784071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 0x7fe784071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe781d9b700 1 -- 172.21.15.12:0/22212709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009940 con 0x7fe7840fdfe0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.742+0000 7fe78259c700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 0x7fe784071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe781d9b700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe76c002a50 tx=0x7fe76c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c007cc0 con 0x7fe7840fdfe0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76c00f040 con 0x7fe7840fdfe0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe784070b00 con 0x7fe7840fdfe0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c009640 con 0x7fe7840fdfe0 2024-03-18T14:53:51.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.743+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7840710c0 con 0x7fe7840fdfe0 2024-03-18T14:53:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.744+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78404c710 con 0x7fe7840fdfe0 2024-03-18T14:53:51.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.744+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 32) v1 ==== 91657+0+0 (secure 0 0 0) 0x7fe76c0097a0 con 0x7fe7840fdfe0 2024-03-18T14:53:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.745+0000 7fe77affd700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 0x7fe77006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.745+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fe76c08cda0 con 0x7fe7840fdfe0 2024-03-18T14:53:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.745+0000 7fe78259c700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 0x7fe77006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.745+0000 7fe78259c700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 0x7fe77006fdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005010 tx=0x7fe77c004d10 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:51.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.750+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe76c05a420 con 0x7fe7840fdfe0 2024-03-18T14:53:51.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.981+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe784102340 con 0x7fe77006d910 2024-03-18T14:53:51.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.982+0000 7fe77affd700 1 -- 172.21.15.12:0/22212709 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fe784102340 con 0x7fe77006d910 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 msgr2=0x7fe77006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 0x7fe77006fdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005010 tx=0x7fe77c004d10 comp rx=0 tx=0).stop 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 msgr2=0x7fe784070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe76c002a50 tx=0x7fe76c0046b0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 shutdown_connections 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe77006d910 0x7fe77006fdc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840fce70 0x7fe784071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 --2- 172.21.15.12:0/22212709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7840fdfe0 0x7fe784070310 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 >> 172.21.15.12:0/22212709 conn(0x7fe7840f8550 msgr2=0x7fe7840f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 shutdown_connections 2024-03-18T14:53:51.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:51.985+0000 7fe788e1f700 1 -- 172.21.15.12:0/22212709 wait complete. 2024-03-18T14:53:51.998 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 -- 172.21.15.12:0/2001946877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 msgr2=0x7f28c80fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 --2- 172.21.15.12:0/2001946877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c80fb620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f28b8004940 tx=0x7f28b8009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 -- 172.21.15.12:0/2001946877 shutdown_connections 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 --2- 172.21.15.12:0/2001946877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c80fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 --2- 172.21.15.12:0/2001946877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c80fb620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 -- 172.21.15.12:0/2001946877 >> 172.21.15.12:0/2001946877 conn(0x7f28c80f6c10 msgr2=0x7f28c80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 -- 172.21.15.12:0/2001946877 shutdown_connections 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.100+0000 7f28cdee5700 1 -- 172.21.15.12:0/2001946877 wait complete. 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 Processor -- start 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 -- start start 2024-03-18T14:53:52.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c81102b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8110990 con 0x7f28c80fb210 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28cdee5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c81152a0 con 0x7f28c80fbb60 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28c77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36690/0 (socket says 172.21.15.12:36690) 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28c77fe700 1 -- 172.21.15.12:0/1077424575 learned_addr learned my addr 172.21.15.12:0/1077424575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.101+0000 7f28c6ffd700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c81102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28c77fe700 1 -- 172.21.15.12:0/1077424575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 msgr2=0x7f28c81102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28c77fe700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c81102b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28c77fe700 1 -- 172.21.15.12:0/1077424575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8005040 con 0x7f28c80fb210 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28c6ffd700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c81102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28c77fe700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b3b0 tx=0x7f28b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b800e040 con 0x7f28c80fb210 2024-03-18T14:53:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28b8004290 con 0x7f28c80fb210 2024-03-18T14:53:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c81154a0 con 0x7f28c80fb210 2024-03-18T14:53:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b8013ea0 con 0x7f28c80fb210 2024-03-18T14:53:52.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.102+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c81159d0 con 0x7f28c80fb210 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.103+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8101d50 con 0x7f28c80fb210 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.103+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 32) v1 ==== 91657+0+0 (secure 0 0 0) 0x7f28b800bbf0 con 0x7f28c80fb210 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.104+0000 7f28affff700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 0x7f28b006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.104+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f28b808e4f0 con 0x7f28c80fb210 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.104+0000 7f28c6ffd700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 0x7f28b006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.104+0000 7f28c6ffd700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 0x7f28b006fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.109+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f28b80580c0 con 0x7f28c80fb210 2024-03-18T14:53:52.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:52 smithi100 ceph-mon[37807]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:52.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.343+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28c8105cb0 con 0x7f28b006d910 2024-03-18T14:53:52.344 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:52 smithi012 ceph-mon[29182]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:52.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.345+0000 7f28affff700 1 -- 172.21.15.12:0/1077424575 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f28c8105cb0 con 0x7f28b006d910 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.346+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 msgr2=0x7f28b006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 0x7f28b006fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc008040 comp rx=0 tx=0).stop 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 msgr2=0x7f28c810fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b3b0 tx=0x7f28b8009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 shutdown_connections 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f28b006d910 0x7f28b006fdc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c80fb210 0x7f28c810fd70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 --2- 172.21.15.12:0/1077424575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28c80fbb60 0x7f28c81102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 >> 172.21.15.12:0/1077424575 conn(0x7f28c80f6c10 msgr2=0x7f28c80f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 shutdown_connections 2024-03-18T14:53:52.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.347+0000 7f28cdee5700 1 -- 172.21.15.12:0/1077424575 wait complete. 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- 172.21.15.12:0/3998141677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fce70 msgr2=0x7f5bd00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/3998141677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fce70 0x7f5bd00fd280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009ce0 tx=0x7f5bc0004d10 comp rx=0 tx=0).stop 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- 172.21.15.12:0/3998141677 shutdown_connections 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/3998141677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd00fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/3998141677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fce70 0x7f5bd00fd280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- 172.21.15.12:0/3998141677 >> 172.21.15.12:0/3998141677 conn(0x7f5bd00f8510 msgr2=0x7f5bd00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- 172.21.15.12:0/3998141677 shutdown_connections 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- 172.21.15.12:0/3998141677 wait complete. 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 Processor -- start 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.451+0000 7f5bd48b9700 1 -- start start 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 0x7f5bd010fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0111090 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0111200 con 0x7f5bd00fce70 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36718/0 (socket says 172.21.15.12:36718) 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 -- 172.21.15.12:0/2165853617 learned_addr learned my addr 172.21.15.12:0/2165853617 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bce59c700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 0x7f5bd010fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 -- 172.21.15.12:0/2165853617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 msgr2=0x7f5bd010fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 0x7f5bd010fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 -- 172.21.15.12:0/2165853617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009940 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bce59c700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 0x7f5bd010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bcdd9b700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c830 tx=0x7f5bb8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb800a760 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0111480 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bb800ad90 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb80113a0 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.452+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd0111a40 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.453+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0101970 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.454+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 32) v1 ==== 91657+0+0 (secure 0 0 0) 0x7f5bb8012400 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.455+0000 7f5bc6ffd700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 0x7f5bbc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.455+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f5bb808cf50 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.455+0000 7f5bce59c700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 0x7f5bbc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.455+0000 7f5bce59c700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 0x7f5bbc06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0015040 tx=0x7f5bc0004d10 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.459+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f5bb80c1040 con 0x7f5bd00fdfe0 2024-03-18T14:53:52.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.690+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd0105a80 con 0x7f5bbc06d900 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.700+0000 7f5bc6ffd700 1 -- 172.21.15.12:0/2165853617 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f5bc00090d0 con 0x7f5bbc06d900 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (2m) 11s ago 10m 26.7M - 0.25.0 c8568f914cd2 086e7618caee 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (11m) 11s ago 11m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (9m) 13s ago 9m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (114s) 11s ago 10m 77.8M - 9.4.7 954c08fa6188 e6a6e1198813 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (5m) 11s ago 5m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (5m) 11s ago 5m 152M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (5m) 13s ago 5m 15.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:53:52.701 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (5m) 13s ago 5m 12.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (79s) 11s ago 12m 495M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (33s) 13s ago 9m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (12m) 11s ago 12m 52.1M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (9m) 13s ago 9m 42.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (21s) 11s ago 10m 7130k - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (17s) 13s ago 9m 3594k - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (8m) 11s ago 8m 256M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (8m) 11s ago 8m 267M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (7m) 11s ago 7m 220M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (7m) 13s ago 7m 288M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (7m) 13s ago 7m 257M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (6m) 13s ago 6m 236M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:53:52.702 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (39s) 11s ago 10m 50.6M - 2.43.0 a07b618ecd1d 976b24346840 2024-03-18T14:53:52.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 msgr2=0x7f5bbc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 0x7f5bbc06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0015040 tx=0x7f5bc0004d10 comp rx=0 tx=0).stop 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 msgr2=0x7f5bd0110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c830 tx=0x7f5bb8004470 comp rx=0 tx=0).stop 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 shutdown_connections 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f5bbc06d900 0x7f5bbc06fdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.702+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bd00fce70 0x7f5bd010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.703+0000 7f5bd48b9700 1 --2- 172.21.15.12:0/2165853617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bd00fdfe0 0x7f5bd0110320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.703+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 >> 172.21.15.12:0/2165853617 conn(0x7f5bd00f8510 msgr2=0x7f5bd00f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.703+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 shutdown_connections 2024-03-18T14:53:52.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.703+0000 7f5bd48b9700 1 -- 172.21.15.12:0/2165853617 wait complete. 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 -- 172.21.15.12:0/3222589905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 msgr2=0x7ff5200fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 --2- 172.21.15.12:0/3222589905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fd710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff514004940 tx=0x7ff514009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 -- 172.21.15.12:0/3222589905 shutdown_connections 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 --2- 172.21.15.12:0/3222589905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5200fdc50 0x7ff5200fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 --2- 172.21.15.12:0/3222589905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fd710 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 -- 172.21.15.12:0/3222589905 >> 172.21.15.12:0/3222589905 conn(0x7ff5200f6be0 msgr2=0x7ff5200f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 -- 172.21.15.12:0/3222589905 shutdown_connections 2024-03-18T14:53:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.806+0000 7ff5257ce700 1 -- 172.21.15.12:0/3222589905 wait complete. 2024-03-18T14:53:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 Processor -- start 2024-03-18T14:53:52.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 -- start start 2024-03-18T14:53:52.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5200fdc50 0x7ff520114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52010e550 con 0x7ff5200fd300 2024-03-18T14:53:52.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201150b0 con 0x7ff5200fdc50 2024-03-18T14:53:52.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36746/0 (socket says 172.21.15.12:36746) 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 -- 172.21.15.12:0/388156089 learned_addr learned my addr 172.21.15.12:0/388156089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 -- 172.21.15.12:0/388156089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5200fdc50 msgr2=0x7ff520114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5200fdc50 0x7ff520114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 -- 172.21.15.12:0/388156089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514005040 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff51effd700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff514009380 tx=0x7ff514009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51400e040 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff51400bcc0 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5201152b0 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.807+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51401c5f0 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.809+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520115870 con 0x7ff5200fd300 2024-03-18T14:53:52.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.809+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52004c710 con 0x7ff5200fd300 2024-03-18T14:53:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.809+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 32) v1 ==== 91657+0+0 (secure 0 0 0) 0x7ff514013a80 con 0x7ff5200fd300 2024-03-18T14:53:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.810+0000 7ff5077fe700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 0x7ff50806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.810+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff51408dc60 con 0x7ff5200fd300 2024-03-18T14:53:52.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.810+0000 7ff51e7fc700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 0x7ff50806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:52.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.810+0000 7ff51e7fc700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 0x7ff50806fdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006db0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:52.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:52.814+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff51405b360 con 0x7ff5200fd300 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.140+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff520115a90 con 0x7ff5200fd300 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.140+0000 7ff5077fe700 1 -- 172.21.15.12:0/388156089 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff51405aef0 con 0x7ff5200fd300 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:53:53.142 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:53:53.143 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 12, 2024-03-18T14:53:53.144 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:53:53.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:53:53.144 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 msgr2=0x7ff50806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 0x7ff50806fdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510006db0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 msgr2=0x7ff5200fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff514009380 tx=0x7ff514009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 shutdown_connections 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff50806d910 0x7ff50806fdc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5200fd300 0x7ff5200fb9b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 --2- 172.21.15.12:0/388156089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5200fdc50 0x7ff520114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.142+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 >> 172.21.15.12:0/388156089 conn(0x7ff5200f6be0 msgr2=0x7ff5200f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.143+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 shutdown_connections 2024-03-18T14:53:53.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.143+0000 7ff5257ce700 1 -- 172.21.15.12:0/388156089 wait complete. 2024-03-18T14:53:53.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:53 smithi012 ceph-mon[29182]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:53.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- 172.21.15.12:0/794844495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fe070 msgr2=0x7fcd700fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 --2- 172.21.15.12:0/794844495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fe070 0x7fcd700fe4e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c004940 tx=0x7fcd5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- 172.21.15.12:0/794844495 shutdown_connections 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 --2- 172.21.15.12:0/794844495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fe070 0x7fcd700fe4e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 --2- 172.21.15.12:0/794844495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fcf00 0x7fcd700fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- 172.21.15.12:0/794844495 >> 172.21.15.12:0/794844495 conn(0x7fcd700f85a0 msgr2=0x7fcd700fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- 172.21.15.12:0/794844495 shutdown_connections 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- 172.21.15.12:0/794844495 wait complete. 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 Processor -- start 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.260+0000 7fcd76f23700 1 -- start start 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd76f23700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd76f23700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 0x7fcd7010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd76f23700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7010e5f0 con 0x7fcd700fcf00 2024-03-18T14:53:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd76f23700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7010ede0 con 0x7fcd700fe070 2024-03-18T14:53:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36760/0 (socket says 172.21.15.12:36760) 2024-03-18T14:53:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 -- 172.21.15.12:0/1968285360 learned_addr learned my addr 172.21.15.12:0/1968285360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:53:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd6ffff700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 0x7fcd7010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 -- 172.21.15.12:0/1968285360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 msgr2=0x7fcd7010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 0x7fcd7010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd74cbf700 1 -- 172.21.15.12:0/1968285360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c005040 con 0x7fcd700fcf00 2024-03-18T14:53:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.261+0000 7fcd6ffff700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 0x7fcd7010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd74cbf700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b790 tx=0x7fcd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6400f770 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7010f0c0 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd6400fda0 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6400d310 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.262+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7010f680 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.264+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7005f5f0 con 0x7fcd700fcf00 2024-03-18T14:53:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.264+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 32) v1 ==== 91657+0+0 (secure 0 0 0) 0x7fcd6400f8d0 con 0x7fcd700fcf00 2024-03-18T14:53:53.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.264+0000 7fcd6d7fa700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 0x7fcd6006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:53:53.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.264+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fcd6408ced0 con 0x7fcd700fcf00 2024-03-18T14:53:53.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.264+0000 7fcd6ffff700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 0x7fcd6006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:53:53.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.265+0000 7fcd6ffff700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 0x7fcd6006fe80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00b7a0 tx=0x7fcd5c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:53:53.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.271+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fcd6405a6c0 con 0x7fcd700fcf00 2024-03-18T14:53:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.501+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd7010f9f0 con 0x7fcd6006d9d0 2024-03-18T14:53:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.502+0000 7fcd6d7fa700 1 -- 172.21.15.12:0/1968285360 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fcd7010f9f0 con 0x7fcd6006d9d0 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr" 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "2/2 daemons upgraded", 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:53:53.503 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.504+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 msgr2=0x7fcd6006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.504+0000 7fcd76f23700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 0x7fcd6006fe80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00b7a0 tx=0x7fcd5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.504+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 msgr2=0x7fcd7010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.504+0000 7fcd76f23700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b790 tx=0x7fcd64009250 comp rx=0 tx=0).stop 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 shutdown_connections 2024-03-18T14:53:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fcd6006d9d0 0x7fcd6006fe80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd700fcf00 0x7fcd7010da20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 --2- 172.21.15.12:0/1968285360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd700fe070 0x7fcd7010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:53:53.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 >> 172.21.15.12:0/1968285360 conn(0x7fcd700f85a0 msgr2=0x7fcd700f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:53:53.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 shutdown_connections 2024-03-18T14:53:53.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:53:53.505+0000 7fcd76f23700 1 -- 172.21.15.12:0/1968285360 wait complete. 2024-03-18T14:53:53.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:53 smithi100 ceph-mon[37807]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/388156089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:54.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:54 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/388156089' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:54.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:54 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:53:55.320 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:55 smithi012 ceph-mon[29182]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:55.320 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:55 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:55.320 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:55 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:55.403 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:55 smithi100 ceph-mon[37807]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:53:55.403 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:55 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:55.403 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:55 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:53:56.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:56 smithi100 ceph-mon[37807]: pgmap v37: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:56.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:56 smithi012 ceph-mon[29182]: pgmap v37: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:58.343 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:58 smithi012 ceph-mon[29182]: pgmap v38: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:53:58.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:58 smithi100 ceph-mon[37807]: pgmap v38: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:00.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:59 smithi012 ceph-mon[29182]: pgmap v39: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:59 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:53:59 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:00.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:59 smithi100 ceph-mon[37807]: pgmap v39: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:59 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:53:59 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:01.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:01 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:01.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:01 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:01.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:01 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:01.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:01 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:02.348 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:02 smithi012 ceph-mon[29182]: pgmap v40: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:02.348 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:02 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:02 smithi100 ceph-mon[37807]: pgmap v40: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:02 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: pgmap v41: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:04.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:04 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: pgmap v41: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:04.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:04.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:04 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:05.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:05 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:54:05.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:05 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-18T14:54:06.374 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:06 smithi012 ceph-mon[29182]: pgmap v42: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:06.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:06 smithi100 ceph-mon[37807]: pgmap v42: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:07.193 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:07 smithi012 ceph-mon[29182]: Upgrade: Updating alertmanager.smithi012 2024-03-18T14:54:07.193 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:07 smithi012 ceph-mon[29182]: Deploying daemon alertmanager.smithi012 on smithi012 2024-03-18T14:54:07.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:07 smithi100 ceph-mon[37807]: Upgrade: Updating alertmanager.smithi012 2024-03-18T14:54:07.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:07 smithi100 ceph-mon[37807]: Deploying daemon alertmanager.smithi012 on smithi012 2024-03-18T14:54:08.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:08 smithi012 ceph-mon[29182]: pgmap v43: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:08.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:08 smithi100 ceph-mon[37807]: pgmap v43: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:10 smithi012 ceph-mon[29182]: pgmap v44: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:10.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:10 smithi100 ceph-mon[37807]: pgmap v44: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:12 smithi012 ceph-mon[29182]: pgmap v45: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:12 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:12 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:12 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:12 smithi100 ceph-mon[37807]: pgmap v45: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:12 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:12 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:12 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:14.348 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:14 smithi100 ceph-mon[37807]: pgmap v46: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:14.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:14 smithi012 ceph-mon[29182]: pgmap v46: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:16.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:16 smithi012 ceph-mon[29182]: pgmap v47: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:16.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:16 smithi100 ceph-mon[37807]: pgmap v47: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:17.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:17 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:17.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:17 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:17.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:17 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:17 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:17 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:17 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:18.458 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:18 smithi012 ceph-mon[29182]: pgmap v48: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:18.458 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:18 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:18.458 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:18 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:18.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:18 smithi100 ceph-mon[37807]: pgmap v48: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:18.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:18 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:18.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:18 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:19 smithi012 ceph-mon[29182]: mgrmap e33: smithi012.tmscgt(active, since 91s), standbys: smithi100.rwnrcn 2024-03-18T14:54:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:19 smithi012 ceph-mon[29182]: pgmap v49: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:19.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:19 smithi100 ceph-mon[37807]: mgrmap e33: smithi012.tmscgt(active, since 91s), standbys: smithi100.rwnrcn 2024-03-18T14:54:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:19 smithi100 ceph-mon[37807]: pgmap v49: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: pgmap v50: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.785 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:21.786 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:21 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: pgmap v50: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:22.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:21 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:23.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- 172.21.15.12:0/172878282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300feda0 msgr2=0x7fbd300faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 --2- 172.21.15.12:0/172878282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300feda0 0x7fbd300faee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd24004940 tx=0x7fbd24009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- 172.21.15.12:0/172878282 shutdown_connections 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 --2- 172.21.15.12:0/172878282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300feda0 0x7fbd300faee0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 --2- 172.21.15.12:0/172878282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300fe450 0x7fbd300fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- 172.21.15.12:0/172878282 >> 172.21.15.12:0/172878282 conn(0x7fbd300f6c10 msgr2=0x7fbd300f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- 172.21.15.12:0/172878282 shutdown_connections 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- 172.21.15.12:0/172878282 wait complete. 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 Processor -- start 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.612+0000 7fbd35141700 1 -- start start 2024-03-18T14:54:23.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd35141700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd35141700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300feda0 0x7fbd300fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd35141700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300fe060 con 0x7fbd300fe450 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd35141700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300fc650 con 0x7fbd300feda0 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60826/0 (socket says 172.21.15.12:60826) 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 -- 172.21.15.12:0/318395866 learned_addr learned my addr 172.21.15.12:0/318395866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2e59c700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300feda0 0x7fbd300fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 -- 172.21.15.12:0/318395866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300feda0 msgr2=0x7fbd300fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300feda0 0x7fbd300fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 -- 172.21.15.12:0/318395866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24005040 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd2ed9d700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c830 tx=0x7fbd28004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd2800a760 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd2800ad90 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.613+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd300fc900 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.614+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd280113a0 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.614+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd300fcec0 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.614+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3004c710 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.615+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7fbd28012400 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.615+0000 7fbd177fe700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 0x7fbd1806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.615+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fbd2808cd70 con 0x7fbd300fe450 2024-03-18T14:54:23.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.615+0000 7fbd2e59c700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 0x7fbd1806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.616+0000 7fbd2e59c700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 0x7fbd1806fd60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400b970 tx=0x7fbd24009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.620+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fbd2805a670 con 0x7fbd300fe450 2024-03-18T14:54:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.844+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd301024f0 con 0x7fbd1806d8b0 2024-03-18T14:54:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.845+0000 7fbd177fe700 1 -- 172.21.15.12:0/318395866 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fbd301024f0 con 0x7fbd1806d8b0 2024-03-18T14:54:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 msgr2=0x7fbd1806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 0x7fbd1806fd60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400b970 tx=0x7fbd24009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 msgr2=0x7fbd300fda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c830 tx=0x7fbd28004470 comp rx=0 tx=0).stop 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 shutdown_connections 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fbd1806d8b0 0x7fbd1806fd60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd300fe450 0x7fbd300fda60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 --2- 172.21.15.12:0/318395866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd300feda0 0x7fbd300fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 >> 172.21.15.12:0/318395866 conn(0x7fbd300f6c10 msgr2=0x7fbd300f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 shutdown_connections 2024-03-18T14:54:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.847+0000 7fbd35141700 1 -- 172.21.15.12:0/318395866 wait complete. 2024-03-18T14:54:23.858 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 -- 172.21.15.12:0/3454657760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 msgr2=0x7f3fac0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 --2- 172.21.15.12:0/3454657760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac0fd280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c004940 tx=0x7f3f9c009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 -- 172.21.15.12:0/3454657760 shutdown_connections 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 --2- 172.21.15.12:0/3454657760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 --2- 172.21.15.12:0/3454657760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac0fd280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 -- 172.21.15.12:0/3454657760 >> 172.21.15.12:0/3454657760 conn(0x7f3fac0f8550 msgr2=0x7f3fac0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 -- 172.21.15.12:0/3454657760 shutdown_connections 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.950+0000 7f3fb1785700 1 -- 172.21.15.12:0/3454657760 wait complete. 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 Processor -- start 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 -- start start 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac10e4b0 con 0x7f3fac0fce70 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3fb1785700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac10e620 con 0x7f3fac0fdfe0 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60838/0 (socket says 172.21.15.12:60838) 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.951+0000 7f3faaffd700 1 -- 172.21.15.12:0/1088158376 learned_addr learned my addr 172.21.15.12:0/1088158376 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faa7fc700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faaffd700 1 -- 172.21.15.12:0/1088158376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 msgr2=0x7f3fac10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faaffd700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faaffd700 1 -- 172.21.15.12:0/1088158376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c005040 con 0x7f3fac0fce70 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faa7fc700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3faaffd700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00b3b0 tx=0x7f3f9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c00e040 con 0x7f3fac0fce70 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac113020 con 0x7f3fac0fce70 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9c00bcc0 con 0x7f3fac0fce70 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c01c630 con 0x7f3fac0fce70 2024-03-18T14:54:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.952+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac1135e0 con 0x7f3fac0fce70 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.953+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac04c710 con 0x7f3fac0fce70 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.953+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f3f9c013980 con 0x7f3fac0fce70 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.954+0000 7f3f937fe700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 0x7f3f9406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.954+0000 7f3faa7fc700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 0x7f3f9406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.954+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f3f9c08da50 con 0x7f3fac0fce70 2024-03-18T14:54:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.954+0000 7f3faa7fc700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 0x7f3f9406fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009fd0 tx=0x7f3fa000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:23.959+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f3f9c05b2f0 con 0x7f3fac0fce70 2024-03-18T14:54:24.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:23 smithi012 ceph-mon[29182]: pgmap v51: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:24.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:23 smithi012 ceph-mon[29182]: Upgrade: Updating grafana.smithi012 2024-03-18T14:54:24.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:23 smithi012 ceph-mon[29182]: Deploying daemon grafana.smithi012 on smithi012 2024-03-18T14:54:24.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.189+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fac102340 con 0x7f3f9406d7e0 2024-03-18T14:54:24.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.190+0000 7f3f937fe700 1 -- 172.21.15.12:0/1088158376 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f3fac102340 con 0x7f3f9406d7e0 2024-03-18T14:54:24.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 msgr2=0x7f3f9406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 0x7f3f9406fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009fd0 tx=0x7f3fa000b040 comp rx=0 tx=0).stop 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 msgr2=0x7f3fac10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00b3b0 tx=0x7f3f9c009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 shutdown_connections 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f3f9406d7e0 0x7f3f9406fc90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fce70 0x7f3fac10d970 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 --2- 172.21.15.12:0/1088158376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fdfe0 0x7f3fac10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 >> 172.21.15.12:0/1088158376 conn(0x7f3fac0f8550 msgr2=0x7f3fac0f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 shutdown_connections 2024-03-18T14:54:24.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.192+0000 7f3fb1785700 1 -- 172.21.15.12:0/1088158376 wait complete. 2024-03-18T14:54:24.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:23 smithi100 ceph-mon[37807]: pgmap v51: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:24.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:23 smithi100 ceph-mon[37807]: Upgrade: Updating grafana.smithi012 2024-03-18T14:54:24.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:23 smithi100 ceph-mon[37807]: Deploying daemon grafana.smithi012 on smithi012 2024-03-18T14:54:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 -- 172.21.15.12:0/1249469594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 msgr2=0x7fabf40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1249469594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fd400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004940 tx=0x7fabe8009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 -- 172.21.15.12:0/1249469594 shutdown_connections 2024-03-18T14:54:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1249469594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fd400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1249469594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 0x7fabf40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 -- 172.21.15.12:0/1249469594 >> 172.21.15.12:0/1249469594 conn(0x7fabf40f7440 msgr2=0x7fabf40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 -- 172.21.15.12:0/1249469594 shutdown_connections 2024-03-18T14:54:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.296+0000 7fabf9b12700 1 -- 172.21.15.12:0/1249469594 wait complete. 2024-03-18T14:54:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 Processor -- start 2024-03-18T14:54:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 -- start start 2024-03-18T14:54:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 0x7fabf40ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40fe910 con 0x7fabf40fcf90 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf9b12700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40fea80 con 0x7fabf40fbce0 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60856/0 (socket says 172.21.15.12:60856) 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.297+0000 7fabf2ffd700 1 -- 172.21.15.12:0/1842605622 learned_addr learned my addr 172.21.15.12:0/1842605622 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf37fe700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 0x7fabf40ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf2ffd700 1 -- 172.21.15.12:0/1842605622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 msgr2=0x7fabf40ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf2ffd700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 0x7fabf40ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf2ffd700 1 -- 172.21.15.12:0/1842605622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8005040 con 0x7fabf40fcf90 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf2ffd700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabe800b3b0 tx=0x7fabe8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe800e040 con 0x7fabf40fcf90 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf40fed00 con 0x7fabf40fcf90 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe800bcc0 con 0x7fabf40fcf90 2024-03-18T14:54:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe801c630 con 0x7fabf40fcf90 2024-03-18T14:54:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.298+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4115a70 con 0x7fabf40fcf90 2024-03-18T14:54:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.299+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4101b20 con 0x7fabf40fcf90 2024-03-18T14:54:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.300+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7fabe8013980 con 0x7fabf40fcf90 2024-03-18T14:54:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.300+0000 7fabdbfff700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 0x7fabdc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.300+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fabe808d9b0 con 0x7fabf40fcf90 2024-03-18T14:54:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.300+0000 7fabf37fe700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 0x7fabdc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.301+0000 7fabf37fe700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 0x7fabdc06fc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe4006fd0 tx=0x7fabe4008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.305+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fabe80577a0 con 0x7fabf40fcf90 2024-03-18T14:54:24.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.526+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf41049c0 con 0x7fabdc06d7e0 2024-03-18T14:54:24.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.536+0000 7fabdbfff700 1 -- 172.21.15.12:0/1842605622 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fabf41049c0 con 0x7fabdc06d7e0 2024-03-18T14:54:24.536 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (14s) 8s ago 11m 15.3M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (11m) 8s ago 11m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (10m) 45s ago 10m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (2m) 8s ago 10m 79.9M - 9.4.7 954c08fa6188 e6a6e1198813 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (5m) 8s ago 5m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (5m) 8s ago 5m 152M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (5m) 45s ago 5m 15.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (5m) 45s ago 5m 12.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (111s) 8s ago 12m 498M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (65s) 45s ago 10m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (12m) 8s ago 12m 53.4M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (10m) 45s ago 10m 42.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (52s) 8s ago 11m 7235k - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:54:24.537 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (48s) 45s ago 10m 3594k - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (9m) 8s ago 9m 256M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (8m) 8s ago 8m 268M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (8m) 8s ago 8m 221M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (8m) 45s ago 8m 288M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (7m) 45s ago 7m 257M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (7m) 45s ago 7m 236M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:54:24.538 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (31s) 8s ago 10m 46.7M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:54:24.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.538+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 msgr2=0x7fabdc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.538+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 0x7fabdc06fc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe4006fd0 tx=0x7fabe4008040 comp rx=0 tx=0).stop 2024-03-18T14:54:24.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.538+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 msgr2=0x7fabf40fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.538+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabe800b3b0 tx=0x7fabe8009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 shutdown_connections 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fabdc06d7e0 0x7fabdc06fc90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf40fbce0 0x7fabf40ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 --2- 172.21.15.12:0/1842605622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf40fcf90 0x7fabf40fe2d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 >> 172.21.15.12:0/1842605622 conn(0x7fabf40f7440 msgr2=0x7fabf40f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 shutdown_connections 2024-03-18T14:54:24.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.539+0000 7fabf9b12700 1 -- 172.21.15.12:0/1842605622 wait complete. 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 -- 172.21.15.12:0/2786722180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 msgr2=0x7ff6c40ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/2786722180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c40ff9d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0004940 tx=0x7ff6c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 -- 172.21.15.12:0/2786722180 shutdown_connections 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/2786722180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c41022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/2786722180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c40ff9d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 -- 172.21.15.12:0/2786722180 >> 172.21.15.12:0/2786722180 conn(0x7ff6c40f73e0 msgr2=0x7ff6c40f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.643+0000 7ff6cc47d700 1 -- 172.21.15.12:0/2786722180 shutdown_connections 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 -- 172.21.15.12:0/2786722180 wait complete. 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 Processor -- start 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 -- start start 2024-03-18T14:54:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c410e580 con 0x7ff6c4066e10 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6cc47d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c410ed70 con 0x7ff6c40fff10 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6ca219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6ca219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60884/0 (socket says 172.21.15.12:60884) 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.644+0000 7ff6ca219700 1 -- 172.21.15.12:0/1330755923 learned_addr learned my addr 172.21.15.12:0/1330755923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6c9a18700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6ca219700 1 -- 172.21.15.12:0/1330755923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 msgr2=0x7ff6c410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6ca219700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6ca219700 1 -- 172.21.15.12:0/1330755923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0005040 con 0x7ff6c4066e10 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6c9a18700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:54:24.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6ca219700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:24.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c000e040 con 0x7ff6c4066e10 2024-03-18T14:54:24.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6c000bcc0 con 0x7ff6c4066e10 2024-03-18T14:54:24.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c410eff0 con 0x7ff6c4066e10 2024-03-18T14:54:24.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c001c630 con 0x7ff6c4066e10 2024-03-18T14:54:24.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.645+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c410f5b0 con 0x7ff6c4066e10 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.646+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c40fb060 con 0x7ff6c4066e10 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.646+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7ff6c0013980 con 0x7ff6c4066e10 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.647+0000 7ff6baffd700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 0x7ff6b006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.647+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff6c008e170 con 0x7ff6c4066e10 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.647+0000 7ff6c9a18700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 0x7ff6b006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.647+0000 7ff6c9a18700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 0x7ff6b006fc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4006fd0 tx=0x7ff6b4008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:24.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.651+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff6c005ba90 con 0x7ff6c4066e10 2024-03-18T14:54:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.943+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6c410f890 con 0x7ff6c4066e10 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.943+0000 7ff6baffd700 1 -- 172.21.15.12:0/1330755923 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff6c005b620 con 0x7ff6c4066e10 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:54:24.944 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 12, 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:54:24.945 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 msgr2=0x7ff6b006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 0x7ff6b006fc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4006fd0 tx=0x7ff6b4008040 comp rx=0 tx=0).stop 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 msgr2=0x7ff6c410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c0009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 shutdown_connections 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff6b006d7e0 0x7ff6b006fc90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6c4066e10 0x7ff6c410d9b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 --2- 172.21.15.12:0/1330755923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c40fff10 0x7ff6c410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 >> 172.21.15.12:0/1330755923 conn(0x7ff6c40f73e0 msgr2=0x7ff6c40f77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 shutdown_connections 2024-03-18T14:54:24.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:24.945+0000 7ff6cc47d700 1 -- 172.21.15.12:0/1330755923 wait complete. 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.050+0000 7fb192d2f700 1 -- 172.21.15.12:0/65648484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 msgr2=0x7fb18c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.050+0000 7fb192d2f700 1 --2- 172.21.15.12:0/65648484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c0fe3b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb178004940 tx=0x7fb178009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 -- 172.21.15.12:0/65648484 shutdown_connections 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 --2- 172.21.15.12:0/65648484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c0fe3b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 --2- 172.21.15.12:0/65648484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c0fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 -- 172.21.15.12:0/65648484 >> 172.21.15.12:0/65648484 conn(0x7fb18c0f6be0 msgr2=0x7fb18c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 -- 172.21.15.12:0/65648484 shutdown_connections 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 -- 172.21.15.12:0/65648484 wait complete. 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 Processor -- start 2024-03-18T14:54:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.051+0000 7fb192d2f700 1 -- start start 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c10db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c10e720 con 0x7fb18c0fbb50 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1130c0 con 0x7fb18c0fb200 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60892/0 (socket says 172.21.15.12:60892) 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 -- 172.21.15.12:0/2991927574 learned_addr learned my addr 172.21.15.12:0/2991927574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb190acb700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c10db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 -- 172.21.15.12:0/2991927574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 msgr2=0x7fb18c10db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c10db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 -- 172.21.15.12:0/2991927574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb178005040 con 0x7fb18c0fbb50 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb190acb700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb18bfff700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb178009150 tx=0x7fb178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17800e040 con 0x7fb18c0fbb50 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb17800bcc0 con 0x7fb18c0fbb50 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c1132c0 con 0x7fb18c0fbb50 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17801c630 con 0x7fb18c0fbb50 2024-03-18T14:54:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.052+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c113830 con 0x7fb18c0fbb50 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.053+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c04c710 con 0x7fb18c0fbb50 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.054+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7fb178013420 con 0x7fb18c0fbb50 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.054+0000 7fb1897fa700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 0x7fb17c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.054+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fb17808d900 con 0x7fb18c0fbb50 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.054+0000 7fb190acb700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 0x7fb17c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:25.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.055+0000 7fb190acb700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 0x7fb17c06fc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb180009b10 tx=0x7fb180008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.059+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fb17805b2c0 con 0x7fb18c0fbb50 2024-03-18T14:54:25.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:24 smithi012 ceph-mon[29182]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:24 smithi012 ceph-mon[29182]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:25.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:24 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1330755923' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.286+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb18c102650 con 0x7fb17c06d7e0 2024-03-18T14:54:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.287+0000 7fb1897fa700 1 -- 172.21.15.12:0/2991927574 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fb18c102650 con 0x7fb17c06d7e0 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr" 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "2/2 daemons upgraded", 2024-03-18T14:54:25.288 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading grafana daemons", 2024-03-18T14:54:25.289 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:54:25.289 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 msgr2=0x7fb17c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 0x7fb17c06fc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb180009b10 tx=0x7fb180008040 comp rx=0 tx=0).stop 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 msgr2=0x7fb18c10e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb178009150 tx=0x7fb178009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 shutdown_connections 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb17c06d7e0 0x7fb17c06fc90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb18c0fb200 0x7fb18c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 --2- 172.21.15.12:0/2991927574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb18c0fbb50 0x7fb18c10e090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 >> 172.21.15.12:0/2991927574 conn(0x7fb18c0f6be0 msgr2=0x7fb18c0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 shutdown_connections 2024-03-18T14:54:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:25.289+0000 7fb192d2f700 1 -- 172.21.15.12:0/2991927574 wait complete. 2024-03-18T14:54:25.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:24 smithi100 ceph-mon[37807]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:25.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:24 smithi100 ceph-mon[37807]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:25.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:24 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1330755923' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:26.161 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:25 smithi012 ceph-mon[29182]: pgmap v52: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:26.161 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:25 smithi012 ceph-mon[29182]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:26.161 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:25 smithi012 ceph-mon[29182]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:25 smithi100 ceph-mon[37807]: pgmap v52: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:25 smithi100 ceph-mon[37807]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:25 smithi100 ceph-mon[37807]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:28.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:27 smithi012 ceph-mon[29182]: pgmap v53: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:27 smithi100 ceph-mon[37807]: pgmap v53: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-18T14:54:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:28.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:30.104 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:29 smithi012 ceph-mon[29182]: pgmap v54: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:30.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:29 smithi100 ceph-mon[37807]: pgmap v54: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-18T14:54:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:31 smithi012 ceph-mon[29182]: pgmap v55: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:32.180 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:32.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:31 smithi100 ceph-mon[37807]: pgmap v55: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:33 smithi012 ceph-mon[29182]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:33 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:33.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:33 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:34.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:33 smithi100 ceph-mon[37807]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:33 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:33 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:35.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:34 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:35.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:34 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:35.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:34 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:35.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:34 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:36.051 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:35 smithi012 ceph-mon[29182]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:35 smithi100 ceph-mon[37807]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:38.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:38.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-18T14:54:38.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-18T14:54:38.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:37 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-18T14:54:38.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-18T14:54:38.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:38.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:54:38.300 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:54:38.300 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:54:38.300 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:37 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:38 smithi012 ceph-mon[29182]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:54:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:38 smithi012 ceph-mon[29182]: Upgrade: Finalizing container_image settings 2024-03-18T14:54:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:38 smithi012 ceph-mon[29182]: Upgrade: Complete! 2024-03-18T14:54:39.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:38 smithi100 ceph-mon[37807]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-18T14:54:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:38 smithi100 ceph-mon[37807]: Upgrade: Finalizing container_image settings 2024-03-18T14:54:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:38 smithi100 ceph-mon[37807]: Upgrade: Complete! 2024-03-18T14:54:40.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:39 smithi012 ceph-mon[29182]: pgmap v59: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:40.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:39 smithi100 ceph-mon[37807]: pgmap v59: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:41 smithi012 ceph-mon[29182]: pgmap v60: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:41 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:42.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:41 smithi100 ceph-mon[37807]: pgmap v60: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:42.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:41 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:54:44.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:43 smithi012 ceph-mon[29182]: pgmap v61: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:43 smithi100 ceph-mon[37807]: pgmap v61: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:45 smithi012 ceph-mon[29182]: pgmap v62: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:45 smithi100 ceph-mon[37807]: pgmap v62: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:47.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:46 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:46 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:54:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:47 smithi012 ceph-mon[29182]: pgmap v63: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:48.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:47 smithi100 ceph-mon[37807]: pgmap v63: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:50.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:49 smithi012 ceph-mon[29182]: pgmap v64: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:50.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:49 smithi100 ceph-mon[37807]: pgmap v64: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:52.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:51 smithi012 ceph-mon[29182]: pgmap v65: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:52.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:51 smithi100 ceph-mon[37807]: pgmap v65: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:53 smithi012 ceph-mon[29182]: pgmap v66: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:54.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:53 smithi100 ceph-mon[37807]: pgmap v66: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:54:55.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 -- 172.21.15.12:0/3903845291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 msgr2=0x7ff4a4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:55.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/3903845291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a4102330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff48c004940 tx=0x7ff48c009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:55.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 -- 172.21.15.12:0/3903845291 shutdown_connections 2024-03-18T14:54:55.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/3903845291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a4102330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/3903845291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a40ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 -- 172.21.15.12:0/3903845291 >> 172.21.15.12:0/3903845291 conn(0x7ff4a40f7440 msgr2=0x7ff4a40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 -- 172.21.15.12:0/3903845291 shutdown_connections 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.407+0000 7ff4a9a55700 1 -- 172.21.15.12:0/3903845291 wait complete. 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 Processor -- start 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 -- start start 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a410e500 con 0x7ff4a40fff10 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.408+0000 7ff4a9a55700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a410e670 con 0x7ff4a4066fd0 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33596/0 (socket says 172.21.15.12:33596) 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 -- 172.21.15.12:0/1323668110 learned_addr learned my addr 172.21.15.12:0/1323668110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:55.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a2ffd700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 -- 172.21.15.12:0/1323668110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 msgr2=0x7ff4a410d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a410d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 -- 172.21.15.12:0/1323668110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c005040 con 0x7ff4a40fff10 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a2ffd700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a27fc700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00b3b0 tx=0x7ff48c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c00e040 con 0x7ff4a40fff10 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff48c00bcc0 con 0x7ff4a40fff10 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4113020 con 0x7ff4a40fff10 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c01c630 con 0x7ff4a40fff10 2024-03-18T14:54:55.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.409+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a41135e0 con 0x7ff4a40fff10 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.410+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a404c710 con 0x7ff4a40fff10 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.411+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7ff48c013980 con 0x7ff4a40fff10 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.412+0000 7ff49b7fe700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 0x7ff49006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.412+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff48c08e1a0 con 0x7ff4a40fff10 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.412+0000 7ff4a2ffd700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 0x7ff49006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:55.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.412+0000 7ff4a2ffd700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 0x7ff49006fc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009fd0 tx=0x7ff49c009f20 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:55.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.416+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff48c05bac0 con 0x7ff4a40fff10 2024-03-18T14:54:55.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.643+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4a40f8f80 con 0x7ff49006d7e0 2024-03-18T14:54:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.644+0000 7ff49b7fe700 1 -- 172.21.15.12:0/1323668110 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff4a40f8f80 con 0x7ff49006d7e0 2024-03-18T14:54:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 msgr2=0x7ff49006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 0x7ff49006fc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009fd0 tx=0x7ff49c009f20 comp rx=0 tx=0).stop 2024-03-18T14:54:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 msgr2=0x7ff4a410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00b3b0 tx=0x7ff48c009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 shutdown_connections 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff49006d7e0 0x7ff49006fc90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a4066fd0 0x7ff4a410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 --2- 172.21.15.12:0/1323668110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a40fff10 0x7ff4a410df00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 >> 172.21.15.12:0/1323668110 conn(0x7ff4a40f7440 msgr2=0x7ff4a40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 shutdown_connections 2024-03-18T14:54:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:55.646+0000 7ff4a9a55700 1 -- 172.21.15.12:0/1323668110 wait complete. 2024-03-18T14:54:56.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:55 smithi012 ceph-mon[29182]: pgmap v67: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:56.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-18T14:54:56.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:55 smithi100 ceph-mon[37807]: pgmap v67: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:54:57.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:56 smithi012 ceph-mon[29182]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:57.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:56 smithi100 ceph-mon[37807]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:54:58.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 -- 172.21.15.12:0/1189347220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840102710 msgr2=0x7f28400fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:58.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 --2- 172.21.15.12:0/1189347220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840102710 0x7f28400fbd80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2834004940 tx=0x7f2834009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:58.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 -- 172.21.15.12:0/1189347220 shutdown_connections 2024-03-18T14:54:58.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 --2- 172.21.15.12:0/1189347220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840102710 0x7f28400fbd80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 --2- 172.21.15.12:0/1189347220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840101d70 0x7f2840102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 -- 172.21.15.12:0/1189347220 >> 172.21.15.12:0/1189347220 conn(0x7f2840072bc0 msgr2=0x7f2840072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 -- 172.21.15.12:0/1189347220 shutdown_connections 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.113+0000 7f2846440700 1 -- 172.21.15.12:0/1189347220 wait complete. 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 Processor -- start 2024-03-18T14:54:58.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 -- start start 2024-03-18T14:54:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840102710 0x7f2840116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28401168d0 con 0x7f2840101d70 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f2846440700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284010fca0 con 0x7f2840102710 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33618/0 (socket says 172.21.15.12:33618) 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 -- 172.21.15.12:0/1174669004 learned_addr learned my addr 172.21.15.12:0/1174669004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283f7fe700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840102710 0x7f2840116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 -- 172.21.15.12:0/1174669004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840102710 msgr2=0x7f2840116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840102710 0x7f2840116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.114+0000 7f283ffff700 1 -- 172.21.15.12:0/1174669004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834005040 con 0x7f2840101d70 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f283ffff700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2830002a50 tx=0x7f28300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2830007cc0 con 0x7f2840101d70 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284010ff80 con 0x7f2840101d70 2024-03-18T14:54:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f283000f040 con 0x7f2840101d70 2024-03-18T14:54:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283000da30 con 0x7f2840101d70 2024-03-18T14:54:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.115+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2840110510 con 0x7f2840101d70 2024-03-18T14:54:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.116+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28400fd4c0 con 0x7f2840101d70 2024-03-18T14:54:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.116+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f283000db90 con 0x7f2840101d70 2024-03-18T14:54:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.117+0000 7f283cff9700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 0x7f282806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:54:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.117+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f283008cb60 con 0x7f2840101d70 2024-03-18T14:54:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.117+0000 7f283f7fe700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 0x7f282806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:54:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.117+0000 7f283f7fe700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 0x7f282806fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2834004890 tx=0x7f2834009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:54:58.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.122+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f283005a400 con 0x7f2840101d70 2024-03-18T14:54:58.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:58 smithi012 ceph-mon[29182]: pgmap v68: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:58.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:58 smithi100 ceph-mon[37807]: pgmap v68: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:54:58.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.415+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f284004bed0 con 0x7f2840101d70 2024-03-18T14:54:58.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.416+0000 7f283cff9700 1 -- 172.21.15.12:0/1174669004 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f2830059f90 con 0x7f2840101d70 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 msgr2=0x7f282806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 0x7f282806fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2834004890 tx=0x7f2834009e30 comp rx=0 tx=0).stop 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 msgr2=0x7f2840115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2830002a50 tx=0x7f28300046b0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 shutdown_connections 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f282806d7e0 0x7f282806fc90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2840101d70 0x7f2840115d40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 --2- 172.21.15.12:0/1174669004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2840102710 0x7f2840116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:54:58.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 >> 172.21.15.12:0/1174669004 conn(0x7f2840072bc0 msgr2=0x7f28400659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:54:58.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 shutdown_connections 2024-03-18T14:54:58.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:54:58.418+0000 7f2846440700 1 -- 172.21.15.12:0/1174669004 wait complete. 2024-03-18T14:54:58.429 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:54:59.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-18T14:54:59.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:54:59 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1174669004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:54:59.310 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:54:59.432 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:54:59 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1174669004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:00.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:00 smithi100 ceph-mon[37807]: pgmap v69: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:00.480 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:00 smithi012 ceph-mon[29182]: pgmap v69: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 -- 172.21.15.12:0/4249795817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc101ce0 msgr2=0x7faadc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 --2- 172.21.15.12:0/4249795817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc101ce0 0x7faadc1020b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faad4009ce0 tx=0x7faad4004d10 comp rx=0 tx=0).stop 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 -- 172.21.15.12:0/4249795817 shutdown_connections 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 --2- 172.21.15.12:0/4249795817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc102680 0x7faadc0fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 --2- 172.21.15.12:0/4249795817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc101ce0 0x7faadc1020b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 -- 172.21.15.12:0/4249795817 >> 172.21.15.12:0/4249795817 conn(0x7faadc0f7870 msgr2=0x7faadc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 -- 172.21.15.12:0/4249795817 shutdown_connections 2024-03-18T14:55:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.151+0000 7faae2fac700 1 -- 172.21.15.12:0/4249795817 wait complete. 2024-03-18T14:55:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.152+0000 7faae2fac700 1 Processor -- start 2024-03-18T14:55:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.152+0000 7faae2fac700 1 -- start start 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae2fac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc101ce0 0x7faadc113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae2fac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae2fac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc114660 con 0x7faadc102680 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae2fac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc10dc80 con 0x7faadc101ce0 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58296/0 (socket says 172.21.15.12:58296) 2024-03-18T14:55:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 -- 172.21.15.12:0/3221495199 learned_addr learned my addr 172.21.15.12:0/3221495199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae0d48700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc101ce0 0x7faadc113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 -- 172.21.15.12:0/3221495199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc101ce0 msgr2=0x7faadc113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc101ce0 0x7faadc113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 -- 172.21.15.12:0/3221495199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4009940 con 0x7faadc102680 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faadbfff700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faad000c890 tx=0x7faad00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faad000a700 con 0x7faadc102680 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faad000ad30 con 0x7faadc102680 2024-03-18T14:55:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc10df60 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.153+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faad00113a0 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.154+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc10e520 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.154+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc0fd430 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.155+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7faad0012400 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.155+0000 7faad97fa700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 0x7faac406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.156+0000 7faae0d48700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 0x7faac406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.156+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7faad008ce20 con 0x7faadc102680 2024-03-18T14:55:01.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.156+0000 7faae0d48700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 0x7faac406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faad400b740 tx=0x7faad4004d10 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:01.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.160+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7faad0056c10 con 0x7faadc102680 2024-03-18T14:55:01.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.452+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faadc108de0 con 0x7faadc102680 2024-03-18T14:55:01.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.453+0000 7faad97fa700 1 -- 172.21.15.12:0/3221495199 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7faad005a2d0 con 0x7faadc102680 2024-03-18T14:55:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 msgr2=0x7faac406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 0x7faac406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faad400b740 tx=0x7faad4004d10 comp rx=0 tx=0).stop 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 msgr2=0x7faadc114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faad000c890 tx=0x7faad00044c0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 shutdown_connections 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7faac406d7e0 0x7faac406fc90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faadc101ce0 0x7faadc113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 --2- 172.21.15.12:0/3221495199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faadc102680 0x7faadc114010 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 >> 172.21.15.12:0/3221495199 conn(0x7faadc0f7870 msgr2=0x7faadc0662f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 shutdown_connections 2024-03-18T14:55:01.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:01.455+0000 7faae2fac700 1 -- 172.21.15.12:0/3221495199 wait complete. 2024-03-18T14:55:01.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)" 2024-03-18T14:55:02.123 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-18T14:55:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:02 smithi100 ceph-mon[37807]: pgmap v70: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:02 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/3221495199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:02 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:02.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:02 smithi012 ceph-mon[29182]: pgmap v70: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:02 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/3221495199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:02.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:02 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:04.101 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:04 smithi012 ceph-mon[29182]: pgmap v71: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 -- 172.21.15.12:0/1184160722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 msgr2=0x7f68040fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 --2- 172.21.15.12:0/1184160722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fe260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67f4004940 tx=0x7f67f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 -- 172.21.15.12:0/1184160722 shutdown_connections 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 --2- 172.21.15.12:0/1184160722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 0x7f6804106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 --2- 172.21.15.12:0/1184160722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fe260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 -- 172.21.15.12:0/1184160722 >> 172.21.15.12:0/1184160722 conn(0x7f6804072bc0 msgr2=0x7f6804072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 -- 172.21.15.12:0/1184160722 shutdown_connections 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.122+0000 7f6808b80700 1 -- 172.21.15.12:0/1184160722 wait complete. 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 Processor -- start 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 -- start start 2024-03-18T14:55:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 0x7f68040fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040fc8b0 con 0x7f68040fde90 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6808b80700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040fc9f0 con 0x7f68040fe830 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58312/0 (socket says 172.21.15.12:58312) 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 -- 172.21.15.12:0/1004025320 learned_addr learned my addr 172.21.15.12:0/1004025320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f6801d9b700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 0x7f68040fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 -- 172.21.15.12:0/1004025320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 msgr2=0x7f68040fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 0x7f68040fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 -- 172.21.15.12:0/1004025320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4005040 con 0x7f68040fde90 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.123+0000 7f680259c700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000c00 tx=0x7f67f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.124+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f400e040 con 0x7f68040fde90 2024-03-18T14:55:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.124+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6804119ed0 con 0x7f68040fde90 2024-03-18T14:55:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.124+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67f4013420 con 0x7f68040fde90 2024-03-18T14:55:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.124+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f401c610 con 0x7f68040fde90 2024-03-18T14:55:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.124+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680411a330 con 0x7f68040fde90 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.125+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6804065e30 con 0x7f68040fde90 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.125+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f67f4013590 con 0x7f68040fde90 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.126+0000 7f67f2ffd700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 0x7f67ec06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.126+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f67f408dfe0 con 0x7f68040fde90 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.126+0000 7f6801d9b700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 0x7f67ec06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.126+0000 7f6801d9b700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 0x7f67ec06fca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006fd0 tx=0x7f67f8008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.130+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f67f4057d50 con 0x7f68040fde90 2024-03-18T14:55:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:04 smithi100 ceph-mon[37807]: pgmap v71: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.422+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f680404bed0 con 0x7f68040fde90 2024-03-18T14:55:04.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.423+0000 7f67f2ffd700 1 -- 172.21.15.12:0/1004025320 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f67f405b410 con 0x7f68040fde90 2024-03-18T14:55:04.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 msgr2=0x7f67ec06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 0x7f67ec06fca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006fd0 tx=0x7f67f8008040 comp rx=0 tx=0).stop 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 msgr2=0x7f68040fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000c00 tx=0x7f67f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 shutdown_connections 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f67ec06d7f0 0x7f67ec06fca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68040fde90 0x7f68040fbd50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 --2- 172.21.15.12:0/1004025320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68040fe830 0x7f68040fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 >> 172.21.15.12:0/1004025320 conn(0x7f6804072bc0 msgr2=0x7f68040ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 shutdown_connections 2024-03-18T14:55:04.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:04.425+0000 7f6808b80700 1 -- 172.21.15.12:0/1004025320 wait complete. 2024-03-18T14:55:04.436 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:55:05.046 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:05 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1004025320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:05.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-18T14:55:05.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:05 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1004025320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:05.310 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:06.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:06 smithi100 ceph-mon[37807]: pgmap v72: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:06.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:06 smithi012 ceph-mon[29182]: pgmap v72: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3043409094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb0101d20 msgr2=0x7f8eb01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3043409094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb0101d20 0x7f8eb01020f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004940 tx=0x7f8ea0009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3043409094 shutdown_connections 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3043409094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb01026c0 0x7f8eb00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3043409094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb0101d20 0x7f8eb01020f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3043409094 >> 172.21.15.12:0/3043409094 conn(0x7f8eb00f7870 msgr2=0x7f8eb00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:07.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3043409094 shutdown_connections 2024-03-18T14:55:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3043409094 wait complete. 2024-03-18T14:55:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.083+0000 7f8eb5ad2700 1 Processor -- start 2024-03-18T14:55:07.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eb5ad2700 1 -- start start 2024-03-18T14:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eb5ad2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 0x7f8eb0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:07.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eb5ad2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eb5ad2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0104b60 con 0x7f8eb01026c0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eb5ad2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0103600 con 0x7f8eb0101d20 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58326/0 (socket says 172.21.15.12:58326) 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 -- 172.21.15.12:0/3241695798 learned_addr learned my addr 172.21.15.12:0/3241695798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaf7fe700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 0x7f8eb0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 -- 172.21.15.12:0/3241695798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 msgr2=0x7f8eb0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 0x7f8eb0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 -- 172.21.15.12:0/3241695798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0005040 con 0x7f8eb01026c0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaf7fe700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 0x7f8eb0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.084+0000 7f8eaeffd700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400b790 tx=0x7f8ea4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.085+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea400f6f0 con 0x7f8eb01026c0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.085+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ea400fd20 con 0x7f8eb01026c0 2024-03-18T14:55:07.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.085+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb01038e0 con 0x7f8eb01026c0 2024-03-18T14:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.085+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea40103a0 con 0x7f8eb01026c0 2024-03-18T14:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.085+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb011a200 con 0x7f8eb01026c0 2024-03-18T14:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.086+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb00fd470 con 0x7f8eb01026c0 2024-03-18T14:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.086+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f8ea4011400 con 0x7f8eb01026c0 2024-03-18T14:55:07.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.086+0000 7f8e97fff700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 0x7f8e9806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.087+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8ea408cbf0 con 0x7f8eb01026c0 2024-03-18T14:55:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.087+0000 7f8eaf7fe700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 0x7f8e9806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:07.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.087+0000 7f8eaf7fe700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 0x7f8e9806fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004560 tx=0x7f8ea0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:07.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.091+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f8ea4056960 con 0x7f8eb01026c0 2024-03-18T14:55:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:07.316+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}) v1 -- 0x7f8eb00f95e0 con 0x7f8e9806d7e0 2024-03-18T14:55:08.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:08 smithi100 ceph-mon[37807]: pgmap v73: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:08 smithi100 ceph-mon[37807]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:08.406 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:08 smithi012 ceph-mon[29182]: pgmap v73: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:08.406 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:08 smithi012 ceph-mon[29182]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.336+0000 7f8e97fff700 1 -- 172.21.15.12:0/3241695798 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f8eb00f95e0 con 0x7f8e9806d7e0 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 msgr2=0x7f8e9806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 0x7f8e9806fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004560 tx=0x7f8ea0009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 msgr2=0x7f8eb01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400b790 tx=0x7f8ea4009220 comp rx=0 tx=0).stop 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 shutdown_connections 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8e9806d7e0 0x7f8e9806fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb0101d20 0x7f8eb0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 --2- 172.21.15.12:0/3241695798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb01026c0 0x7f8eb01030c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 >> 172.21.15.12:0/3241695798 conn(0x7f8eb00f7870 msgr2=0x7f8eb00f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 shutdown_connections 2024-03-18T14:55:09.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:09.338+0000 7f8eb5ad2700 1 -- 172.21.15.12:0/3241695798 wait complete. 2024-03-18T14:55:09.350 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:55:09.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-18T14:55:10.225 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:10 smithi012 ceph-mon[29182]: pgmap v74: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:10 smithi100 ceph-mon[37807]: pgmap v74: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 -- 172.21.15.12:0/3948446748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 msgr2=0x7f15fc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 --2- 172.21.15.12:0/3948446748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc1020f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f15f4004940 tx=0x7f15f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:11.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 -- 172.21.15.12:0/3948446748 shutdown_connections 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 --2- 172.21.15.12:0/3948446748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 --2- 172.21.15.12:0/3948446748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc1020f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 -- 172.21.15.12:0/3948446748 >> 172.21.15.12:0/3948446748 conn(0x7f15fc0f7870 msgr2=0x7f15fc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 -- 172.21.15.12:0/3948446748 shutdown_connections 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 -- 172.21.15.12:0/3948446748 wait complete. 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.958+0000 7f1601723700 1 Processor -- start 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f1601723700 1 -- start start 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f1601723700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f1601723700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f1601723700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc10f600 con 0x7f15fc101d20 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f1601723700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc10f740 con 0x7f15fc1026c0 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52184/0 (socket says 172.21.15.12:52184) 2024-03-18T14:55:11.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 -- 172.21.15.12:0/4135313061 learned_addr learned my addr 172.21.15.12:0/4135313061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15fa7fc700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 -- 172.21.15.12:0/4135313061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 msgr2=0x7f15fc10f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc10f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 -- 172.21.15.12:0/4135313061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4005040 con 0x7f15fc101d20 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15fa7fc700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15faffd700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b3b0 tx=0x7f15f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.959+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f400e040 con 0x7f15fc101d20 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.960+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15f400bbc0 con 0x7f15fc101d20 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.960+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc10f9c0 con 0x7f15fc101d20 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.960+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f401c3f0 con 0x7f15fc101d20 2024-03-18T14:55:11.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.960+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc117ef0 con 0x7f15fc101d20 2024-03-18T14:55:11.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.961+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc0fd470 con 0x7f15fc101d20 2024-03-18T14:55:11.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.961+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f15f400bd30 con 0x7f15fc101d20 2024-03-18T14:55:11.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.961+0000 7f15f37fe700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 0x7f15e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:11.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.962+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f15f408dfe0 con 0x7f15fc101d20 2024-03-18T14:55:11.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.962+0000 7f15fa7fc700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 0x7f15e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:11.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.962+0000 7f15fa7fc700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 0x7f15e806fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009fd0 tx=0x7f15e400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:11.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:11.966+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f15f4057d50 con 0x7f15fc101d20 2024-03-18T14:55:12.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.181+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15fc0668a0 con 0x7f15e806d7e0 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.191+0000 7f15f37fe700 1 -- 172.21.15.12:0/4135313061 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f15fc0668a0 con 0x7f15e806d7e0 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (61s) 39s ago 12m 15.4M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (12m) 39s ago 12m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (11m) 93s ago 11m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:55:12.192 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (45s) 39s ago 11m 55.3M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (6m) 39s ago 6m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (6m) 39s ago 6m 152M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (6m) 93s ago 6m 15.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (6m) 93s ago 6m 12.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (2m) 39s ago 13m 500M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (112s) 93s ago 11m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (13m) 39s ago 13m 54.8M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (10m) 93s ago 10m 42.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (100s) 39s ago 12m 7277k - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (96s) 93s ago 11m 3594k - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (10m) 39s ago 10m 256M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (9m) 39s ago 9m 268M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (9m) 39s ago 9m 221M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (8m) 93s ago 8m 288M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:55:12.193 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (8m) 93s ago 8m 257M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:55:12.194 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (8m) 93s ago 8m 236M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:55:12.194 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (78s) 39s ago 11m 46.9M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:55:12.194 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:12 smithi012 ceph-mon[29182]: pgmap v75: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.193+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 msgr2=0x7f15e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.193+0000 7f1601723700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 0x7f15e806fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009fd0 tx=0x7f15e400b040 comp rx=0 tx=0).stop 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.193+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 msgr2=0x7f15fc10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.193+0000 7f1601723700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b3b0 tx=0x7f15f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 shutdown_connections 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f15e806d7e0 0x7f15e806fc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15fc101d20 0x7f15fc10eb80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 --2- 172.21.15.12:0/4135313061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc1026c0 0x7f15fc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 >> 172.21.15.12:0/4135313061 conn(0x7f15fc0f7870 msgr2=0x7f15fc0f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 shutdown_connections 2024-03-18T14:55:12.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:12.194+0000 7f1601723700 1 -- 172.21.15.12:0/4135313061 wait complete. 2024-03-18T14:55:12.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:12 smithi100 ceph-mon[37807]: pgmap v75: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:12.813 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-18T14:55:12.814 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:55:12.814 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-18T14:55:13.076 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:13.199 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:13 smithi012 ceph-mon[29182]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:13.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:13 smithi100 ceph-mon[37807]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:14 smithi100 ceph-mon[37807]: pgmap v76: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:14.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:14 smithi012 ceph-mon[29182]: pgmap v76: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 -- 172.21.15.12:0/3484953215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 msgr2=0x7f7c00106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 --2- 172.21.15.12:0/3484953215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c00106de0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7be8004940 tx=0x7f7be8009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 -- 172.21.15.12:0/3484953215 shutdown_connections 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 --2- 172.21.15.12:0/3484953215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c00106de0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 --2- 172.21.15.12:0/3484953215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 -- 172.21.15.12:0/3484953215 >> 172.21.15.12:0/3484953215 conn(0x7f7c000f78d0 msgr2=0x7f7c000f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 -- 172.21.15.12:0/3484953215 shutdown_connections 2024-03-18T14:55:14.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.963+0000 7f7c0570e700 1 -- 172.21.15.12:0/3484953215 wait complete. 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 Processor -- start 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 -- start start 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000fc790 con 0x7f7c000fe7a0 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7c0570e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000fc8d0 con 0x7f7c000fde00 2024-03-18T14:55:14.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52208/0 (socket says 172.21.15.12:52208) 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 -- 172.21.15.12:0/1719432039 learned_addr learned my addr 172.21.15.12:0/1719432039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 -- 172.21.15.12:0/1719432039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 msgr2=0x7f7c000fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfeffd700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 -- 172.21.15.12:0/1719432039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8005040 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfeffd700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.964+0000 7f7bfe7fc700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009380 tx=0x7f7be8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.965+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be800e040 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.965+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7be8009af0 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.965+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c00119e80 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.965+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be801c630 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.965+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0011a2e0 con 0x7f7c000fe7a0 2024-03-18T14:55:14.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.966+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c00066770 con 0x7f7c000fe7a0 2024-03-18T14:55:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.966+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f7be800bbc0 con 0x7f7c000fe7a0 2024-03-18T14:55:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.967+0000 7f7bf77fe700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 0x7f7bec06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.967+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f7be808e0a0 con 0x7f7c000fe7a0 2024-03-18T14:55:14.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.967+0000 7f7bfeffd700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 0x7f7bec06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:14.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.967+0000 7f7bfeffd700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 0x7f7bec06fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007fd0 tx=0x7f7bf00074f0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:14.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:14.971+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f7be8057e90 con 0x7f7c000fe7a0 2024-03-18T14:55:15.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.174+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f7c00109f80 con 0x7f7c000fe7a0 2024-03-18T14:55:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.178+0000 7f7bf77fe700 1 -- 172.21.15.12:0/1719432039 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7be805b550 con 0x7f7c000fe7a0 2024-03-18T14:55:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 msgr2=0x7f7bec06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 0x7f7bec06fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007fd0 tx=0x7f7bf00074f0 comp rx=0 tx=0).stop 2024-03-18T14:55:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 msgr2=0x7f7c000fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009380 tx=0x7f7be8009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 shutdown_connections 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f7bec06d7e0 0x7f7bec06fc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000fde00 0x7f7c000fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 --2- 172.21.15.12:0/1719432039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c000fe7a0 0x7f7c000fc170 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.180+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 >> 172.21.15.12:0/1719432039 conn(0x7f7c000f78d0 msgr2=0x7f7c00103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.181+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 shutdown_connections 2024-03-18T14:55:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:15.181+0000 7f7c0570e700 1 -- 172.21.15.12:0/1719432039 wait complete. 2024-03-18T14:55:15.840 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-18T14:55:15.841 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:55:15.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-18T14:55:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: pgmap v77: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/1719432039' entity='client.admin' 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:16.474 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:16 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: pgmap v77: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/1719432039' entity='client.admin' 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:16.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:16 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:17.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:17 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:17.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:17 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:17.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- 172.21.15.12:0/2642427511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 msgr2=0x7ff9180fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:17.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 --2- 172.21.15.12:0/2642427511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff9180fe150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff914004940 tx=0x7ff914009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- 172.21.15.12:0/2642427511 shutdown_connections 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 --2- 172.21.15.12:0/2642427511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 0x7ff9180fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 --2- 172.21.15.12:0/2642427511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff9180fe150 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- 172.21.15.12:0/2642427511 >> 172.21.15.12:0/2642427511 conn(0x7ff918071530 msgr2=0x7ff918071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- 172.21.15.12:0/2642427511 shutdown_connections 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- 172.21.15.12:0/2642427511 wait complete. 2024-03-18T14:55:17.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 Processor -- start 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.841+0000 7ff91fe86700 1 -- start start 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 0x7ff918115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181166c0 con 0x7ff9180fdd80 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91810fca0 con 0x7ff9180fb540 2024-03-18T14:55:17.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52236/0 (socket says 172.21.15.12:52236) 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 -- 172.21.15.12:0/1726047337 learned_addr learned my addr 172.21.15.12:0/1726047337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91dc22700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 0x7ff918115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 -- 172.21.15.12:0/1726047337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 msgr2=0x7ff918115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 0x7ff918115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 -- 172.21.15.12:0/1726047337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914005040 con 0x7ff9180fdd80 2024-03-18T14:55:17.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91d421700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff90800c830 tx=0x7ff908004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:17.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90800a760 con 0x7ff9180fdd80 2024-03-18T14:55:17.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff90800ad90 con 0x7ff9180fdd80 2024-03-18T14:55:17.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91810ff80 con 0x7ff9180fdd80 2024-03-18T14:55:17.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9080113a0 con 0x7ff9180fdd80 2024-03-18T14:55:17.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.842+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff918110540 con 0x7ff9180fdd80 2024-03-18T14:55:17.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.843+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9181073f0 con 0x7ff9180fdd80 2024-03-18T14:55:17.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.844+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7ff908012400 con 0x7ff9180fdd80 2024-03-18T14:55:17.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.844+0000 7ff90e7fc700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 0x7ff90406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:17.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.844+0000 7ff91dc22700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 0x7ff90406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:17.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.844+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff90808cde0 con 0x7ff9180fdd80 2024-03-18T14:55:17.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.845+0000 7ff91dc22700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 0x7ff90406fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff914004f90 tx=0x7ff914009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:17.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:17.849+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff90805a700 con 0x7ff9180fdd80 2024-03-18T14:55:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.052+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7ff918111180 con 0x7ff9180fdd80 2024-03-18T14:55:18.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.053+0000 7ff90e7fc700 1 -- 172.21.15.12:0/1726047337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7ff90805a290 con 0x7ff9180fdd80 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 msgr2=0x7ff90406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 0x7ff90406fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff914004f90 tx=0x7ff914009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 msgr2=0x7ff918116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff90800c830 tx=0x7ff908004470 comp rx=0 tx=0).stop 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 shutdown_connections 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7ff90406d7e0 0x7ff90406fc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180fb540 0x7ff918115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 --2- 172.21.15.12:0/1726047337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9180fdd80 0x7ff918116070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 >> 172.21.15.12:0/1726047337 conn(0x7ff918071530 msgr2=0x7ff9180f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:18.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 shutdown_connections 2024-03-18T14:55:18.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:18.055+0000 7ff91fe86700 1 -- 172.21.15.12:0/1726047337 wait complete. 2024-03-18T14:55:18.427 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:18 smithi012 ceph-mon[29182]: pgmap v78: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:18.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:18 smithi100 ceph-mon[37807]: pgmap v78: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:18.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-18T14:55:19.024 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:20.423 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:20 smithi012 ceph-mon[29182]: pgmap v79: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:20 smithi100 ceph-mon[37807]: pgmap v79: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.703+0000 7fb390483700 1 -- 172.21.15.12:0/379097252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 msgr2=0x7fb388106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.703+0000 7fb390483700 1 --2- 172.21.15.12:0/379097252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb388106ec0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb384004940 tx=0x7fb384009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 -- 172.21.15.12:0/379097252 shutdown_connections 2024-03-18T14:55:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 --2- 172.21.15.12:0/379097252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb388106ec0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 --2- 172.21.15.12:0/379097252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 0x7fb3880fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 -- 172.21.15.12:0/379097252 >> 172.21.15.12:0/379097252 conn(0x7fb3880f78f0 msgr2=0x7fb3880f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 -- 172.21.15.12:0/379097252 shutdown_connections 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 -- 172.21.15.12:0/379097252 wait complete. 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 Processor -- start 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.704+0000 7fb390483700 1 -- start start 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb390483700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 0x7fb3880fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb390483700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb390483700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880fdc60 con 0x7fb3880fe880 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb390483700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880fc790 con 0x7fb3880fdee0 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60982/0 (socket says 172.21.15.12:60982) 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 -- 172.21.15.12:0/3720753488 learned_addr learned my addr 172.21.15.12:0/3720753488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38e21f700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 0x7fb3880fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 -- 172.21.15.12:0/3720753488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 msgr2=0x7fb3880fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 0x7fb3880fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 -- 172.21.15.12:0/3720753488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384005040 con 0x7fb3880fe880 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb38da1e700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb384000c00 tx=0x7fb384009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb384021070 con 0x7fb3880fe880 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb384013c00 con 0x7fb3880fe880 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.705+0000 7fb390483700 1 -- 172.21.15.12:0/3720753488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880fca10 con 0x7fb3880fe880 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.706+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38400e070 con 0x7fb3880fe880 2024-03-18T14:55:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.706+0000 7fb390483700 1 -- 172.21.15.12:0/3720753488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388119f10 con 0x7fb3880fe880 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.707+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3700052b0 con 0x7fb3880fe880 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.708+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7fb38401c9d0 con 0x7fb3880fe880 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.708+0000 7fb37effd700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 0x7fb37406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.708+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fb38408e170 con 0x7fb3880fe880 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.708+0000 7fb38e21f700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 0x7fb37406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.708+0000 7fb38e21f700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 0x7fb37406fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb378009b80 tx=0x7fb37800b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:20.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.717+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fb384057f60 con 0x7fb3880fe880 2024-03-18T14:55:20.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.922+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fb370005560 con 0x7fb3880fe880 2024-03-18T14:55:20.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.923+0000 7fb37effd700 1 -- 172.21.15.12:0/3720753488 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb38405b620 con 0x7fb3880fe880 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 msgr2=0x7fb37406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 0x7fb37406fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb378009b80 tx=0x7fb37800b040 comp rx=0 tx=0).stop 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 msgr2=0x7fb3880fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb384000c00 tx=0x7fb384009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 shutdown_connections 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fb37406d7e0 0x7fb37406fc90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3880fdee0 0x7fb3880fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 --2- 172.21.15.12:0/3720753488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3880fe880 0x7fb3880fc250 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 >> 172.21.15.12:0/3720753488 conn(0x7fb3880f78f0 msgr2=0x7fb3881036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 shutdown_connections 2024-03-18T14:55:20.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:20.925+0000 7fb37cff9700 1 -- 172.21.15.12:0/3720753488 wait complete. 2024-03-18T14:55:21.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-18T14:55:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:22.355 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:22 smithi012 ceph-mon[29182]: pgmap v80: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:22.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:22 smithi100 ceph-mon[37807]: pgmap v80: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 -- 172.21.15.12:0/3160220362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fe830 msgr2=0x7f6bc4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3160220362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fe830 0x7f6bc4106f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004940 tx=0x7f6bac009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 -- 172.21.15.12:0/3160220362 shutdown_connections 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3160220362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fe830 0x7f6bc4106f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3160220362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fde90 0x7f6bc40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 -- 172.21.15.12:0/3160220362 >> 172.21.15.12:0/3160220362 conn(0x7f6bc4072bc0 msgr2=0x7f6bc4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 -- 172.21.15.12:0/3160220362 shutdown_connections 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 -- 172.21.15.12:0/3160220362 wait complete. 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.646+0000 7f6bc9154700 1 Processor -- start 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 -- start start 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 0x7f6bc41140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4114730 con 0x7f6bc40fde90 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc410db40 con 0x7f6bc40fe830 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60992/0 (socket says 172.21.15.12:60992) 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 -- 172.21.15.12:0/3767077388 learned_addr learned my addr 172.21.15.12:0/3767077388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc259c700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 0x7f6bc41140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 -- 172.21.15.12:0/3767077388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 msgr2=0x7f6bc41140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 0x7f6bc41140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 -- 172.21.15.12:0/3767077388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac005040 con 0x7f6bc40fde90 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc259c700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 0x7f6bc41140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc2d9d700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001d00 tx=0x7f6bb4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bb4004020 con 0x7f6bc40fde90 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bb400f040 con 0x7f6bc40fde90 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.647+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc410de20 con 0x7f6bc40fde90 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.648+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bb4009310 con 0x7f6bc40fde90 2024-03-18T14:55:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.648+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc410e360 con 0x7f6bc40fde90 2024-03-18T14:55:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.649+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc4065e30 con 0x7f6bc40fde90 2024-03-18T14:55:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.649+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f6bb4009580 con 0x7f6bc40fde90 2024-03-18T14:55:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.650+0000 7f6bbb7fe700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 0x7f6bb006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.650+0000 7f6bc259c700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 0x7f6bb006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.650+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f6bb408c760 con 0x7f6bc40fde90 2024-03-18T14:55:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.650+0000 7f6bc259c700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 0x7f6bb006fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bc410f440 tx=0x7f6bac009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:23.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.655+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f6bb40564d0 con 0x7f6bc40fde90 2024-03-18T14:55:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.859+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f6bc404bf40 con 0x7f6bc40fde90 2024-03-18T14:55:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.859+0000 7f6bbb7fe700 1 -- 172.21.15.12:0/3767077388 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f6bb4059b90 con 0x7f6bc40fde90 2024-03-18T14:55:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 msgr2=0x7f6bb006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 0x7f6bb006fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bc410f440 tx=0x7f6bac009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 msgr2=0x7f6bc4113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4001d00 tx=0x7f6bb4009f30 comp rx=0 tx=0).stop 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 shutdown_connections 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f6bb006d7e0 0x7f6bb006fc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bc40fde90 0x7f6bc4113ba0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 --2- 172.21.15.12:0/3767077388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bc40fe830 0x7f6bc41140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.861+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 >> 172.21.15.12:0/3767077388 conn(0x7f6bc4072bc0 msgr2=0x7f6bc4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.862+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 shutdown_connections 2024-03-18T14:55:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:23.862+0000 7f6bc9154700 1 -- 172.21.15.12:0/3767077388 wait complete. 2024-03-18T14:55:24.475 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:24 smithi012 ceph-mon[29182]: pgmap v81: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:24.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-18T14:55:24.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:24 smithi100 ceph-mon[37807]: pgmap v81: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:24.764 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3110604090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac102e10 msgr2=0x7f8dac1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3110604090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac102e10 0x7f8dac1031e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3110604090 shutdown_connections 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3110604090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dac1037b0 0x7f8dac0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3110604090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac102e10 0x7f8dac1031e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3110604090 >> 172.21.15.12:0/3110604090 conn(0x7f8dac0f89a0 msgr2=0x7f8dac0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3110604090 shutdown_connections 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.425+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3110604090 wait complete. 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 Processor -- start 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 -- start start 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dac102e10 0x7f8dac113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac114720 con 0x7f8dac1037b0 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8db0a9c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac10db80 con 0x7f8dac102e10 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8da9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8da9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32774/0 (socket says 172.21.15.12:32774) 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.426+0000 7f8da9d9b700 1 -- 172.21.15.12:0/3395656223 learned_addr learned my addr 172.21.15.12:0/3395656223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8da9d9b700 1 -- 172.21.15.12:0/3395656223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dac102e10 msgr2=0x7f8dac113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8da9d9b700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dac102e10 0x7f8dac113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8da9d9b700 1 -- 172.21.15.12:0/3395656223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8da9d9b700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8da0002a50 tx=0x7f8da0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da0007cc0 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac10de60 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8da000f040 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da000da40 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.427+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac10e420 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.428+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac100120 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.428+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f8da00097c0 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.429+0000 7f8d9affd700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 0x7f8d9406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.429+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8da008cbc0 con 0x7f8dac1037b0 2024-03-18T14:55:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.429+0000 7f8daa59c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 0x7f8d9406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.429+0000 7f8daa59c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 0x7f8d9406fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004f90 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.433+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f8da00568d0 con 0x7f8dac1037b0 2024-03-18T14:55:26.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:26 smithi012 ceph-mon[29182]: pgmap v82: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:26.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:26 smithi100 ceph-mon[37807]: pgmap v82: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:26.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.655+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dac109f10 con 0x7f8d9406d7e0 2024-03-18T14:55:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.660+0000 7f8d9affd700 1 -- 172.21.15.12:0/3395656223 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f8dac109f10 con 0x7f8d9406d7e0 2024-03-18T14:55:26.661 INFO:teuthology.orchestra.run.smithi012.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:55:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.662+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 msgr2=0x7f8d9406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.662+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 0x7f8d9406fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004f90 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.662+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 msgr2=0x7f8dac1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.662+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8da0002a50 tx=0x7f8da0004600 comp rx=0 tx=0).stop 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 shutdown_connections 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f8d9406d7e0 0x7f8d9406fc90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dac102e10 0x7f8dac113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 --2- 172.21.15.12:0/3395656223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8dac1037b0 0x7f8dac1140d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 >> 172.21.15.12:0/3395656223 conn(0x7f8dac0f89a0 msgr2=0x7f8dac0facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 shutdown_connections 2024-03-18T14:55:26.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:26.663+0000 7f8db0a9c700 1 -- 172.21.15.12:0/3395656223 wait complete. 2024-03-18T14:55:27.466 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-18T14:55:27.467 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T14:55:27.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-18T14:55:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T14:55:27.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: pgmap v83: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:27.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:27.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:27 smithi012 ceph-mon[29182]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: pgmap v83: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:28.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:28.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:28.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:27 smithi100 ceph-mon[37807]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T14:55:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:29 smithi012 ceph-mon[29182]: pgmap v84: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:30.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:29 smithi100 ceph-mon[37807]: pgmap v84: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 -- 172.21.15.12:0/3327273353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fe7b0 msgr2=0x7f7850106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 --2- 172.21.15.12:0/3327273353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fe7b0 0x7f7850106df0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7844004940 tx=0x7f7844009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 -- 172.21.15.12:0/3327273353 shutdown_connections 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 --2- 172.21.15.12:0/3327273353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fe7b0 0x7f7850106df0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 --2- 172.21.15.12:0/3327273353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fde10 0x7f78500fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 -- 172.21.15.12:0/3327273353 >> 172.21.15.12:0/3327273353 conn(0x7f78500f7880 msgr2=0x7f78500f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 -- 172.21.15.12:0/3327273353 shutdown_connections 2024-03-18T14:55:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.665+0000 7f78580a7700 1 -- 172.21.15.12:0/3327273353 wait complete. 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 Processor -- start 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 -- start start 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 0x7f78500fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500fc7a0 con 0x7f78500fde10 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f78580a7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500fc8e0 con 0x7f78500fe7b0 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f7855e43700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f7855e43700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55744/0 (socket says 172.21.15.12:55744) 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.666+0000 7f7855e43700 1 -- 172.21.15.12:0/1889078986 learned_addr learned my addr 172.21.15.12:0/1889078986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855642700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 0x7f78500fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855e43700 1 -- 172.21.15.12:0/1889078986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 msgr2=0x7f78500fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855e43700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 0x7f78500fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855e43700 1 -- 172.21.15.12:0/1889078986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844005040 con 0x7f78500fde10 2024-03-18T14:55:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855642700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 0x7f78500fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f7855e43700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b790 tx=0x7f784c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c00f770 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785011a1b0 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c00fda0 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c00d310 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.667+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785011a640 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.668+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850066770 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.669+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f784c010420 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.669+0000 7f78427fc700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 0x7f783c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.670+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f784c08ce50 con 0x7f78500fde10 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.670+0000 7f7855642700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 0x7f783c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.670+0000 7f7855642700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 0x7f783c06fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7844000c00 tx=0x7f7844009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:30.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.678+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f784c056ba0 con 0x7f78500fde10 2024-03-18T14:55:30.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.901+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78500fd330 con 0x7f783c06d7e0 2024-03-18T14:55:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.903+0000 7f78427fc700 1 -- 172.21.15.12:0/1889078986 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f78500fd330 con 0x7f783c06d7e0 2024-03-18T14:55:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 msgr2=0x7f783c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 0x7f783c06fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7844000c00 tx=0x7f7844009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 msgr2=0x7f78500fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b790 tx=0x7f784c009250 comp rx=0 tx=0).stop 2024-03-18T14:55:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 shutdown_connections 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f783c06d7e0 0x7f783c06fc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78500fde10 0x7f78500fbc40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 --2- 172.21.15.12:0/1889078986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78500fe7b0 0x7f78500fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 >> 172.21.15.12:0/1889078986 conn(0x7f78500f7880 msgr2=0x7f7850103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 shutdown_connections 2024-03-18T14:55:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:30.905+0000 7f78580a7700 1 -- 172.21.15.12:0/1889078986 wait complete. 2024-03-18T14:55:30.917 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:55:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 -- 172.21.15.12:0/1046999282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 msgr2=0x7f23980fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 --2- 172.21.15.12:0/1046999282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f23980fe260 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2388004940 tx=0x7f2388009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 -- 172.21.15.12:0/1046999282 shutdown_connections 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 --2- 172.21.15.12:0/1046999282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 0x7f2398106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 --2- 172.21.15.12:0/1046999282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f23980fe260 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.012+0000 7f239d253700 1 -- 172.21.15.12:0/1046999282 >> 172.21.15.12:0/1046999282 conn(0x7f2398072bc0 msgr2=0x7f2398072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 -- 172.21.15.12:0/1046999282 shutdown_connections 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 -- 172.21.15.12:0/1046999282 wait complete. 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 Processor -- start 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 -- start start 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 0x7f2398114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23981146c0 con 0x7f23980fde90 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.013+0000 7f239d253700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239810dab0 con 0x7f23980fe830 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55764/0 (socket says 172.21.15.12:55764) 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 -- 172.21.15.12:0/2963159207 learned_addr learned my addr 172.21.15.12:0/2963159207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f239659c700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 0x7f2398114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 -- 172.21.15.12:0/2963159207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 msgr2=0x7f2398114070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 0x7f2398114070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 -- 172.21.15.12:0/2963159207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388005040 con 0x7f23980fde90 2024-03-18T14:55:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f2396d9d700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2388021070 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239810dd30 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2388013c00 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f238800e070 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.014+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239810e270 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.015+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2398065e30 con 0x7f23980fde90 2024-03-18T14:55:31.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.016+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f238801c400 con 0x7f23980fde90 2024-03-18T14:55:31.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.016+0000 7f237f7fe700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 0x7f238006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.016+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f238808e180 con 0x7f23980fde90 2024-03-18T14:55:31.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.016+0000 7f239659c700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 0x7f238006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.017+0000 7f239659c700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 0x7f238006fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f239810ec70 tx=0x7f238c00b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.021+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f238805baa0 con 0x7f23980fde90 2024-03-18T14:55:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.255+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2398109f00 con 0x7f238006d7e0 2024-03-18T14:55:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.256+0000 7f237f7fe700 1 -- 172.21.15.12:0/2963159207 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2398109f00 con 0x7f238006d7e0 2024-03-18T14:55:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.258+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 msgr2=0x7f238006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.258+0000 7f239d253700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 0x7f238006fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f239810ec70 tx=0x7f238c00b040 comp rx=0 tx=0).stop 2024-03-18T14:55:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.258+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 msgr2=0x7f2398113b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.258+0000 7f239d253700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c00 tx=0x7f2388009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 shutdown_connections 2024-03-18T14:55:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f238006d7e0 0x7f238006fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23980fde90 0x7f2398113b30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 --2- 172.21.15.12:0/2963159207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23980fe830 0x7f2398114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 >> 172.21.15.12:0/2963159207 conn(0x7f2398072bc0 msgr2=0x7f23981037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 shutdown_connections 2024-03-18T14:55:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.259+0000 7f239d253700 1 -- 172.21.15.12:0/2963159207 wait complete. 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 -- 172.21.15.12:0/895377467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b81026c0 msgr2=0x7f85b80fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 --2- 172.21.15.12:0/895377467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b81026c0 0x7f85b80fc530 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f85a8004940 tx=0x7f85a8009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 -- 172.21.15.12:0/895377467 shutdown_connections 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 --2- 172.21.15.12:0/895377467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b81026c0 0x7f85b80fc530 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 --2- 172.21.15.12:0/895377467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b8101d20 0x7f85b81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 -- 172.21.15.12:0/895377467 >> 172.21.15.12:0/895377467 conn(0x7f85b8072380 msgr2=0x7f85b8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 -- 172.21.15.12:0/895377467 shutdown_connections 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.366+0000 7f85c07d6700 1 -- 172.21.15.12:0/895377467 wait complete. 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 Processor -- start 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 -- start start 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 0x7f85b8113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81145a0 con 0x7f85b8101d20 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85c07d6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b810dbf0 con 0x7f85b81026c0 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85be572700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85be572700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55774/0 (socket says 172.21.15.12:55774) 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85be572700 1 -- 172.21.15.12:0/3518136208 learned_addr learned my addr 172.21.15.12:0/3518136208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85bdd71700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 0x7f85b8113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85be572700 1 -- 172.21.15.12:0/3518136208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 msgr2=0x7f85b8113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.367+0000 7f85be572700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 0x7f85b8113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85be572700 1 -- 172.21.15.12:0/3518136208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8005040 con 0x7f85b8101d20 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85bdd71700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 0x7f85b8113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85be572700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85b4002a50 tx=0x7f85b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b4007cc0 con 0x7f85b8101d20 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b810ded0 con 0x7f85b8101d20 2024-03-18T14:55:31.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85b400f040 con 0x7f85b8101d20 2024-03-18T14:55:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b4009640 con 0x7f85b8101d20 2024-03-18T14:55:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.368+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b810e410 con 0x7f85b8101d20 2024-03-18T14:55:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.369+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b8105200 con 0x7f85b8101d20 2024-03-18T14:55:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.370+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f85b40097a0 con 0x7f85b8101d20 2024-03-18T14:55:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.371+0000 7f85aeffd700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 0x7f85a406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.371+0000 7f85bdd71700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 0x7f85a406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.371+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f85b408d1c0 con 0x7f85b8101d20 2024-03-18T14:55:31.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.371+0000 7f85bdd71700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 0x7f85a406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85b810f240 tx=0x7f85a8009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.376+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f85b405aae0 con 0x7f85b8101d20 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (squid) 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: Upgrade: Setting container_image for all mgr 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:31.438 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-18T14:55:31.439 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: pgmap v85: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:31.439 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:31.439 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:31 smithi012 ceph-mon[29182]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:31.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.597+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f85b810eff0 con 0x7f85a406d7e0 2024-03-18T14:55:31.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.607+0000 7f85aeffd700 1 -- 172.21.15.12:0/3518136208 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f85b810eff0 con 0x7f85a406d7e0 2024-03-18T14:55:31.608 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:55:31.608 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (81s) 58s ago 12m 15.4M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (12m) 58s ago 12m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (11m) 112s ago 11m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (64s) 58s ago 12m 55.3M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (6m) 58s ago 6m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (6m) 58s ago 6m 152M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (6m) 112s ago 6m 15.6M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (6m) 112s ago 6m 12.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (2m) 58s ago 13m 500M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (2m) 112s ago 11m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:55:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (13m) 58s ago 14m 54.8M 2048M 17.2.7-684-g6716c9f3 19857db3290c e15261801aa4 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (11m) 112s ago 11m 42.2M 2048M 17.2.7-684-g6716c9f3 19857db3290c ba097be81c80 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (2m) 58s ago 12m 7277k - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (115s) 112s ago 11m 3594k - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (10m) 58s ago 10m 256M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (10m) 58s ago 10m 268M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (9m) 58s ago 9m 221M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (9m) 112s ago 9m 288M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (8m) 112s ago 8m 257M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (8m) 112s ago 8m 236M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:55:31.610 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (98s) 58s ago 11m 46.9M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 msgr2=0x7f85a406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 0x7f85a406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85b810f240 tx=0x7f85a8009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 msgr2=0x7f85b8113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85b4002a50 tx=0x7f85b40046b0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 shutdown_connections 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f85a406d7e0 0x7f85a406fc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85b8101d20 0x7f85b8113a10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 --2- 172.21.15.12:0/3518136208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b81026c0 0x7f85b8113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 >> 172.21.15.12:0/3518136208 conn(0x7f85b8072380 msgr2=0x7f85b80fb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 shutdown_connections 2024-03-18T14:55:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.610+0000 7f85c07d6700 1 -- 172.21.15.12:0/3518136208 wait complete. 2024-03-18T14:55:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 -- 172.21.15.12:0/2889046286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 msgr2=0x7fe5fc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 --2- 172.21.15.12:0/2889046286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc1031e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec004940 tx=0x7fe5ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 -- 172.21.15.12:0/2889046286 shutdown_connections 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 --2- 172.21.15.12:0/2889046286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 --2- 172.21.15.12:0/2889046286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc1031e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 -- 172.21.15.12:0/2889046286 >> 172.21.15.12:0/2889046286 conn(0x7fe5fc0f89a0 msgr2=0x7fe5fc0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 -- 172.21.15.12:0/2889046286 shutdown_connections 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.713+0000 7fe601dc5700 1 -- 172.21.15.12:0/2889046286 wait complete. 2024-03-18T14:55:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 Processor -- start 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 -- start start 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1147b0 con 0x7fe5fc102e10 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.714+0000 7fe601dc5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc10db10 con 0x7fe5fc1037b0 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5faffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5fb7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55790/0 (socket says 172.21.15.12:55790) 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5fb7fe700 1 -- 172.21.15.12:0/554444782 learned_addr learned my addr 172.21.15.12:0/554444782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5faffd700 1 -- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 msgr2=0x7fe5fc113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5faffd700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5faffd700 1 -- 172.21.15.12:0/554444782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec005040 con 0x7fe5fc1037b0 2024-03-18T14:55:31.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.715+0000 7fe5faffd700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc1140d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b550 tx=0x7fe5e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.716+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e400f6f0 con 0x7fe5fc1037b0 2024-03-18T14:55:31.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.716+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5e400fd20 con 0x7fe5fc1037b0 2024-03-18T14:55:31.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.716+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc10dd90 con 0x7fe5fc1037b0 2024-03-18T14:55:31.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.716+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc10e350 con 0x7fe5fc1037b0 2024-03-18T14:55:31.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.716+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e4011e90 con 0x7fe5fc1037b0 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.717+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc0fe560 con 0x7fe5fc1037b0 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.717+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7fe5e400f850 con 0x7fe5fc1037b0 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.718+0000 7fe5f3fff700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 0x7fe5e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.718+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fe5e408cb90 con 0x7fe5fc1037b0 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.718+0000 7fe5fb7fe700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 0x7fe5e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:31.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.718+0000 7fe5fb7fe700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 0x7fe5e806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec004f90 tx=0x7fe5ec009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:31.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:31.723+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe5e405a450 con 0x7fe5fc1037b0 2024-03-18T14:55:31.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:31.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (squid) 2024-03-18T14:55:31.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-18T14:55:31.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:31.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: Upgrade: Setting container_image for all mgr 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: pgmap v85: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:31.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:31 smithi100 ceph-mon[37807]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:32.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.014+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5fc109f80 con 0x7fe5fc1037b0 2024-03-18T14:55:32.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.015+0000 7fe5f3fff700 1 -- 172.21.15.12:0/554444782 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe5e4059fe0 con 0x7fe5fc1037b0 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:55:32.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:55:32.017 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:55:32.017 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 12, 2024-03-18T14:55:32.017 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:55:32.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:55:32.017 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:55:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 msgr2=0x7fe5e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 0x7fe5e806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec004f90 tx=0x7fe5ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 msgr2=0x7fe5fc1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc1140d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b550 tx=0x7fe5e40091a0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 shutdown_connections 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7fe5e806d7e0 0x7fe5e806fc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5fc102e10 0x7fe5fc113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 --2- 172.21.15.12:0/554444782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5fc1037b0 0x7fe5fc1140d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.017+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 >> 172.21.15.12:0/554444782 conn(0x7fe5fc0f89a0 msgr2=0x7fe5fc0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.018+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 shutdown_connections 2024-03-18T14:55:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.018+0000 7fe601dc5700 1 -- 172.21.15.12:0/554444782 wait complete. 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 -- 172.21.15.12:0/3186524456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e81037b0 msgr2=0x7f76e80fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/3186524456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e81037b0 0x7f76e80fce20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f76dc004940 tx=0x7f76dc009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 -- 172.21.15.12:0/3186524456 shutdown_connections 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/3186524456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e81037b0 0x7f76e80fce20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/3186524456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8102e10 0x7f76e81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 -- 172.21.15.12:0/3186524456 >> 172.21.15.12:0/3186524456 conn(0x7f76e80f8980 msgr2=0x7f76e80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 -- 172.21.15.12:0/3186524456 shutdown_connections 2024-03-18T14:55:32.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.123+0000 7f76ee0a9700 1 -- 172.21.15.12:0/3186524456 wait complete. 2024-03-18T14:55:32.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 Processor -- start 2024-03-18T14:55:32.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 -- start start 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e81037b0 0x7f76e81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8114740 con 0x7f76e8102e10 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76ee0a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e810db80 con 0x7f76e81037b0 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76e77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55810/0 (socket says 172.21.15.12:55810) 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.124+0000 7f76e77fe700 1 -- 172.21.15.12:0/506401760 learned_addr learned my addr 172.21.15.12:0/506401760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76e6ffd700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e81037b0 0x7f76e81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76e77fe700 1 -- 172.21.15.12:0/506401760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e81037b0 msgr2=0x7f76e81140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76e77fe700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e81037b0 0x7f76e81140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76e77fe700 1 -- 172.21.15.12:0/506401760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc005040 con 0x7f76e8102e10 2024-03-18T14:55:32.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76e77fe700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c830 tx=0x7f76d8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d800a760 con 0x7f76e8102e10 2024-03-18T14:55:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76d800ad90 con 0x7f76e8102e10 2024-03-18T14:55:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e810de60 con 0x7f76e8102e10 2024-03-18T14:55:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d80113a0 con 0x7f76e8102e10 2024-03-18T14:55:32.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.125+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e810e420 con 0x7f76e8102e10 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.126+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e81052e0 con 0x7f76e8102e10 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.127+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f76d8012400 con 0x7f76e8102e10 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.127+0000 7f76cffff700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 0x7f76d006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.127+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f76d808cde0 con 0x7f76e8102e10 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.127+0000 7f76e6ffd700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 0x7f76d006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.128+0000 7f76e6ffd700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 0x7f76d006fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76e810ecf0 tx=0x7f76dc009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.132+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f76d805a700 con 0x7f76e8102e10 2024-03-18T14:55:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.397+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76e810f200 con 0x7f76e8102e10 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.397+0000 7f76cffff700 1 -- 172.21.15.12:0/506401760 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f76d805a290 con 0x7f76e8102e10 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:55:32.398 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:55:32.399 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:55:32.400 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:55:32.401 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:55:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:55:32.402 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:32.402 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 msgr2=0x7f76d006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 0x7f76d006fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76e810ecf0 tx=0x7f76dc009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 msgr2=0x7f76e8113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c830 tx=0x7f76d8004470 comp rx=0 tx=0).stop 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 shutdown_connections 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f76d006d7e0 0x7f76d006fc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8102e10 0x7f76e8113bb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 --2- 172.21.15.12:0/506401760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e81037b0 0x7f76e81140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 >> 172.21.15.12:0/506401760 conn(0x7f76e80f8980 msgr2=0x7f76e8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 shutdown_connections 2024-03-18T14:55:32.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.400+0000 7f76ee0a9700 1 -- 172.21.15.12:0/506401760 wait complete. 2024-03-18T14:55:32.404 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:55:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 -- 172.21.15.12:0/1222930309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f0102e10 msgr2=0x7f65f01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 --2- 172.21.15.12:0/1222930309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f0102e10 0x7f65f01031e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65ec004940 tx=0x7f65ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 -- 172.21.15.12:0/1222930309 shutdown_connections 2024-03-18T14:55:32.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 --2- 172.21.15.12:0/1222930309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f01037b0 0x7f65f00fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 --2- 172.21.15.12:0/1222930309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f0102e10 0x7f65f01031e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 -- 172.21.15.12:0/1222930309 >> 172.21.15.12:0/1222930309 conn(0x7f65f00f8980 msgr2=0x7f65f00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 -- 172.21.15.12:0/1222930309 shutdown_connections 2024-03-18T14:55:32.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.507+0000 7f65f7948700 1 -- 172.21.15.12:0/1222930309 wait complete. 2024-03-18T14:55:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 Processor -- start 2024-03-18T14:55:32.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 -- start start 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f0102e10 0x7f65f0113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0114730 con 0x7f65f01037b0 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.508+0000 7f65f7948700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f010db40 con 0x7f65f0102e10 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55828/0 (socket says 172.21.15.12:55828) 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 -- 172.21.15.12:0/1760800678 learned_addr learned my addr 172.21.15.12:0/1760800678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f56e4700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f0102e10 0x7f65f0113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 -- 172.21.15.12:0/1760800678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f0102e10 msgr2=0x7f65f0113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f0102e10 0x7f65f0113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 -- 172.21.15.12:0/1760800678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec005040 con 0x7f65f01037b0 2024-03-18T14:55:32.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f4ee3700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f65e0002a50 tx=0x7f65e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65e0007cc0 con 0x7f65f01037b0 2024-03-18T14:55:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f010de20 con 0x7f65f01037b0 2024-03-18T14:55:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65e000f040 con 0x7f65f01037b0 2024-03-18T14:55:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.509+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65e000d9c0 con 0x7f65f01037b0 2024-03-18T14:55:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.510+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f010e3e0 con 0x7f65f01037b0 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.510+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f0100120 con 0x7f65f01037b0 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.511+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f65e0009730 con 0x7f65f01037b0 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.512+0000 7f65e5ffb700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 0x7f65dc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.512+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f65e008cae0 con 0x7f65f01037b0 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.512+0000 7f65f56e4700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 0x7f65dc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.512+0000 7f65f56e4700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 0x7f65dc06fc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65ec004f90 tx=0x7f65ec009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.521+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f65e00567b0 con 0x7f65f01037b0 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/554444782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: Upgrade: Updating mon.smithi012 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:55:32.714 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:32.715 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: Deploying daemon mon.smithi012 on smithi012 2024-03-18T14:55:32.715 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:32 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/506401760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:55:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.749+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 --> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65f010f290 con 0x7f65dc06d7e0 2024-03-18T14:55:32.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.751+0000 7f65e5ffb700 1 -- 172.21.15.12:0/1760800678 <== mgr.14686 v2:172.21.15.12:6800/3117137762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f65f010f290 con 0x7f65dc06d7e0 2024-03-18T14:55:32.752 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:55:32.752 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:55:32.752 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr" 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "2/21 daemons upgraded", 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mon daemons", 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:55:32.753 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 msgr2=0x7f65dc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 0x7f65dc06fc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65ec004f90 tx=0x7f65ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 msgr2=0x7f65f01140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f65e0002a50 tx=0x7f65e0004600 comp rx=0 tx=0).stop 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 shutdown_connections 2024-03-18T14:55:32.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f65dc06d7e0 0x7f65dc06fc90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f0102e10 0x7f65f0113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 --2- 172.21.15.12:0/1760800678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65f01037b0 0x7f65f01140e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.753+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 >> 172.21.15.12:0/1760800678 conn(0x7f65f00f8980 msgr2=0x7f65f0107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.754+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 shutdown_connections 2024-03-18T14:55:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.754+0000 7f65f7948700 1 -- 172.21.15.12:0/1760800678 wait complete. 2024-03-18T14:55:32.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:32.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:32.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/554444782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: Upgrade: Updating mon.smithi012 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:32.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: Deploying daemon mon.smithi012 on smithi012 2024-03-18T14:55:32.797 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:32 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/506401760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:55:32.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.856+0000 7f46952f0700 1 -- 172.21.15.12:0/1963378082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690101d70 msgr2=0x7f4690102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.856+0000 7f46952f0700 1 --2- 172.21.15.12:0/1963378082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690101d70 0x7f4690102140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ce0 tx=0x7f4680004d10 comp rx=0 tx=0).stop 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- 172.21.15.12:0/1963378082 shutdown_connections 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 --2- 172.21.15.12:0/1963378082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690102710 0x7f46900fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 --2- 172.21.15.12:0/1963378082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690101d70 0x7f4690102140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- 172.21.15.12:0/1963378082 >> 172.21.15.12:0/1963378082 conn(0x7f4690072bc0 msgr2=0x7f4690072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- 172.21.15.12:0/1963378082 shutdown_connections 2024-03-18T14:55:32.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- 172.21.15.12:0/1963378082 wait complete. 2024-03-18T14:55:32.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 Processor -- start 2024-03-18T14:55:32.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- start start 2024-03-18T14:55:32.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690101d70 0x7f4690113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.857+0000 7f46952f0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4690114910 con 0x7f4690102710 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46952f0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469010dc50 con 0x7f4690101d70 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55838/0 (socket says 172.21.15.12:55838) 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 -- 172.21.15.12:0/4141675249 learned_addr learned my addr 172.21.15.12:0/4141675249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468effd700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690101d70 0x7f4690113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 -- 172.21.15.12:0/4141675249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690101d70 msgr2=0x7f4690113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690101d70 0x7f4690113cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 -- 172.21.15.12:0/4141675249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680009940 con 0x7f4690102710 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f468e7fc700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f467800b790 tx=0x7f4678009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f467800f6f0 con 0x7f4690102710 2024-03-18T14:55:32.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469010ded0 con 0x7f4690102710 2024-03-18T14:55:32.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f467800fd20 con 0x7f4690102710 2024-03-18T14:55:32.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46780103a0 con 0x7f4690102710 2024-03-18T14:55:32.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.858+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469010e490 con 0x7f4690102710 2024-03-18T14:55:32.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.859+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46900fd4c0 con 0x7f4690102710 2024-03-18T14:55:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.860+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 33) v1 ==== 91555+0+0 (secure 0 0 0) 0x7f4678011400 con 0x7f4690102710 2024-03-18T14:55:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.860+0000 7f46877fe700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 0x7f467c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:55:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.860+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f467808cc10 con 0x7f4690102710 2024-03-18T14:55:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.860+0000 7f468effd700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 0x7f467c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:55:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.861+0000 7f468effd700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 0x7f467c06fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f468000b740 tx=0x7f4680004d10 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:55:32.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:32.865+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f467805a4b0 con 0x7f4690102710 2024-03-18T14:55:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.165+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f469004bed0 con 0x7f4690102710 2024-03-18T14:55:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.165+0000 7f46877fe700 1 -- 172.21.15.12:0/4141675249 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f467805a040 con 0x7f4690102710 2024-03-18T14:55:33.166 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 msgr2=0x7f467c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 0x7f467c06fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f468000b740 tx=0x7f4680004d10 comp rx=0 tx=0).stop 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 msgr2=0x7f4690114230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f467800b790 tx=0x7f4678009220 comp rx=0 tx=0).stop 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 shutdown_connections 2024-03-18T14:55:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:6800/3117137762,v1:172.21.15.12:6801/3117137762] conn(0x7f467c06d7e0 0x7f467c06fc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:33.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4690101d70 0x7f4690113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:33.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 --2- 172.21.15.12:0/4141675249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4690102710 0x7f4690114230 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:55:33.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 >> 172.21.15.12:0/4141675249 conn(0x7f4690072bc0 msgr2=0x7f4690065910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:55:33.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 shutdown_connections 2024-03-18T14:55:33.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:55:33.167+0000 7f46952f0700 1 -- 172.21.15.12:0/4141675249 wait complete. 2024-03-18T14:55:34.101 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:33 smithi012 ceph-mon[29182]: pgmap v86: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:34.101 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:33 smithi012 ceph-mon[29182]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:34.102 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:33 smithi012 ceph-mon[29182]: from='client.? 172.21.15.12:0/4141675249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:55:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:33 smithi100 ceph-mon[37807]: pgmap v86: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:55:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:33 smithi100 ceph-mon[37807]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:55:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:33 smithi100 ceph-mon[37807]: from='client.? 172.21.15.12:0/4141675249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:55:34.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 systemd[1]: Stopping Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:55:34.717 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[29178]: 2024-03-18T14:55:34.373+0000 7fcec0443700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi012 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:55:34.717 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[29178]: 2024-03-18T14:55:34.373+0000 7fcec0443700 -1 mon.smithi012@0(leader) e2 *** Got Signal Terminated *** 2024-03-18T14:55:34.717 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 podman[111725]: 2024-03-18 14:55:34.435126488 +0000 UTC m=+0.188901498 container died e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=quincy-6716c9f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T14:55:34.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 podman[111725]: 2024-03-18 14:55:34.83287416 +0000 UTC m=+0.586649204 container cleanup e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-6716c9f, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:55:34.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:34 smithi012 bash[111725]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012 2024-03-18T14:55:35.424 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 podman[111750]: 2024-03-18 14:55:35.0674098 +0000 UTC m=+0.633402548 container remove e15261801aa4648970679e84084e0608972ebc196cded2ee46451c8a6750ab6f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_CLEAN=True, RELEASE=quincy-6716c9f, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T14:55:35.424 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service: Deactivated successfully. 2024-03-18T14:55:35.424 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 systemd[1]: Stopped Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:55:35.424 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service: Consumed 13.409s CPU time. 2024-03-18T14:55:35.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 systemd[1]: Starting Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:55:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 podman[111884]: 2024-03-18 14:55:35.543216062 +0000 UTC m=+0.018973758 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:55:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 podman[111884]: 2024-03-18 14:55:35.64375439 +0000 UTC m=+0.119512123 container create 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T14:55:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 podman[111884]: 2024-03-18 14:55:35.821423133 +0000 UTC m=+0.297180828 container init 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-18T14:55:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 podman[111884]: 2024-03-18 14:55:35.824397045 +0000 UTC m=+0.300154733 container start 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:55:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T14:55:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-18T14:55:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: pidfile_write: ignore empty --pid-file 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: load: jerasure load: lrc 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: RocksDB version: 7.9.2 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Git sha 0 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: DB SUMMARY 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: DB Session ID: T3OZI1VU5AN9VRUEK7TL 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: CURRENT file: CURRENT 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1167 Bytes 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi012/store.db dir, Total Num: 1, files: 000033.sst 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi012/store.db: 000031.log size: 4307113 ; 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.error_if_exists: 0 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.create_if_missing: 0 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.paranoid_checks: 1 2024-03-18T14:55:36.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.env: 0x5581582cbc60 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.info_log: 0x558159a43980 2024-03-18T14:55:36.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.statistics: (nil) 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.use_fsync: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_log_file_size: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_fallocate: 1 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T14:55:36.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.use_direct_reads: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.db_log_dir: 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.wal_dir: 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.write_buffer_manager: 0x558159a161e0 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T14:55:36.247 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.unordered_write: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.row_cache: None 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.wal_filter: None 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T14:55:36.248 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.two_write_queues: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.wal_compression: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.atomic_flush: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.log_readahead_size: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T14:55:36.249 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_background_jobs: 2 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_background_compactions: -1 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_subcompactions: 1 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_open_files: -1 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T14:55:36.250 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_background_flushes: -1 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Compression algorithms supported: 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kZSTD supported: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kXpressCompression supported: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kZlibCompression supported: 1 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kSnappyCompression supported: 1 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kLZ4Compression supported: 1 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: kBZip2Compression supported: 0 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T14:55:36.251 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi012/store.db/MANIFEST-000015 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.merge_operator: 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_filter: None 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558159a43a80) 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_type: 0 2024-03-18T14:55:36.252 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: data_block_index_type: 0 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_shortening: 1 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: checksum: 4 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: no_block_cache: 0 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache: 0x558159a49090 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_options: 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: capacity : 536870912 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: num_shard_bits : 4 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: strict_capacity_limit : 0 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_cache_compressed: (nil) 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: persistent_cache: (nil) 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_size: 4096 2024-03-18T14:55:36.253 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_size_deviation: 10 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_restart_interval: 16 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: index_block_restart_interval: 1 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: metadata_block_size: 4096 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: partition_filters: 0 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: use_delta_encoding: 1 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: filter_policy: bloomfilter 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: whole_key_filtering: 1 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: verify_compression: 0 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: format_version: 5 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: enable_index_compression: 1 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: block_align: 0 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: max_auto_readahead_size: 262144 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: prepopulate_block_cache: 0 2024-03-18T14:55:36.254 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: initial_auto_readahead_size: 8192 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression: NoCompression 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.num_levels: 7 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T14:55:36.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T14:55:36.256 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T14:55:36.257 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T14:55:36.258 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.table_properties_collectors: 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.inplace_update_support: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.bloom_locality: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.max_successive_merges: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T14:55:36.259 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.ttl: 2592000 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enable_blob_files: false 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.min_blob_size: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T14:55:36.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi012/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12018, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2aa264f1-b5b4-4df3-8f2c-581da17794f1 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773735853912, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773735867984, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 3819896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12012, "largest_seqno": 15216, "table_properties": {"data_size": 3807493, "index_size": 7670, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3717, "raw_key_size": 32528, "raw_average_key_size": 22, "raw_value_size": 3778901, "raw_average_value_size": 2579, "num_data_blocks": 347, "num_entries": 1465, "num_filter_entries": 1465, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710773735, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2aa264f1-b5b4-4df3-8f2c-581da17794f1", "db_session_id": "T3OZI1VU5AN9VRUEK7TL", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773735868092, "job": 1, "event": "recovery_finished"} 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi012/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558159b3c000 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: DB pointer 0x558159b26000 2024-03-18T14:55:36.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** DB Stats ** 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: L0 1/0 3.64 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-03-18T14:55:36.262 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: L6 1/0 5.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Sum 2/0 8.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 499.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** Compaction Stats [default] ** 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 499.9 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:55:36.263 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Cumulative compaction: 0.00 GB write, 219.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Interval compaction: 0.00 GB write, 219.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Block cache BinnedLRUCache@0x558159a49090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: starting mon.smithi012 rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi012 fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???) e2 preinit fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).mds e15 new map 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).mds e15 print_map 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: e15 2024-03-18T14:55:36.264 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: legacy client fscid: 1 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Filesystem 'cephfs' (1) 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: fs_name cephfs 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: epoch 15 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: tableserver 0 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: root 0 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: session_timeout 60 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: session_autoclose 300 2024-03-18T14:55:36.265 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: max_file_size 1099511627776 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: max_xattr_size 65536 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: required_client_features {} 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: last_failure 0 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: last_failure_osd_epoch 0 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: max_mds 1 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: in 0 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: up {0=14476} 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: failed 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: damaged 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: stopped 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: data_pools [3] 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: metadata_pool 2 2024-03-18T14:55:36.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: inline_data disabled 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: balancer 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: bal_rank_mask -1 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: standby_count_wanted 1 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: qdb_cluster leader: 0 members: 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: [mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: [mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: Standby daemons: 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: [mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout: [mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-18T14:55:36.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:55:36.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:55:36.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:55:36.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 ceph-mon[111904]: mon.smithi012@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-18T14:55:36.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 bash[111884]: 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 2024-03-18T14:55:36.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:35 smithi012 systemd[1]: Started Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: pgmap v87: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: pgmap v88: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: mon.smithi012 calling monitor election 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: monmap epoch 2 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: last_changed 2024-03-18T14:44:17.192582+0000 2024-03-18T14:55:39.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: created 2024-03-18T14:41:22.518337+0000 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: min_mon_release 17 (quincy) 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: election_strategy: 1 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.smithi012 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: 1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.smithi100 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: osdmap e47: 6 total, 6 up, 6 in 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: mgrmap e33: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: overall HEALTH_OK 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: from='mgr.14686 ' entity='' 2024-03-18T14:55:39.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:38 smithi012 ceph-mon[111904]: mgrmap e34: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: from='mgr.14686 172.21.15.12:0/1794433993' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: pgmap v87: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: pgmap v88: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: mon.smithi012 calling monitor election 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: monmap epoch 2 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:55:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: last_changed 2024-03-18T14:44:17.192582+0000 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: created 2024-03-18T14:41:22.518337+0000 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: min_mon_release 17 (quincy) 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: election_strategy: 1 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.smithi012 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: 1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.smithi100 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: osdmap e47: 6 total, 6 up, 6 in 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: mgrmap e33: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: overall HEALTH_OK 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: from='mgr.14686 ' entity='' 2024-03-18T14:55:39.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:38 smithi100 ceph-mon[37807]: mgrmap e34: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:55:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:45 smithi012 ceph-mon[111904]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn restarted 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: Standby manager daemon smithi100.rwnrcn started 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/crt"}]: dispatch 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi100.rwnrcn/key"}]: dispatch 2024-03-18T14:55:46.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:45 smithi100 ceph-mon[37807]: from='mgr.? 172.21.15.100:0/2824419965' entity='mgr.smithi100.rwnrcn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T14:55:47.216 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: mgrmap e35: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: Active manager daemon smithi012.tmscgt restarted 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: osdmap e48: 6 total, 6 up, 6 in 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: mgrmap e36: smithi012.tmscgt(active, starting, since 0.00648751s), standbys: smithi100.rwnrcn 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:55:47.217 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:55:47.218 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:55:47.219 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:55:47.220 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:55:47.220 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:47.220 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:47.220 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:47.221 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:55:47.221 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:46 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:55:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: mgrmap e35: smithi012.tmscgt(active, since 2m), standbys: smithi100.rwnrcn 2024-03-18T14:55:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: Active manager daemon smithi012.tmscgt restarted 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: Activating manager daemon smithi012.tmscgt 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: osdmap e48: 6 total, 6 up, 6 in 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: mgrmap e36: smithi012.tmscgt(active, starting, since 0.00648751s), standbys: smithi100.rwnrcn 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T14:55:47.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi012.tmscgt", "id": "smithi012.tmscgt"}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr metadata", "who": "smithi100.rwnrcn", "id": "smithi100.rwnrcn"}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:55:47.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: Manager daemon smithi012.tmscgt is now available 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:47.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:47.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:55:47.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/mirror_snapshot_schedule"}]: dispatch 2024-03-18T14:55:47.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi012.tmscgt/trash_purge_schedule"}]: dispatch 2024-03-18T14:55:48.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:47 smithi012 ceph-mon[111904]: [18/Mar/2024:14:55:46] ENGINE Bus STARTING 2024-03-18T14:55:48.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:47 smithi012 ceph-mon[111904]: [18/Mar/2024:14:55:46] ENGINE Serving on http://172.21.15.12:8765 2024-03-18T14:55:48.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:47 smithi012 ceph-mon[111904]: [18/Mar/2024:14:55:46] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:55:48.070 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:47 smithi012 ceph-mon[111904]: [18/Mar/2024:14:55:46] ENGINE Bus STARTED 2024-03-18T14:55:48.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:47 smithi012 ceph-mon[111904]: mgrmap e37: smithi012.tmscgt(active, since 1.00942s), standbys: smithi100.rwnrcn 2024-03-18T14:55:48.150 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:47 smithi100 ceph-mon[37807]: [18/Mar/2024:14:55:46] ENGINE Bus STARTING 2024-03-18T14:55:48.150 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:47 smithi100 ceph-mon[37807]: [18/Mar/2024:14:55:46] ENGINE Serving on http://172.21.15.12:8765 2024-03-18T14:55:48.150 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:47 smithi100 ceph-mon[37807]: [18/Mar/2024:14:55:46] ENGINE Serving on https://172.21.15.12:7150 2024-03-18T14:55:48.150 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:47 smithi100 ceph-mon[37807]: [18/Mar/2024:14:55:46] ENGINE Bus STARTED 2024-03-18T14:55:48.150 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:47 smithi100 ceph-mon[37807]: mgrmap e37: smithi012.tmscgt(active, since 1.00942s), standbys: smithi100.rwnrcn 2024-03-18T14:55:48.987 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:48 smithi100 ceph-mon[37807]: pgmap v3: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:48.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:48 smithi012 ceph-mon[111904]: pgmap v3: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:50.177 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:49 smithi012 ceph-mon[111904]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:50.177 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:49 smithi012 ceph-mon[111904]: mgrmap e38: smithi012.tmscgt(active, since 3s), standbys: smithi100.rwnrcn 2024-03-18T14:55:50.177 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:50.177 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:50.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:49 smithi100 ceph-mon[37807]: pgmap v4: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:49 smithi100 ceph-mon[37807]: mgrmap e38: smithi012.tmscgt(active, since 3s), standbys: smithi100.rwnrcn 2024-03-18T14:55:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:49 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:50.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:49 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:51.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: mgrmap e39: smithi012.tmscgt(active, since 4s), standbys: smithi100.rwnrcn 2024-03-18T14:55:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:51.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:52.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: pgmap v5: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: mgrmap e39: smithi012.tmscgt(active, since 4s), standbys: smithi100.rwnrcn 2024-03-18T14:55:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:51 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:53 smithi012 ceph-mon[111904]: pgmap v6: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:53 smithi100 ceph-mon[37807]: pgmap v6: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-18T14:55:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:53 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:53 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:55.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:55 smithi012 ceph-mon[111904]: pgmap v7: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-18T14:55:56.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:55 smithi100 ceph-mon[37807]: pgmap v7: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: pgmap v8: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Detected new or changed devices on smithi012 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:55:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.493 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: pgmap v8: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Detected new or changed devices on smithi012 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.conf 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T14:55:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: Updating smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:57.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:57 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:58.107 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:58 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:58.107 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:58 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:58.107 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:58 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-18T14:55:58.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:55:58.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:55:58.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: pgmap v9: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: Upgrade: Updating mon.smithi100 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:55:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:55:59 smithi012 ceph-mon[111904]: Deploying daemon mon.smithi100 on smithi100 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: pgmap v9: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: Upgrade: Updating mon.smithi100 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:55:59 smithi100 ceph-mon[37807]: Deploying daemon mon.smithi100 on smithi100 2024-03-18T14:56:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:00 smithi100 systemd[1]: Stopping Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:56:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100[37779]: 2024-03-18T14:56:00.682+0000 7f405d208700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi100 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:56:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100[37779]: 2024-03-18T14:56:00.682+0000 7f405d208700 -1 mon.smithi100@1(peon) e2 *** Got Signal Terminated *** 2024-03-18T14:56:01.094 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:00 smithi100 podman[84698]: 2024-03-18 14:56:00.805596432 +0000 UTC m=+0.218245039 container died ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=quincy-6716c9f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-18T14:56:01.426 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 podman[84698]: 2024-03-18 14:56:01.191398954 +0000 UTC m=+0.604047568 container cleanup ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=quincy-6716c9f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T14:56:01.426 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 bash[84698]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100 2024-03-18T14:56:01.780 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 podman[84712]: 2024-03-18 14:56:01.426445543 +0000 UTC m=+0.622994832 container remove ba097be81c80f6b180b5a317614967b0d96f51c36559285de799f6af6f250718 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=quincy-6716c9f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-18T14:56:01.780 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi100.service: Deactivated successfully. 2024-03-18T14:56:01.780 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 systemd[1]: Stopped Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:56:01.780 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi100.service: Consumed 7.053s CPU time. 2024-03-18T14:56:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 systemd[1]: Starting Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:56:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:01 smithi100 podman[84849]: 2024-03-18 14:56:01.898849458 +0000 UTC m=+0.018558722 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:56:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 podman[84849]: 2024-03-18 14:56:02.027868837 +0000 UTC m=+0.147578136 container create 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:56:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 podman[84849]: 2024-03-18 14:56:02.255499227 +0000 UTC m=+0.375208482 container init 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:56:02.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 podman[84849]: 2024-03-18 14:56:02.258582085 +0000 UTC m=+0.378291340 container start 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: pidfile_write: ignore empty --pid-file 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: load: jerasure load: lrc 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: RocksDB version: 7.9.2 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Git sha 0 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: DB SUMMARY 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: DB Session ID: RF6HOH91UXBT0KFL6E6T 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: CURRENT file: CURRENT 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1092 Bytes 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi100/store.db dir, Total Num: 1, files: 000027.sst 2024-03-18T14:56:02.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi100/store.db: 000025.log size: 9301647 ; 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.error_if_exists: 0 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.create_if_missing: 0 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.paranoid_checks: 1 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.env: 0x55e437f6ac60 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.info_log: 0x55e439371980 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.statistics: (nil) 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.use_fsync: 0 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_log_file_size: 0 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T14:56:02.547 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_fallocate: 1 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.use_direct_reads: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.db_log_dir: 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.wal_dir: 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T14:56:02.548 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.write_buffer_manager: 0x55e4393441e0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.unordered_write: 0 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T14:56:02.549 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T14:56:02.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T14:56:02.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T14:56:02.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.row_cache: None 2024-03-18T14:56:02.550 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.wal_filter: None 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.two_write_queues: 0 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.wal_compression: 0 2024-03-18T14:56:02.551 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.atomic_flush: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.log_readahead_size: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_background_jobs: 2 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_background_compactions: -1 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_subcompactions: 1 2024-03-18T14:56:02.552 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_open_files: -1 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_background_flushes: -1 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Compression algorithms supported: 2024-03-18T14:56:02.553 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kZSTD supported: 0 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kXpressCompression supported: 0 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kZlibCompression supported: 1 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kSnappyCompression supported: 1 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kLZ4Compression supported: 1 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: kBZip2Compression supported: 0 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi100/store.db/MANIFEST-000009 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.merge_operator: 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_filter: None 2024-03-18T14:56:02.554 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e439371a80) 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_type: 0 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: data_block_index_type: 0 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_shortening: 1 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: checksum: 4 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: no_block_cache: 0 2024-03-18T14:56:02.555 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache: 0x55e439377090 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_options: 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: capacity : 536870912 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: num_shard_bits : 4 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: strict_capacity_limit : 0 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_cache_compressed: (nil) 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: persistent_cache: (nil) 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_size: 4096 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_size_deviation: 10 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_restart_interval: 16 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: index_block_restart_interval: 1 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: metadata_block_size: 4096 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: partition_filters: 0 2024-03-18T14:56:02.556 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: use_delta_encoding: 1 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: filter_policy: bloomfilter 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: whole_key_filtering: 1 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: verify_compression: 0 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: format_version: 5 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: enable_index_compression: 1 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: block_align: 0 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: max_auto_readahead_size: 262144 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: prepopulate_block_cache: 0 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: initial_auto_readahead_size: 8192 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression: NoCompression 2024-03-18T14:56:02.557 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.num_levels: 7 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T14:56:02.558 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T14:56:02.559 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T14:56:02.560 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.table_properties_collectors: 2024-03-18T14:56:02.561 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.inplace_update_support: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.bloom_locality: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.max_successive_merges: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.ttl: 2592000 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T14:56:02.562 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enable_blob_files: false 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.min_blob_size: 0 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi100/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12081, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-18T14:56:02.563 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 013bb95c-ee2d-4e09-98c4-2b340d1477e0 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773762287796, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773762309717, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 5513486, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12078, "largest_seqno": 16094, "table_properties": {"data_size": 5499107, "index_size": 9134, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4229, "raw_key_size": 37245, "raw_average_key_size": 22, "raw_value_size": 5466264, "raw_average_value_size": 3281, "num_data_blocks": 410, "num_entries": 1666, "num_filter_entries": 1666, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710773762, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "013bb95c-ee2d-4e09-98c4-2b340d1477e0", "db_session_id": "RF6HOH91UXBT0KFL6E6T", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710773762309817, "job": 1, "event": "recovery_finished"} 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi100/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e43946a000 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: DB pointer 0x55e439454000 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** DB Stats ** 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T14:56:02.564 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: L0 1/0 5.26 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 577.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: L6 1/0 5.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Sum 2/0 10.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 577.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 577.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-18T14:56:02.565 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 577.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Cumulative compaction: 0.01 GB write, 213.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Interval compaction: 0.01 GB write, 213.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T14:56:02.566 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Block cache BinnedLRUCache@0x55e439377090#2 capacity: 512.00 MB usage: 13.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: starting mon.smithi100 rank 1 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi100 fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???) e2 preinit fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).mds e15 new map 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).mds e15 print_map 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: e15 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: legacy client fscid: 1 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Filesystem 'cephfs' (1) 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: fs_name cephfs 2024-03-18T14:56:02.567 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: epoch 15 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: tableserver 0 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: root 0 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: session_timeout 60 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: session_autoclose 300 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: max_file_size 1099511627776 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: max_xattr_size 65536 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: required_client_features {} 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: last_failure 0 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: last_failure_osd_epoch 0 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: max_mds 1 2024-03-18T14:56:02.568 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: in 0 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: up {0=14476} 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: failed 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: damaged 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: stopped 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: data_pools [3] 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: metadata_pool 2 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: inline_data disabled 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: balancer 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: bal_rank_mask -1 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: standby_count_wanted 1 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: qdb_cluster leader: 0 members: 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: [mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: [mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.569 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: Standby daemons: 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: [mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout: [mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 ceph-mon[84868]: mon.smithi100@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 bash[84849]: 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 2024-03-18T14:56:02.570 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:02 smithi100 systemd[1]: Started Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:56:03.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- 172.21.15.12:0/1108791846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4664102e00 msgr2=0x7f46641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 --2- 172.21.15.12:0/1108791846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4664102e00 0x7f46641031d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4658004940 tx=0x7f4658009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- 172.21.15.12:0/1108791846 shutdown_connections 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 --2- 172.21.15.12:0/1108791846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46641037a0 0x7f46640fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 --2- 172.21.15.12:0/1108791846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4664102e00 0x7f46641031d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- 172.21.15.12:0/1108791846 >> 172.21.15.12:0/1108791846 conn(0x7f46640f89b0 msgr2=0x7f46640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- 172.21.15.12:0/1108791846 shutdown_connections 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- 172.21.15.12:0/1108791846 wait complete. 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 Processor -- start 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.275+0000 7f466b17a700 1 -- start start 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f466b17a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4664102e00 0x7f46640702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f466b17a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f466b17a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46640722a0 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f466b17a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664070d40 con 0x7f4664102e00 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53392/0 (socket says 172.21.15.12:53392) 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 -- 172.21.15.12:0/619442405 learned_addr learned my addr 172.21.15.12:0/619442405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4668f16700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4664102e00 0x7f46640702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 -- 172.21.15.12:0/619442405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4664102e00 msgr2=0x7f46640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4664102e00 0x7f46640702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 -- 172.21.15.12:0/619442405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658005040 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f4663fff700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f465000b790 tx=0x7f4650009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f465000f6f0 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4664071020 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f465000fd20 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.276+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46500103a0 con 0x7f46641037a0 2024-03-18T14:56:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.277+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466411a190 con 0x7f46641037a0 2024-03-18T14:56:03.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.277+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46640fe550 con 0x7f46641037a0 2024-03-18T14:56:03.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.278+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f4650011400 con 0x7f46641037a0 2024-03-18T14:56:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.278+0000 7f46617fa700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 0x7f46540704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.278+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f465008d5a0 con 0x7f46641037a0 2024-03-18T14:56:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.278+0000 7f4668f16700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 0x7f46540704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.279+0000 7f4668f16700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 0x7f46540704c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4658004f90 tx=0x7f4658009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.283+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4650056a80 con 0x7f46641037a0 2024-03-18T14:56:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.512+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4664109fe0 con 0x7f465406e010 2024-03-18T14:56:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.514+0000 7f46617fa700 1 -- 172.21.15.12:0/619442405 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f4664109fe0 con 0x7f465406e010 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 msgr2=0x7f46540704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 0x7f46540704c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4658004f90 tx=0x7f4658009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 msgr2=0x7f4664070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f465000b790 tx=0x7f4650009220 comp rx=0 tx=0).stop 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 shutdown_connections 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f465406e010 0x7f46540704c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4664102e00 0x7f46640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 --2- 172.21.15.12:0/619442405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46641037a0 0x7f4664070800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 >> 172.21.15.12:0/619442405 conn(0x7f46640f89b0 msgr2=0x7f46640f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 shutdown_connections 2024-03-18T14:56:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.516+0000 7f466b17a700 1 -- 172.21.15.12:0/619442405 wait complete. 2024-03-18T14:56:03.528 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 -- 172.21.15.12:0/2340047347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 msgr2=0x7efd5c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2340047347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c1031d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd4c004940 tx=0x7efd4c009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 -- 172.21.15.12:0/2340047347 shutdown_connections 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2340047347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2340047347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c1031d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 -- 172.21.15.12:0/2340047347 >> 172.21.15.12:0/2340047347 conn(0x7efd5c0f89b0 msgr2=0x7efd5c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 -- 172.21.15.12:0/2340047347 shutdown_connections 2024-03-18T14:56:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.620+0000 7efd63c8f700 1 -- 172.21.15.12:0/2340047347 wait complete. 2024-03-18T14:56:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 Processor -- start 2024-03-18T14:56:03.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 -- start start 2024-03-18T14:56:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c1147d0 con 0x7efd5c102e00 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd63c8f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c10db30 con 0x7efd5c1037a0 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53412/0 (socket says 172.21.15.12:53412) 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 -- 172.21.15.12:0/2298574327 learned_addr learned my addr 172.21.15.12:0/2298574327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd6122a700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 -- 172.21.15.12:0/2298574327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 msgr2=0x7efd5c1140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c1140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 -- 172.21.15.12:0/2298574327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c005040 con 0x7efd5c102e00 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd6122a700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c1140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.621+0000 7efd61a2b700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009150 tx=0x7efd4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.622+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4c00e040 con 0x7efd5c102e00 2024-03-18T14:56:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.622+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd4c013420 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.622+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c10ddb0 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.622+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4c01c630 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.622+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c10e370 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.623+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c0fe550 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.623+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7efd4c013590 con 0x7efd5c102e00 2024-03-18T14:56:03.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.624+0000 7efd527fc700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 0x7efd480704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.624+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7efd4c08eab0 con 0x7efd5c102e00 2024-03-18T14:56:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.624+0000 7efd6122a700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 0x7efd480704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.624+0000 7efd6122a700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 0x7efd480704b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd58006fd0 tx=0x7efd58008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.628+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd4c05ba50 con 0x7efd5c102e00 2024-03-18T14:56:03.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.783+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4c093020 con 0x7efd5c102e00 2024-03-18T14:56:03.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.855+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd5c109f70 con 0x7efd4806e000 2024-03-18T14:56:03.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.856+0000 7efd527fc700 1 -- 172.21.15.12:0/2298574327 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7efd5c109f70 con 0x7efd4806e000 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 msgr2=0x7efd480704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 0x7efd480704b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd58006fd0 tx=0x7efd58008040 comp rx=0 tx=0).stop 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 msgr2=0x7efd5c113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009150 tx=0x7efd4c009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 shutdown_connections 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7efd4806e000 0x7efd480704b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd5c102e00 0x7efd5c113bb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 --2- 172.21.15.12:0/2298574327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1037a0 0x7efd5c1140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 >> 172.21.15.12:0/2298574327 conn(0x7efd5c0f89b0 msgr2=0x7efd5c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:03.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 shutdown_connections 2024-03-18T14:56:03.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.859+0000 7efd63c8f700 1 -- 172.21.15.12:0/2298574327 wait complete. 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 -- 172.21.15.12:0/4251931069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc101d30 msgr2=0x7fc9cc102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 --2- 172.21.15.12:0/4251931069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc101d30 0x7fc9cc102100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009ce0 tx=0x7fc9bc004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 -- 172.21.15.12:0/4251931069 shutdown_connections 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 --2- 172.21.15.12:0/4251931069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc0fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 --2- 172.21.15.12:0/4251931069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc101d30 0x7fc9cc102100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 -- 172.21.15.12:0/4251931069 >> 172.21.15.12:0/4251931069 conn(0x7fc9cc0f7860 msgr2=0x7fc9cc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 -- 172.21.15.12:0/4251931069 shutdown_connections 2024-03-18T14:56:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.960+0000 7fc9d4097700 1 -- 172.21.15.12:0/4251931069 wait complete. 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 Processor -- start 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 -- start start 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc113640 con 0x7fc9cc1026d0 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d4097700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10cc20 con 0x7fc9cc101d30 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d1632700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d1632700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53438/0 (socket says 172.21.15.12:53438) 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d1632700 1 -- 172.21.15.12:0/3933344820 learned_addr learned my addr 172.21.15.12:0/3933344820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.961+0000 7fc9d1e33700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1e33700 1 -- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 msgr2=0x7fc9cc112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1e33700 1 -- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 msgr2=0x7fc9cc112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:56:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1e33700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1e33700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1632700 1 -- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 msgr2=0x7fc9cc112ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1632700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1632700 1 -- 172.21.15.12:0/3933344820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc009940 con 0x7fc9cc1026d0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d1632700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0002a80 tx=0x7fc9c0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c0007cc0 con 0x7fc9cc1026d0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9c000f040 con 0x7fc9cc1026d0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc10cf00 con 0x7fc9cc1026d0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c000db30 con 0x7fc9cc1026d0 2024-03-18T14:56:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.962+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc10d4c0 con 0x7fc9cc1026d0 2024-03-18T14:56:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.963+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc0fd480 con 0x7fc9cc1026d0 2024-03-18T14:56:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.964+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fc9c000dcd0 con 0x7fc9cc1026d0 2024-03-18T14:56:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.964+0000 7fc9ca7fc700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 0x7fc9b40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.964+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fc9c008cff0 con 0x7fc9cc1026d0 2024-03-18T14:56:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.964+0000 7fc9d1e33700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 0x7fc9b40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.964+0000 7fc9d1e33700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 0x7fc9b40704c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b740 tx=0x7fc9bc004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:03.969+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc9c0059e70 con 0x7fc9cc1026d0 2024-03-18T14:56:04.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.184+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9cc108ec0 con 0x7fc9b406e010 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.196+0000 7fc9ca7fc700 1 -- 172.21.15.12:0/3933344820 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc9cc108ec0 con 0x7fc9b406e010 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (113s) 12s ago 12m 15.4M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (13m) 12s ago 13m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (12m) 14s ago 12m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (97s) 12s ago 12m 78.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:56:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (7m) 12s ago 7m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (7m) 12s ago 7m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (7m) 14s ago 7m 16.4M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (7m) 14s ago 7m 13.2M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (3m) 12s ago 14m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (2m) 14s ago 11m 446M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (28s) 12s ago 14m 32.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 starting - - - 2048M 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (2m) 12s ago 13m 15.0M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (2m) 14s ago 11m 16.1M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (11m) 12s ago 11m 257M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (10m) 12s ago 10m 270M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (10m) 12s ago 10m 222M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:56:04.199 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (9m) 14s ago 9m 290M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:56:04.200 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (9m) 14s ago 9m 259M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:56:04.200 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (8m) 14s ago 8m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:56:04.200 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (2m) 12s ago 12m 61.8M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:56:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 msgr2=0x7fc9b40704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 0x7fc9b40704c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b740 tx=0x7fc9bc004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 msgr2=0x7fc9cc112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0002a80 tx=0x7fc9c0004630 comp rx=0 tx=0).stop 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 shutdown_connections 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc9b406e010 0x7fc9b40704c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc101d30 0x7fc9cc112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 --2- 172.21.15.12:0/3933344820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9cc1026d0 0x7fc9cc112ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 >> 172.21.15.12:0/3933344820 conn(0x7fc9cc0f7860 msgr2=0x7fc9cc0f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 shutdown_connections 2024-03-18T14:56:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.199+0000 7fc9d4097700 1 -- 172.21.15.12:0/3933344820 wait complete. 2024-03-18T14:56:04.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 -- 172.21.15.12:0/4136874277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e20 msgr2=0x7f2da01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/4136874277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e20 0x7f2da01031f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009ce0 tx=0x7f2d90004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 -- 172.21.15.12:0/4136874277 shutdown_connections 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/4136874277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da01037c0 0x7f2da00fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/4136874277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da0102e20 0x7f2da01031f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 -- 172.21.15.12:0/4136874277 >> 172.21.15.12:0/4136874277 conn(0x7f2da00f89b0 msgr2=0x7f2da00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 -- 172.21.15.12:0/4136874277 shutdown_connections 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.300+0000 7f2da5cdd700 1 -- 172.21.15.12:0/4136874277 wait complete. 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 Processor -- start 2024-03-18T14:56:04.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 -- start start 2024-03-18T14:56:04.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0072210 con 0x7f2da01037c0 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2da5cdd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0070d40 con 0x7f2da0102e20 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53468/0 (socket says 172.21.15.12:53468) 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9effd700 1 -- 172.21.15.12:0/1998252541 learned_addr learned my addr 172.21.15.12:0/1998252541 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9f7fe700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9f7fe700 1 -- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 msgr2=0x7f2da00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9f7fe700 1 -- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 msgr2=0x7f2da00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:56:04.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9f7fe700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.301+0000 7f2d9f7fe700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d9effd700 1 -- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 msgr2=0x7f2da00702c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d9effd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d9effd700 1 -- 172.21.15.12:0/1998252541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90009940 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d9effd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d88002a80 tx=0x7f2d88004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d88007cc0 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d8800f040 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da0071020 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8800db30 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.302+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da011a2b0 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.303+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da0100130 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.303+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f2d8800dcd0 con 0x7f2da01037c0 2024-03-18T14:56:04.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.304+0000 7f2d97fff700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 0x7f2d8c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.304+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f2d8808cf20 con 0x7f2da01037c0 2024-03-18T14:56:04.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.304+0000 7f2d9f7fe700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 0x7f2d8c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.304+0000 7f2d9f7fe700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 0x7f2d8c0704c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000b740 tx=0x7f2d90004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:04.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.309+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d880c1740 con 0x7f2da01037c0 2024-03-18T14:56:04.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.606+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2da0109f40 con 0x7f2da01037c0 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.607+0000 7f2d97fff700 1 -- 172.21.15.12:0/1998252541 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f2d880562f0 con 0x7f2da01037c0 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:56:04.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 10, 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:56:04.610 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 msgr2=0x7f2d8c0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 0x7f2d8c0704c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000b740 tx=0x7f2d90004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 msgr2=0x7f2da0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d88002a80 tx=0x7f2d88004630 comp rx=0 tx=0).stop 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 shutdown_connections 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d8c06e010 0x7f2d8c0704c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da0102e20 0x7f2da00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 --2- 172.21.15.12:0/1998252541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2da01037c0 0x7f2da0070800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 >> 172.21.15.12:0/1998252541 conn(0x7f2da00f89b0 msgr2=0x7f2da01073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 shutdown_connections 2024-03-18T14:56:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.610+0000 7f2da5cdd700 1 -- 172.21.15.12:0/1998252541 wait complete. 2024-03-18T14:56:04.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- 172.21.15.12:0/529594578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 msgr2=0x7f2e340fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/529594578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e340fb910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e24004a50 tx=0x7f2e2400af40 comp rx=0 tx=0).stop 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- 172.21.15.12:0/529594578 shutdown_connections 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/529594578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e340fb910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/529594578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- 172.21.15.12:0/529594578 >> 172.21.15.12:0/529594578 conn(0x7f2e340714c0 msgr2=0x7f2e340718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- 172.21.15.12:0/529594578 shutdown_connections 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- 172.21.15.12:0/529594578 wait complete. 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 Processor -- start 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.712+0000 7f2e3c45a700 1 -- start start 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3c45a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3c45a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3c45a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34114700 con 0x7f2e340fb4a0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3c45a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3410daf0 con 0x7f2e341014f0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53492/0 (socket says 172.21.15.12:53492) 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 -- 172.21.15.12:0/2547730448 learned_addr learned my addr 172.21.15.12:0/2547730448 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e399f5700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e399f5700 1 -- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 msgr2=0x7f2e341140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e399f5700 1 -- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 msgr2=0x7f2e341140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e399f5700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e399f5700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 -- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 msgr2=0x7f2e341140b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 -- 172.21.15.12:0/2547730448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24005040 con 0x7f2e340fb4a0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3a1f6700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e300041d0 tx=0x7f2e300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:04.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e30007cc0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e3000f040 con 0x7f2e340fb4a0 2024-03-18T14:56:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.713+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3410ddd0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.714+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3000dbe0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.714+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3410e390 con 0x7f2e340fb4a0 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.715+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e340fcbd0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.715+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f2e3000dd40 con 0x7f2e340fb4a0 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.715+0000 7f2e2affd700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 0x7f2e200704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.715+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f2e3008d0c0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.716+0000 7f2e399f5700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 0x7f2e200704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:04.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.716+0000 7f2e399f5700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 0x7f2e200704b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e3410ecb0 tx=0x7f2e2400af40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.720+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e30059f40 con 0x7f2e340fb4a0 2024-03-18T14:56:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.971+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e3410f220 con 0x7f2e340fb4a0 2024-03-18T14:56:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.972+0000 7f2e2affd700 1 -- 172.21.15.12:0/2547730448 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f2e30059ad0 con 0x7f2e340fb4a0 2024-03-18T14:56:04.973 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: mon.smithi012 calling monitor election 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: mon.smithi100 calling monitor election 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: monmap epoch 3 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: last_changed 2024-03-18T14:56:03.771169+0000 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: created 2024-03-18T14:41:22.518337+0000 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: min_mon_release 19 (squid) 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: election_strategy: 1 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.smithi012 2024-03-18T14:56:04.974 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: 1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.smithi100 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: osdmap e48: 6 total, 6 up, 6 in 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: mgrmap e39: smithi012.tmscgt(active, since 17s), standbys: smithi100.rwnrcn 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: overall HEALTH_OK 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:04.975 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:04 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1998252541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:04.975 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:56:04.975 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:56:04.975 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:56:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:56:04.977 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:56:04.978 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:04.979 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:04.979 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.974+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 msgr2=0x7f2e200704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.974+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 0x7f2e200704b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e3410ecb0 tx=0x7f2e2400af40 comp rx=0 tx=0).stop 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.974+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 msgr2=0x7f2e34113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.974+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e300041d0 tx=0x7f2e300046b0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 shutdown_connections 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e2006e000 0x7f2e200704b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e340fb4a0 0x7f2e34113b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 --2- 172.21.15.12:0/2547730448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e341014f0 0x7f2e341140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 >> 172.21.15.12:0/2547730448 conn(0x7f2e340714c0 msgr2=0x7f2e340f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 shutdown_connections 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:04.975+0000 7f2e3c45a700 1 -- 172.21.15.12:0/2547730448 wait complete. 2024-03-18T14:56:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi012"}]: dispatch 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: mon.smithi012 calling monitor election 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: mon.smithi100 calling monitor election 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mon metadata", "id": "smithi100"}]: dispatch 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: mon.smithi012 is new leader, mons smithi012,smithi100 in quorum (ranks 0,1) 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: monmap epoch 3 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: fsid 74f0a062-e535-11ee-95c9-87774f69a715 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: last_changed 2024-03-18T14:56:03.771169+0000 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: created 2024-03-18T14:41:22.518337+0000 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: min_mon_release 19 (squid) 2024-03-18T14:56:05.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: election_strategy: 1 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.smithi012 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: 1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.smithi100 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: osdmap e48: 6 total, 6 up, 6 in 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: mgrmap e39: smithi012.tmscgt(active, since 17s), standbys: smithi100.rwnrcn 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: overall HEALTH_OK 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:05.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:04 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1998252541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- 172.21.15.12:0/3721762280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 msgr2=0x7f91480fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 --2- 172.21.15.12:0/3721762280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fbd30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9140009ce0 tx=0x7f9140004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- 172.21.15.12:0/3721762280 shutdown_connections 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 --2- 172.21.15.12:0/3721762280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fbd30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 --2- 172.21.15.12:0/3721762280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91481020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- 172.21.15.12:0/3721762280 >> 172.21.15.12:0/3721762280 conn(0x7f91480f7870 msgr2=0x7f91480f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- 172.21.15.12:0/3721762280 shutdown_connections 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- 172.21.15.12:0/3721762280 wait complete. 2024-03-18T14:56:05.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 Processor -- start 2024-03-18T14:56:05.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.078+0000 7f914ee0d700 1 -- start start 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914ee0d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914ee0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914ee0d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480ff940 con 0x7f91481026c0 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914ee0d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480fe420 con 0x7f9148101d20 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914cba9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53508/0 (socket says 172.21.15.12:53508) 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 -- 172.21.15.12:0/2580224861 learned_addr learned my addr 172.21.15.12:0/2580224861 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914cba9700 1 -- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 msgr2=0x7f91480fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914cba9700 1 -- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 msgr2=0x7f91480fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914cba9700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f914cba9700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 -- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 msgr2=0x7f91480fd9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 -- 172.21.15.12:0/2580224861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140009940 con 0x7f91481026c0 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.079+0000 7f9147fff700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f914000be40 tx=0x7f9140004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.080+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914000e040 con 0x7f91481026c0 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.080+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9140004610 con 0x7f91481026c0 2024-03-18T14:56:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.080+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91480fe670 con 0x7f91481026c0 2024-03-18T14:56:05.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.080+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914001c640 con 0x7f91481026c0 2024-03-18T14:56:05.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.080+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914811a270 con 0x7f91481026c0 2024-03-18T14:56:05.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.081+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91480fd470 con 0x7f91481026c0 2024-03-18T14:56:05.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.081+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f9140012a70 con 0x7f91481026c0 2024-03-18T14:56:05.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.082+0000 7f91457fa700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 0x7f91340704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.082+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f914008f3a0 con 0x7f91481026c0 2024-03-18T14:56:05.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.082+0000 7f914cba9700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 0x7f91340704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.082+0000 7f914cba9700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 0x7f91340704c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9130009e50 tx=0x7f9130009f10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:05.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.087+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91400586f0 con 0x7f91481026c0 2024-03-18T14:56:05.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.314+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9148066910 con 0x7f913406e010 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.315+0000 7f91457fa700 1 -- 172.21.15.12:0/2580224861 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9148066910 con 0x7f913406e010 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr" 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "3/21 daemons upgraded", 2024-03-18T14:56:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mon daemons", 2024-03-18T14:56:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:56:05.318 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 msgr2=0x7f91340704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 0x7f91340704c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9130009e50 tx=0x7f9130009f10 comp rx=0 tx=0).stop 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 msgr2=0x7f91480fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f914000be40 tx=0x7f9140004d10 comp rx=0 tx=0).stop 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 shutdown_connections 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f913406e010 0x7f91340704c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9148101d20 0x7f91480fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 --2- 172.21.15.12:0/2580224861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91481026c0 0x7f91480fdee0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 >> 172.21.15.12:0/2580224861 conn(0x7f91480f7870 msgr2=0x7f9148066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 shutdown_connections 2024-03-18T14:56:05.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.317+0000 7f914ee0d700 1 -- 172.21.15.12:0/2580224861 wait complete. 2024-03-18T14:56:05.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 -- 172.21.15.12:0/1015660087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 msgr2=0x7f7f441020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1015660087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f441020c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f34004940 tx=0x7f7f34009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 -- 172.21.15.12:0/1015660087 shutdown_connections 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1015660087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f440fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1015660087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f441020c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 -- 172.21.15.12:0/1015660087 >> 172.21.15.12:0/1015660087 conn(0x7f7f44072bc0 msgr2=0x7f7f44072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 -- 172.21.15.12:0/1015660087 shutdown_connections 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 -- 172.21.15.12:0/1015660087 wait complete. 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.419+0000 7f7f48a98700 1 Processor -- start 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f48a98700 1 -- start start 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f48a98700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f48a98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f48a98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4410f680 con 0x7f7f44101cf0 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f48a98700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4410f7c0 con 0x7f7f440fbca0 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53526/0 (socket says 172.21.15.12:53526) 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 -- 172.21.15.12:0/1599099112 learned_addr learned my addr 172.21.15.12:0/1599099112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f4259c700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f4259c700 1 -- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 msgr2=0x7f7f4410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 -- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 msgr2=0x7f7f4410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f4259c700 1 -- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 msgr2=0x7f7f4410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f4259c700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f4259c700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 -- 172.21.15.12:0/1599099112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34005040 con 0x7f7f44101cf0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.420+0000 7f7f41d9b700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f38002a50 tx=0x7f7f380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f38007cc0 con 0x7f7f44101cf0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f3800f040 con 0x7f7f44101cf0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f44117ab0 con 0x7f7f44101cf0 2024-03-18T14:56:05.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f38009640 con 0x7f7f44101cf0 2024-03-18T14:56:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f44118070 con 0x7f7f44101cf0 2024-03-18T14:56:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.421+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440fd390 con 0x7f7f44101cf0 2024-03-18T14:56:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.422+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f7f380097a0 con 0x7f7f44101cf0 2024-03-18T14:56:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.423+0000 7f7f32ffd700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 0x7f7f2c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.423+0000 7f7f4259c700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 0x7f7f2c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.423+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f7f3808d6c0 con 0x7f7f44101cf0 2024-03-18T14:56:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.423+0000 7f7f4259c700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 0x7f7f2c0704b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b3b0 tx=0x7f7f34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.428+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f38056a10 con 0x7f7f44101cf0 2024-03-18T14:56:05.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.719+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f44108d80 con 0x7f7f44101cf0 2024-03-18T14:56:05.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.719+0000 7f7f32ffd700 1 -- 172.21.15.12:0/1599099112 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7f3805a0d0 con 0x7f7f44101cf0 2024-03-18T14:56:05.721 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.721+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 msgr2=0x7f7f2c0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.721+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 0x7f7f2c0704b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b3b0 tx=0x7f7f34009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.721+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 msgr2=0x7f7f4410f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.721+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f38002a50 tx=0x7f7f380046b0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 shutdown_connections 2024-03-18T14:56:05.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7f2c06e000 0x7f7f2c0704b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f440fbca0 0x7f7f4410ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 --2- 172.21.15.12:0/1599099112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f44101cf0 0x7f7f4410f140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:05.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 >> 172.21.15.12:0/1599099112 conn(0x7f7f44072bc0 msgr2=0x7f7f441049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:05.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 shutdown_connections 2024-03-18T14:56:05.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:05.722+0000 7f7f48a98700 1 -- 172.21.15.12:0/1599099112 wait complete. 2024-03-18T14:56:05.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:05 smithi012 ceph-mon[111904]: pgmap v12: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-18T14:56:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:05 smithi012 ceph-mon[111904]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:05 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2547730448' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:56:05.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:05 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1599099112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:56:06.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:05 smithi100 ceph-mon[84868]: pgmap v12: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-18T14:56:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:05 smithi100 ceph-mon[84868]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:05 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2547730448' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:56:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:05 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1599099112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:56:07.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:06 smithi012 ceph-mon[111904]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:06 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:06 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:07.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:06 smithi100 ceph-mon[84868]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:07.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:06 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:07.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:06 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:08.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:07 smithi012 ceph-mon[111904]: pgmap v13: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:08.291 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:07 smithi100 ceph-mon[84868]: pgmap v13: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:08.292 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:08.292 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:09 smithi012 ceph-mon[111904]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:10.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:09 smithi100 ceph-mon[84868]: pgmap v14: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:11.823 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:11.823 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:11.823 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:11.823 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:11.823 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:56:11.824 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:12.138 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: pgmap v15: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:12.138 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:12.138 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:12.138 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:12.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:56:12.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:12.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:56:12.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:56:12.139 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:56:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:56:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:13.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:12 smithi012 ceph-mon[111904]: Detected new or changed devices on smithi100 2024-03-18T14:56:13.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:12 smithi012 ceph-mon[111904]: Reconfiguring mon.smithi012 (monmap changed)... 2024-03-18T14:56:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:12 smithi012 ceph-mon[111904]: Reconfiguring daemon mon.smithi012 on smithi012 2024-03-18T14:56:13.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:12 smithi100 ceph-mon[84868]: Detected new or changed devices on smithi100 2024-03-18T14:56:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:12 smithi100 ceph-mon[84868]: Reconfiguring mon.smithi012 (monmap changed)... 2024-03-18T14:56:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:12 smithi100 ceph-mon[84868]: Reconfiguring daemon mon.smithi012 on smithi012 2024-03-18T14:56:14.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:14.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:14.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:14.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: Reconfiguring mgr.smithi012.tmscgt (monmap changed)... 2024-03-18T14:56:14.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:56:14.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:56:14.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:14.197 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:13 smithi100 ceph-mon[84868]: Reconfiguring daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: pgmap v16: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: Reconfiguring mgr.smithi012.tmscgt (monmap changed)... 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi012.tmscgt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:13 smithi012 ceph-mon[111904]: Reconfiguring daemon mgr.smithi012.tmscgt on smithi012 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: Reconfiguring crash.smithi012 (monmap changed)... 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:15.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:15 smithi012 ceph-mon[111904]: Reconfiguring daemon crash.smithi012 on smithi012 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: pgmap v17: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: Reconfiguring crash.smithi012 (monmap changed)... 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:15 smithi100 ceph-mon[84868]: Reconfiguring daemon crash.smithi012 on smithi012 2024-03-18T14:56:16.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:16.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:16.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:16.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:56:16.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:17.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:17.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:17.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:17.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:56:17.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:17.889 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:17 smithi012 ceph-mon[111904]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:17.889 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:17 smithi012 ceph-mon[111904]: Reconfiguring osd.0 (monmap changed)... 2024-03-18T14:56:17.889 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:17 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.0 on smithi012 2024-03-18T14:56:18.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:17 smithi100 ceph-mon[84868]: pgmap v18: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:17 smithi100 ceph-mon[84868]: Reconfiguring osd.0 (monmap changed)... 2024-03-18T14:56:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:17 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.0 on smithi012 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:18 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: Reconfiguring osd.1 (monmap changed)... 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.1 on smithi012 2024-03-18T14:56:19.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:19 smithi012 ceph-mon[111904]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:19.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: Reconfiguring osd.1 (monmap changed)... 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.1 on smithi012 2024-03-18T14:56:19.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:19 smithi100 ceph-mon[84868]: pgmap v19: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:20.954 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:20.955 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:20.955 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: Reconfiguring osd.2 (monmap changed)... 2024-03-18T14:56:20.955 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:56:20.955 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:20.955 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:20 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.2 on smithi012 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: Reconfiguring osd.2 (monmap changed)... 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:21.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:20 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.2 on smithi012 2024-03-18T14:56:21.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:21 smithi012 ceph-mon[111904]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:21.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:21 smithi100 ceph-mon[84868]: pgmap v20: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:22.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:22.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:22 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:22.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:22 smithi012 ceph-mon[111904]: Reconfiguring mds.cephfs.smithi012.xtadxo (monmap changed)... 2024-03-18T14:56:22.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:22 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:22.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:22 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:22.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:22 smithi012 ceph-mon[111904]: Reconfiguring daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T14:56:23.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:22 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:23.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:22 smithi100 ceph-mon[84868]: Reconfiguring mds.cephfs.smithi012.xtadxo (monmap changed)... 2024-03-18T14:56:23.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:22 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:23.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:22 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:23.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:22 smithi100 ceph-mon[84868]: Reconfiguring daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T14:56:23.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:23 smithi012 ceph-mon[111904]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:23 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:23 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:23 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:23.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:23 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:23 smithi100 ceph-mon[84868]: pgmap v21: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:23 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:23 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:23 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:23 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:24.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:24 smithi012 ceph-mon[111904]: Reconfiguring mds.cephfs.smithi012.oheuui (monmap changed)... 2024-03-18T14:56:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:24 smithi012 ceph-mon[111904]: Reconfiguring daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T14:56:25.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:24 smithi100 ceph-mon[84868]: Reconfiguring mds.cephfs.smithi012.oheuui (monmap changed)... 2024-03-18T14:56:25.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:24 smithi100 ceph-mon[84868]: Reconfiguring daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T14:56:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:25 smithi012 ceph-mon[111904]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:25 smithi100 ceph-mon[84868]: pgmap v22: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:27.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: Reconfiguring crash.smithi100 (monmap changed)... 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: Reconfiguring daemon crash.smithi100 on smithi100 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:56:27.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: Reconfiguring crash.smithi100 (monmap changed)... 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: Reconfiguring daemon crash.smithi100 on smithi100 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi100.rwnrcn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T14:56:27.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:27.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:27 smithi100 ceph-mon[84868]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:27.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:27 smithi100 ceph-mon[84868]: Reconfiguring mgr.smithi100.rwnrcn (monmap changed)... 2024-03-18T14:56:27.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:27 smithi100 ceph-mon[84868]: Reconfiguring daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:56:28.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:27 smithi012 ceph-mon[111904]: pgmap v23: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:27 smithi012 ceph-mon[111904]: Reconfiguring mgr.smithi100.rwnrcn (monmap changed)... 2024-03-18T14:56:28.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:27 smithi012 ceph-mon[111904]: Reconfiguring daemon mgr.smithi100.rwnrcn on smithi100 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: Reconfiguring mon.smithi100 (monmap changed)... 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:29 smithi100 ceph-mon[84868]: Reconfiguring daemon mon.smithi100 on smithi100 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: pgmap v24: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: Reconfiguring mon.smithi100 (monmap changed)... 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:29.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:29 smithi012 ceph-mon[111904]: Reconfiguring daemon mon.smithi100 on smithi100 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: Reconfiguring osd.3 (monmap changed)... 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:31 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.3 on smithi100 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: pgmap v25: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: Reconfiguring osd.3 (monmap changed)... 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:31.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:31 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.3 on smithi100 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: Reconfiguring osd.4 (monmap changed)... 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:32 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.4 on smithi100 2024-03-18T14:56:32.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: Reconfiguring osd.4 (monmap changed)... 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:32.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:32 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.4 on smithi100 2024-03-18T14:56:33.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:33 smithi100 ceph-mon[84868]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:33.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:33 smithi012 ceph-mon[111904]: pgmap v26: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:34.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: Reconfiguring osd.5 (monmap changed)... 2024-03-18T14:56:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:56:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:34.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:34 smithi100 ceph-mon[84868]: Reconfiguring daemon osd.5 on smithi100 2024-03-18T14:56:34.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: Reconfiguring osd.5 (monmap changed)... 2024-03-18T14:56:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:56:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:34.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:34 smithi012 ceph-mon[111904]: Reconfiguring daemon osd.5 on smithi100 2024-03-18T14:56:35.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:35 smithi012 ceph-mon[111904]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:35.747 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:35 smithi100 ceph-mon[84868]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:35.747 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:35.747 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:35.747 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:35.748 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:35.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 -- 172.21.15.12:0/3455690546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 msgr2=0x7f02e40fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:35.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/3455690546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fbd30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f02d8004940 tx=0x7f02d8009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 -- 172.21.15.12:0/3455690546 shutdown_connections 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/3455690546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fbd30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/3455690546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 0x7f02e41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 -- 172.21.15.12:0/3455690546 >> 172.21.15.12:0/3455690546 conn(0x7f02e40f7870 msgr2=0x7f02e40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 -- 172.21.15.12:0/3455690546 shutdown_connections 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.841+0000 7f02e9ab0700 1 -- 172.21.15.12:0/3455690546 wait complete. 2024-03-18T14:56:35.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 Processor -- start 2024-03-18T14:56:35.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 -- start start 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 0x7f02e40fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40ff940 con 0x7f02e41026c0 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.842+0000 7f02e9ab0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40fe420 con 0x7f02e4101d20 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34016/0 (socket says 172.21.15.12:34016) 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 -- 172.21.15.12:0/2469271890 learned_addr learned my addr 172.21.15.12:0/2469271890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e37fe700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 0x7f02e40fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 -- 172.21.15.12:0/2469271890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 msgr2=0x7f02e40fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 0x7f02e40fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 -- 172.21.15.12:0/2469271890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8005040 con 0x7f02e41026c0 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e2ffd700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02d8000c00 tx=0x7f02d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:35.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d8021070 con 0x7f02e41026c0 2024-03-18T14:56:35.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02d8013c00 con 0x7f02e41026c0 2024-03-18T14:56:35.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d800e070 con 0x7f02e41026c0 2024-03-18T14:56:35.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40fe670 con 0x7f02e41026c0 2024-03-18T14:56:35.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.843+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4119ff0 con 0x7f02e41026c0 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.844+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40fd470 con 0x7f02e41026c0 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.845+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f02d801ca60 con 0x7f02e41026c0 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.845+0000 7f02cbfff700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 0x7f02cc0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.845+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f02d808eca0 con 0x7f02e41026c0 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.845+0000 7f02e37fe700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 0x7f02cc0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:35.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.846+0000 7f02e37fe700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 0x7f02cc0704c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009b80 tx=0x7f02d400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:35.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:35.850+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02d8093070 con 0x7f02e41026c0 2024-03-18T14:56:36.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.078+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02e4108e70 con 0x7f02cc06e010 2024-03-18T14:56:36.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.079+0000 7f02cbfff700 1 -- 172.21.15.12:0/2469271890 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f02e4108e70 con 0x7f02cc06e010 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 msgr2=0x7f02cc0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 0x7f02cc0704c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009b80 tx=0x7f02d400b040 comp rx=0 tx=0).stop 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 msgr2=0x7f02e40fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02d8000c00 tx=0x7f02d8009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 shutdown_connections 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f02cc06e010 0x7f02cc0704c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02e4101d20 0x7f02e40fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 --2- 172.21.15.12:0/2469271890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02e41026c0 0x7f02e40fdee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 >> 172.21.15.12:0/2469271890 conn(0x7f02e40f7870 msgr2=0x7f02e4066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 shutdown_connections 2024-03-18T14:56:36.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.081+0000 7f02e9ab0700 1 -- 172.21.15.12:0/2469271890 wait complete. 2024-03-18T14:56:36.092 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:56:36.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 -- 172.21.15.12:0/68641564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 msgr2=0x7f49f80fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 --2- 172.21.15.12:0/68641564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fbd30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004940 tx=0x7f49e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 -- 172.21.15.12:0/68641564 shutdown_connections 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 --2- 172.21.15.12:0/68641564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fbd30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 --2- 172.21.15.12:0/68641564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 -- 172.21.15.12:0/68641564 >> 172.21.15.12:0/68641564 conn(0x7f49f80f7870 msgr2=0x7f49f80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 -- 172.21.15.12:0/68641564 shutdown_connections 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.186+0000 7f49ff138700 1 -- 172.21.15.12:0/68641564 wait complete. 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 Processor -- start 2024-03-18T14:56:36.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 -- start start 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80ff8f0 con 0x7f49f81026c0 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49ff138700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80fe420 con 0x7f49f8101d20 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.187+0000 7f49fced4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:40854/0 (socket says 172.21.15.12:40854) 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 -- 172.21.15.12:0/1312843829 learned_addr learned my addr 172.21.15.12:0/1312843829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:36.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49f7fff700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 -- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 msgr2=0x7f49f80fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 -- 172.21.15.12:0/1312843829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4005040 con 0x7f49f8101d20 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49f7fff700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.188+0000 7f49fced4700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49ec002a50 tx=0x7f49ec0046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.189+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49ec007cc0 con 0x7f49f8101d20 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.189+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49ec00f040 con 0x7f49f8101d20 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.189+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f80fe700 con 0x7f49f8101d20 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.189+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49ec00da70 con 0x7f49f8101d20 2024-03-18T14:56:36.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.189+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f811a100 con 0x7f49f8101d20 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.190+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f80fd470 con 0x7f49f8101d20 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.190+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f49ec00dc10 con 0x7f49f8101d20 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.191+0000 7f49f57fa700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 0x7f49e80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.191+0000 7f49f7fff700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 0x7f49e80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.191+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f49ec08cf70 con 0x7f49f8101d20 2024-03-18T14:56:36.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.191+0000 7f49f7fff700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 0x7f49e80704c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004890 tx=0x7f49e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.195+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49ec0562a0 con 0x7f49f8101d20 2024-03-18T14:56:36.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.420+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f80f8f70 con 0x7f49e806e010 2024-03-18T14:56:36.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.422+0000 7f49f57fa700 1 -- 172.21.15.12:0/1312843829 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f49f80f8f70 con 0x7f49e806e010 2024-03-18T14:56:36.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 msgr2=0x7f49e80704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 0x7f49e80704c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004890 tx=0x7f49e4009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 msgr2=0x7f49f80fd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49ec002a50 tx=0x7f49ec0046b0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 shutdown_connections 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f49e806e010 0x7f49e80704c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49f8101d20 0x7f49f80fd9a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 --2- 172.21.15.12:0/1312843829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49f81026c0 0x7f49f80fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 >> 172.21.15.12:0/1312843829 conn(0x7f49f80f7870 msgr2=0x7f49f80f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 shutdown_connections 2024-03-18T14:56:36.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.424+0000 7f49ff138700 1 -- 172.21.15.12:0/1312843829 wait complete. 2024-03-18T14:56:36.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 -- 172.21.15.12:0/2826558757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 msgr2=0x7f4e701033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/2826558757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e701033a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e64004940 tx=0x7f4e64009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 -- 172.21.15.12:0/2826558757 shutdown_connections 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/2826558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e700fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/2826558757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e701033a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 -- 172.21.15.12:0/2826558757 >> 172.21.15.12:0/2826558757 conn(0x7f4e70071530 msgr2=0x7f4e70071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 -- 172.21.15.12:0/2826558757 shutdown_connections 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.526+0000 7f4e76ef5700 1 -- 172.21.15.12:0/2826558757 wait complete. 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 Processor -- start 2024-03-18T14:56:36.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 -- start start 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e70113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701144b0 con 0x7f4e70102fd0 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e76ef5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7010dab0 con 0x7f4e700fb540 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e70113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e74c91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e74c91700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:40874/0 (socket says 172.21.15.12:40874) 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e74c91700 1 -- 172.21.15.12:0/3811712592 learned_addr learned my addr 172.21.15.12:0/3811712592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:36.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.527+0000 7f4e6ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e70113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34058/0 (socket says 172.21.15.12:34058) 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e74c91700 1 -- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 msgr2=0x7f4e70113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e74c91700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e70113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e74c91700 1 -- 172.21.15.12:0/3811712592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64005040 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e74c91700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b3b0 tx=0x7f4e64009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6400e040 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e64013420 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7010dd30 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6401c630 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.528+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7010e2f0 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.529+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70100270 con 0x7f4e700fb540 2024-03-18T14:56:36.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.529+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f4e6400baf0 con 0x7f4e700fb540 2024-03-18T14:56:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.530+0000 7f4e6d7fa700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 0x7f4e600704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.530+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f4e6408ebc0 con 0x7f4e700fb540 2024-03-18T14:56:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.530+0000 7f4e6ffff700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 0x7f4e600704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.530+0000 7f4e6ffff700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 0x7f4e600704c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e7010ec70 tx=0x7f4e5c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.535+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e6405ba20 con 0x7f4e700fb540 2024-03-18T14:56:36.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:36 smithi012 ceph-mon[111904]: Reconfiguring mds.cephfs.smithi100.dvtbrf (monmap changed)... 2024-03-18T14:56:36.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:36 smithi012 ceph-mon[111904]: Reconfiguring daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T14:56:36.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.752+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e7010ef70 con 0x7f4e6006e010 2024-03-18T14:56:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.763+0000 7f4e6d7fa700 1 -- 172.21.15.12:0/3811712592 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4e7010ef70 con 0x7f4e6006e010 2024-03-18T14:56:36.763 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:56:36.763 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (2m) 45s ago 13m 15.4M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (13m) 45s ago 13m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c68a6d9a09d2 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (12m) 30s ago 12m 6660k - 17.2.7-684-g6716c9f3 19857db3290c c4ec51570b1e 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (2m) 45s ago 13m 78.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (7m) 45s ago 7m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (7m) 45s ago 7m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (7m) 30s ago 7m 16.5M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (7m) 30s ago 7m 13.4M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (4m) 45s ago 14m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (3m) 30s ago 12m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (61s) 45s ago 15m 32.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (34s) 30s ago 12m 26.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (3m) 45s ago 13m 15.0M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:56:36.764 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (3m) 30s ago 12m 15.8M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (11m) 45s ago 11m 257M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (11m) 45s ago 11m 270M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (10m) 45s ago 10m 222M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (10m) 30s ago 10m 290M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (9m) 30s ago 9m 259M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (9m) 30s ago 9m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:56:36.765 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (2m) 45s ago 12m 61.8M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 msgr2=0x7f4e600704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 0x7f4e600704c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4e7010ec70 tx=0x7f4e5c00b040 comp rx=0 tx=0).stop 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 msgr2=0x7f4e70113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b3b0 tx=0x7f4e64009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 shutdown_connections 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f4e6006e010 0x7f4e600704c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e700fb540 0x7f4e70113920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 --2- 172.21.15.12:0/3811712592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e70102fd0 0x7f4e70113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 >> 172.21.15.12:0/3811712592 conn(0x7f4e70071530 msgr2=0x7f4e700fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 shutdown_connections 2024-03-18T14:56:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.765+0000 7f4e76ef5700 1 -- 172.21.15.12:0/3811712592 wait complete. 2024-03-18T14:56:36.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:36 smithi100 ceph-mon[84868]: Reconfiguring mds.cephfs.smithi100.dvtbrf (monmap changed)... 2024-03-18T14:56:36.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:36 smithi100 ceph-mon[84868]: Reconfiguring daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T14:56:36.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 -- 172.21.15.12:0/2918383638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 msgr2=0x7f3e501031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 --2- 172.21.15.12:0/2918383638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e501031d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004940 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:36.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 -- 172.21.15.12:0/2918383638 shutdown_connections 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 --2- 172.21.15.12:0/2918383638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 0x7f3e500fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 --2- 172.21.15.12:0/2918383638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e501031d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 -- 172.21.15.12:0/2918383638 >> 172.21.15.12:0/2918383638 conn(0x7f3e500f89b0 msgr2=0x7f3e500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 -- 172.21.15.12:0/2918383638 shutdown_connections 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.866+0000 7f3e55e06700 1 -- 172.21.15.12:0/2918383638 wait complete. 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 Processor -- start 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 -- start start 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 0x7f3e50113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50114520 con 0x7f3e50102e00 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e55e06700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5010db70 con 0x7f3e501037a0 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34076/0 (socket says 172.21.15.12:34076) 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.867+0000 7f3e4f7fe700 1 -- 172.21.15.12:0/3272013447 learned_addr learned my addr 172.21.15.12:0/3272013447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e4effd700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 0x7f3e50113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e4f7fe700 1 -- 172.21.15.12:0/3272013447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 msgr2=0x7f3e50113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e4f7fe700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 0x7f3e50113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e4f7fe700 1 -- 172.21.15.12:0/3272013447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40005040 con 0x7f3e50102e00 2024-03-18T14:56:36.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e4f7fe700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000b3b0 tx=0x7f3e40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4000e040 con 0x7f3e50102e00 2024-03-18T14:56:36.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e40013420 con 0x7f3e50102e00 2024-03-18T14:56:36.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4001c630 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5010ddf0 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.868+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5010e3b0 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.869+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50100110 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.870+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f3e40013590 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.870+0000 7f3e37fff700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 0x7f3e380704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.870+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f3e4008eb10 con 0x7f3e50102e00 2024-03-18T14:56:36.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.870+0000 7f3e4effd700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 0x7f3e380704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:36.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.871+0000 7f3e4effd700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 0x7f3e380704c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e5010ece0 tx=0x7f3e4400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:36.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:36.875+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3e4005ba10 con 0x7f3e50102e00 2024-03-18T14:56:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.172+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3e5010efe0 con 0x7f3e50102e00 2024-03-18T14:56:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.172+0000 7f3e37fff700 1 -- 172.21.15.12:0/3272013447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f3e4005b5a0 con 0x7f3e50102e00 2024-03-18T14:56:37.173 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:56:37.173 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:56:37.173 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 10, 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:56:37.174 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 msgr2=0x7f3e380704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 0x7f3e380704c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e5010ece0 tx=0x7f3e4400b040 comp rx=0 tx=0).stop 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 msgr2=0x7f3e50113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000b3b0 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 shutdown_connections 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3e3806e010 0x7f3e380704c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e50102e00 0x7f3e50113990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 --2- 172.21.15.12:0/3272013447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e501037a0 0x7f3e50113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 >> 172.21.15.12:0/3272013447 conn(0x7f3e500f89b0 msgr2=0x7f3e501073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 shutdown_connections 2024-03-18T14:56:37.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.175+0000 7f3e55e06700 1 -- 172.21.15.12:0/3272013447 wait complete. 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 -- 172.21.15.12:0/2396716841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 msgr2=0x7feca4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 --2- 172.21.15.12:0/2396716841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4103340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec98004940 tx=0x7fec98009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 -- 172.21.15.12:0/2396716841 shutdown_connections 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 --2- 172.21.15.12:0/2396716841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 0x7feca40fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 --2- 172.21.15.12:0/2396716841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4103340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.277+0000 7fecac7a7700 1 -- 172.21.15.12:0/2396716841 >> 172.21.15.12:0/2396716841 conn(0x7feca4072310 msgr2=0x7feca4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 -- 172.21.15.12:0/2396716841 shutdown_connections 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 -- 172.21.15.12:0/2396716841 wait complete. 2024-03-18T14:56:37.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 Processor -- start 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 -- start start 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 0x7feca4113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41144d0 con 0x7feca4102f70 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7fecac7a7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca410dad0 con 0x7feca40fb3b0 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7feca9d42700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7feca9d42700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34084/0 (socket says 172.21.15.12:34084) 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.278+0000 7feca9d42700 1 -- 172.21.15.12:0/3408596131 learned_addr learned my addr 172.21.15.12:0/3408596131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fecaa543700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 0x7feca4113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7feca9d42700 1 -- 172.21.15.12:0/3408596131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 msgr2=0x7feca4113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7feca9d42700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 0x7feca4113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7feca9d42700 1 -- 172.21.15.12:0/3408596131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec98005040 con 0x7feca4102f70 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7feca9d42700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feca000eb40 tx=0x7feca000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca0007cc0 con 0x7feca4102f70 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feca000f040 con 0x7feca4102f70 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca410dd50 con 0x7feca4102f70 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca000ca10 con 0x7feca4102f70 2024-03-18T14:56:37.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.279+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca410e290 con 0x7feca4102f70 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.280+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca40fe6d0 con 0x7feca4102f70 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.280+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7feca000cb70 con 0x7feca4102f70 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.281+0000 7fec96ffd700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 0x7fec900704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.281+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7feca008d3f0 con 0x7feca4102f70 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.281+0000 7fecaa543700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 0x7fec900704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.281+0000 7fecaa543700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 0x7fec900704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec98009b30 tx=0x7fec98009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.286+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feca005a3a0 con 0x7feca4102f70 2024-03-18T14:56:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.538+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feca4105390 con 0x7feca4102f70 2024-03-18T14:56:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.539+0000 7fec96ffd700 1 -- 172.21.15.12:0/3408596131 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7feca0059f30 con 0x7feca4102f70 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:56:37.540 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:56:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:37.542 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:56:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 msgr2=0x7fec900704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 0x7fec900704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec98009b30 tx=0x7fec98009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 msgr2=0x7feca4113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feca000eb40 tx=0x7feca000b040 comp rx=0 tx=0).stop 2024-03-18T14:56:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 shutdown_connections 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec9006e010 0x7fec900704c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feca40fb3b0 0x7feca4113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 --2- 172.21.15.12:0/3408596131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feca4102f70 0x7feca4113e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 >> 172.21.15.12:0/3408596131 conn(0x7feca4072310 msgr2=0x7feca40f9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 shutdown_connections 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.541+0000 7fecac7a7700 1 -- 172.21.15.12:0/3408596131 wait complete. 2024-03-18T14:56:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- 172.21.15.12:0/30038498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 msgr2=0x7fe8000fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 --2- 172.21.15.12:0/30038498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe8000fce20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4004940 tx=0x7fe7f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- 172.21.15.12:0/30038498 shutdown_connections 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 --2- 172.21.15.12:0/30038498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe8000fce20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 --2- 172.21.15.12:0/30038498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 0x7fe8001031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- 172.21.15.12:0/30038498 >> 172.21.15.12:0/30038498 conn(0x7fe8000f8980 msgr2=0x7fe8000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- 172.21.15.12:0/30038498 shutdown_connections 2024-03-18T14:56:37.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- 172.21.15.12:0/30038498 wait complete. 2024-03-18T14:56:37.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 Processor -- start 2024-03-18T14:56:37.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.645+0000 7fe80662e700 1 -- start start 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe80662e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 0x7fe8000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe80662e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe80662e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800070d40 con 0x7fe8001037b0 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe80662e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800070e80 con 0x7fe800102e10 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34098/0 (socket says 172.21.15.12:34098) 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 -- 172.21.15.12:0/3110274056 learned_addr learned my addr 172.21.15.12:0/3110274056 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7fffff700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 0x7fe8000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 -- 172.21.15.12:0/3110274056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 msgr2=0x7fe8000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 0x7fe8000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 -- 172.21.15.12:0/3110274056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4005040 con 0x7fe8001037b0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.646+0000 7fe7ff7fe700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009100 tx=0x7fe7f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.647+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f400e040 con 0x7fe8001037b0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.647+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7f4013420 con 0x7fe8001037b0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.647+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8000710d0 con 0x7fe8001037b0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.647+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f401c610 con 0x7fe8001037b0 2024-03-18T14:56:37.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.647+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80011a290 con 0x7fe8001037b0 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.648+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8001052e0 con 0x7fe8001037b0 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.648+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fe7f4013590 con 0x7fe8001037b0 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.649+0000 7fe7fcff9700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 0x7fe7e80704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.649+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fe7f408eb40 con 0x7fe8001037b0 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.649+0000 7fe7fffff700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 0x7fe7e80704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.649+0000 7fe7fffff700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 0x7fe7e80704b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00041d0 tx=0x7fe7f0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.653+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7f4057f10 con 0x7fe8001037b0 2024-03-18T14:56:37.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: Reconfiguring mds.cephfs.smithi100.rutxmj (monmap changed)... 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: Reconfiguring daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3272013447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:37.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:37 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3408596131' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:56:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.878+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe800109ef0 con 0x7fe7e806e000 2024-03-18T14:56:37.879 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: Reconfiguring mds.cephfs.smithi100.rutxmj (monmap changed)... 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: Reconfiguring daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3272013447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:37.880 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:37 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3408596131' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.880+0000 7fe7fcff9700 1 -- 172.21.15.12:0/3110274056 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe800109ef0 con 0x7fe7e806e000 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: "mon" 2024-03-18T14:56:37.881 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:56:37.882 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "4/21 daemons upgraded", 2024-03-18T14:56:37.882 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mon daemons", 2024-03-18T14:56:37.882 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:56:37.882 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 msgr2=0x7fe7e80704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 0x7fe7e80704b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00041d0 tx=0x7fe7f0009040 comp rx=0 tx=0).stop 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 msgr2=0x7fe800070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009100 tx=0x7fe7f4009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 shutdown_connections 2024-03-18T14:56:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7e806e000 0x7fe7e80704b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe800102e10 0x7fe8000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 --2- 172.21.15.12:0/3110274056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8001037b0 0x7fe800070800 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.882+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 >> 172.21.15.12:0/3110274056 conn(0x7fe8000f8980 msgr2=0x7fe8001021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.883+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 shutdown_connections 2024-03-18T14:56:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.883+0000 7fe80662e700 1 -- 172.21.15.12:0/3110274056 wait complete. 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 -- 172.21.15.12:0/1611738757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c102710 msgr2=0x7f148c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 --2- 172.21.15.12:0/1611738757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c102710 0x7f148c0fbd80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1474004940 tx=0x7f1474009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 -- 172.21.15.12:0/1611738757 shutdown_connections 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 --2- 172.21.15.12:0/1611738757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c102710 0x7f148c0fbd80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 --2- 172.21.15.12:0/1611738757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c101d70 0x7f148c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 -- 172.21.15.12:0/1611738757 >> 172.21.15.12:0/1611738757 conn(0x7f148c072bc0 msgr2=0x7f148c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 -- 172.21.15.12:0/1611738757 shutdown_connections 2024-03-18T14:56:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.986+0000 7f1490bd3700 1 -- 172.21.15.12:0/1611738757 wait complete. 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 Processor -- start 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 -- start start 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c102710 0x7f148c1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1146f0 con 0x7f148c101d70 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1490bd3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c10dae0 con 0x7f148c102710 2024-03-18T14:56:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34118/0 (socket says 172.21.15.12:34118) 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 -- 172.21.15.12:0/951609139 learned_addr learned my addr 172.21.15.12:0/951609139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f1489d9b700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c102710 0x7f148c1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 -- 172.21.15.12:0/951609139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c102710 msgr2=0x7f148c1140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c102710 0x7f148c1140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 -- 172.21.15.12:0/951609139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474005040 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.987+0000 7f148a59c700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f147c002a50 tx=0x7f147c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f147c007cc0 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f147c00f040 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c10ddc0 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f147c00da30 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c10e350 con 0x7f148c101d70 2024-03-18T14:56:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.988+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c0fd4c0 con 0x7f148c101d70 2024-03-18T14:56:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.989+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f147c00dbb0 con 0x7f148c101d70 2024-03-18T14:56:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.989+0000 7f1482ffd700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 0x7f14780704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:56:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.990+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f147c08d6b0 con 0x7f148c101d70 2024-03-18T14:56:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.990+0000 7f1489d9b700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 0x7f14780704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:56:37.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.990+0000 7f1489d9b700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 0x7f14780704b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1474004890 tx=0x7f1474009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:56:37.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:37.994+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f147c05a530 con 0x7f148c101d70 2024-03-18T14:56:38.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.298+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f148c108e50 con 0x7f148c101d70 2024-03-18T14:56:38.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.299+0000 7f1482ffd700 1 -- 172.21.15.12:0/951609139 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f147c05a0c0 con 0x7f148c101d70 2024-03-18T14:56:38.300 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 msgr2=0x7f14780704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 0x7f14780704b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1474004890 tx=0x7f1474009e30 comp rx=0 tx=0).stop 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 msgr2=0x7f148c113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f147c002a50 tx=0x7f147c0046b0 comp rx=0 tx=0).stop 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 shutdown_connections 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f147806e000 0x7f14780704b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f148c101d70 0x7f148c113b60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:38.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 --2- 172.21.15.12:0/951609139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f148c102710 0x7f148c1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:56:38.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 >> 172.21.15.12:0/951609139 conn(0x7f148c072bc0 msgr2=0x7f148c0659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:56:38.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 shutdown_connections 2024-03-18T14:56:38.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:56:38.301+0000 7f1490bd3700 1 -- 172.21.15.12:0/951609139 wait complete. 2024-03-18T14:56:38.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:38.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:38.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:38.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:38.852 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi012"}]: dispatch 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi012"}]': finished 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi100"}]: dispatch 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi100"}]': finished 2024-03-18T14:56:38.853 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:38 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/951609139' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi012"}]: dispatch 2024-03-18T14:56:39.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi012"}]': finished 2024-03-18T14:56:39.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi100"}]: dispatch 2024-03-18T14:56:39.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi100"}]': finished 2024-03-18T14:56:39.046 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:38 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/951609139' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:56:39.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:39 smithi012 ceph-mon[111904]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:39.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:39 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all mon 2024-03-18T14:56:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:39 smithi100 ceph-mon[84868]: pgmap v29: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:39 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all mon 2024-03-18T14:56:40.983 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:40 smithi012 ceph-mon[111904]: Upgrade: Updating crash.smithi012 (1/2) 2024-03-18T14:56:40.984 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:40 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:40.984 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:40 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:40.984 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:40 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:40.984 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:40 smithi012 ceph-mon[111904]: Deploying daemon crash.smithi012 on smithi012 2024-03-18T14:56:41.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:40 smithi100 ceph-mon[84868]: Upgrade: Updating crash.smithi012 (1/2) 2024-03-18T14:56:41.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:40 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:41.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:40 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi012", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:41.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:40 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:41.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:40 smithi100 ceph-mon[84868]: Deploying daemon crash.smithi012 on smithi012 2024-03-18T14:56:41.855 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:41 smithi012 ceph-mon[111904]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:42.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:41 smithi100 ceph-mon[84868]: pgmap v30: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:44.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:43 smithi100 ceph-mon[84868]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:43 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:44.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:43 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:44.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:43 smithi012 ceph-mon[111904]: pgmap v31: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:44.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:43 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:44.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:43 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:45.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:45 smithi100 ceph-mon[84868]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:45.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:45.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:45.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:45 smithi012 ceph-mon[111904]: pgmap v32: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi100", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-18T14:56:45.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:46.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:46 smithi100 ceph-mon[84868]: Upgrade: Updating crash.smithi100 (2/2) 2024-03-18T14:56:46.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:46 smithi100 ceph-mon[84868]: Deploying daemon crash.smithi100 on smithi100 2024-03-18T14:56:46.869 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:46 smithi012 ceph-mon[111904]: Upgrade: Updating crash.smithi100 (2/2) 2024-03-18T14:56:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:46 smithi012 ceph-mon[111904]: Deploying daemon crash.smithi100 on smithi100 2024-03-18T14:56:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:56:48.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:47 smithi100 ceph-mon[84868]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:48.143 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:47 smithi012 ceph-mon[111904]: pgmap v33: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:49.793 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:49 smithi100 ceph-mon[84868]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:49.793 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:49.793 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:49.793 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:49.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:49 smithi012 ceph-mon[111904]: pgmap v34: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:49.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:49.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:49.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:51.965 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:51 smithi012 ceph-mon[111904]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:52.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:51 smithi100 ceph-mon[84868]: pgmap v35: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:52.979 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:52 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:52.979 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:52 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:53.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:52 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:53.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:52 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:53 smithi012 ceph-mon[111904]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:54.032 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:53 smithi100 ceph-mon[84868]: pgmap v36: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:54.032 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:54.032 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:55.428 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:55 smithi100 ceph-mon[84868]: pgmap v37: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:55.428 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:55 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:55.428 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:55 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:55.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:55 smithi012 ceph-mon[111904]: pgmap v37: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:55.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:55 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:55 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:56 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:56 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:56 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:56 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:56 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:56 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:56 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:56.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:56 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:57.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:57 smithi012 ceph-mon[111904]: pgmap v38: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:57.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:57 smithi100 ceph-mon[84868]: pgmap v38: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:56:59.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: pgmap v39: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi012"}]: dispatch 2024-03-18T14:56:59.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi012"}]': finished 2024-03-18T14:56:59.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi100"}]: dispatch 2024-03-18T14:56:59.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi100"}]': finished 2024-03-18T14:56:59.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:56:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: pgmap v39: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:56:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi012"}]: dispatch 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi012"}]': finished 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi100"}]: dispatch 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi100"}]': finished 2024-03-18T14:56:59.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:56:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-18T14:57:00.668 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:00 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all crash 2024-03-18T14:57:00.668 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:00 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-18T14:57:00.668 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:00 smithi012 ceph-mon[111904]: Upgrade: osd.0 is safe to restart 2024-03-18T14:57:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:00 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all crash 2024-03-18T14:57:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:00 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-18T14:57:00.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:00 smithi100 ceph-mon[84868]: Upgrade: osd.0 is safe to restart 2024-03-18T14:57:01.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: pgmap v40: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: Upgrade: Updating osd.0 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: Deploying daemon osd.0 on smithi012 2024-03-18T14:57:01.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:02.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: pgmap v40: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: Upgrade: Updating osd.0 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: Deploying daemon osd.0 on smithi012 2024-03-18T14:57:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:03.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:03 smithi012 ceph-mon[111904]: pgmap v41: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:57:04.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:03 smithi100 ceph-mon[84868]: pgmap v41: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:57:04.388 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:04 smithi012 systemd[1]: Stopping Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:57:04.741 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:04 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[47480]: 2024-03-18T14:57:04.387+0000 7f22d40a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:57:04.742 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:04 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[47480]: 2024-03-18T14:57:04.387+0000 7f22d40a9700 -1 osd.0 48 *** Got signal Terminated *** 2024-03-18T14:57:04.742 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:04 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[47480]: 2024-03-18T14:57:04.387+0000 7f22d40a9700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T14:57:05.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:04 smithi100 ceph-mon[84868]: osd.0 marked itself down and dead 2024-03-18T14:57:05.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:04 smithi012 ceph-mon[111904]: osd.0 marked itself down and dead 2024-03-18T14:57:05.589 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:05 smithi012 podman[122811]: 2024-03-18 14:57:05.308619511 +0000 UTC m=+1.041324677 container died 7bd6dd299fdb8995c9d47456505a204f2759162d51b7aa1e983973de2a407915 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=quincy-6716c9f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-18T14:57:05.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:05 smithi012 ceph-mon[111904]: pgmap v42: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:05.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:05 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:57:05.948 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:05 smithi012 ceph-mon[111904]: osdmap e49: 6 total, 5 up, 6 in 2024-03-18T14:57:05.948 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:05 smithi012 podman[122811]: 2024-03-18 14:57:05.721584134 +0000 UTC m=+1.454289295 container cleanup 7bd6dd299fdb8995c9d47456505a204f2759162d51b7aa1e983973de2a407915 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=quincy-6716c9f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-18T14:57:05.948 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:05 smithi012 bash[122811]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0 2024-03-18T14:57:06.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:05 smithi100 ceph-mon[84868]: pgmap v42: 65 pgs: 65 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:06.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:05 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:57:06.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:05 smithi100 ceph-mon[84868]: osdmap e49: 6 total, 5 up, 6 in 2024-03-18T14:57:06.241 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:05 smithi012 podman[122828]: 2024-03-18 14:57:05.948053504 +0000 UTC m=+0.637838420 container remove 7bd6dd299fdb8995c9d47456505a204f2759162d51b7aa1e983973de2a407915 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, RELEASE=quincy-6716c9f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-18T14:57:06.651 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.29554551 +0000 UTC m=+0.018218336 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:06.651 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.424019866 +0000 UTC m=+0.146692689 container create 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:57:06.651 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.651549485 +0000 UTC m=+0.374222312 container init 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-18T14:57:06.972 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:06 smithi012 ceph-mon[111904]: osdmap e50: 6 total, 5 up, 6 in 2024-03-18T14:57:06.973 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.654632297 +0000 UTC m=+0.377305132 container start 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:57:06.973 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.743302039 +0000 UTC m=+0.465974872 container attach 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-18T14:57:07.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:06 smithi100 ceph-mon[84868]: osdmap e50: 6 total, 5 up, 6 in 2024-03-18T14:57:07.241 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:06 smithi012 podman[122921]: 2024-03-18 14:57:06.972865882 +0000 UTC m=+0.695538715 container died 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T14:57:07.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:07 smithi012 ceph-mon[111904]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:57:07.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:07 smithi012 podman[122947]: 2024-03-18 14:57:07.655602698 +0000 UTC m=+0.677632522 container remove 4d79d961d7aad80892d9eee0e3da30415ecadb85feda1f9b577f050f38d58f8b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-18T14:57:07.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:07 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-18T14:57:07.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:07 smithi012 systemd[1]: Stopped Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:57:07.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:07 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0.service: Consumed 1min 3.285s CPU time. 2024-03-18T14:57:07.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:07 smithi012 systemd[1]: Starting Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:57:08.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:07 smithi100 ceph-mon[84868]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:57:08.384 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.015889388 +0000 UTC m=+0.019379017 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:08.384 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.139937907 +0000 UTC m=+0.143427519 container create bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-18T14:57:08.384 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.384396051 +0000 UTC m=+0.387885670 container init bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 -- 172.21.15.12:0/2640464355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 msgr2=0x7f0e6c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 --2- 172.21.15.12:0/2640464355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c1031d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c004940 tx=0x7f0e5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 -- 172.21.15.12:0/2640464355 shutdown_connections 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 --2- 172.21.15.12:0/2640464355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 --2- 172.21.15.12:0/2640464355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c1031d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 -- 172.21.15.12:0/2640464355 >> 172.21.15.12:0/2640464355 conn(0x7f0e6c0f89b0 msgr2=0x7f0e6c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 -- 172.21.15.12:0/2640464355 shutdown_connections 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.411+0000 7f0e71eea700 1 -- 172.21.15.12:0/2640464355 wait complete. 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 Processor -- start 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 -- start start 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c070d90 con 0x7f0e6c102e00 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e71eea700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c070ed0 con 0x7f0e6c1037a0 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55500/0 (socket says 172.21.15.12:55500) 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e6b7fe700 1 -- 172.21.15.12:0/414603833 learned_addr learned my addr 172.21.15.12:0/414603833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.412+0000 7f0e6affd700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e6b7fe700 1 -- 172.21.15.12:0/414603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 msgr2=0x7f0e6c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e6b7fe700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e6b7fe700 1 -- 172.21.15.12:0/414603833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c005040 con 0x7f0e6c102e00 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e6affd700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e6b7fe700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009b30 tx=0x7f0e5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c00e070 con 0x7f0e6c102e00 2024-03-18T14:57:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c071120 con 0x7f0e6c102e00 2024-03-18T14:57:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e5c013cd0 con 0x7f0e6c102e00 2024-03-18T14:57:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c00b720 con 0x7f0e6c102e00 2024-03-18T14:57:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.413+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c11a1e0 con 0x7f0e6c102e00 2024-03-18T14:57:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.414+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c100110 con 0x7f0e6c102e00 2024-03-18T14:57:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.415+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f0e5c01cab0 con 0x7f0e6c102e00 2024-03-18T14:57:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.415+0000 7f0e63fff700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 0x7f0e580704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.415+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f0e5c08f200 con 0x7f0e6c102e00 2024-03-18T14:57:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.415+0000 7f0e6affd700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 0x7f0e580704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.416+0000 7f0e6affd700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 0x7f0e580704b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004600 tx=0x7f0e5400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.420+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e5c0585d0 con 0x7f0e6c102e00 2024-03-18T14:57:08.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.646+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e6c04bed0 con 0x7f0e5806e000 2024-03-18T14:57:08.647 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.387856479 +0000 UTC m=+0.391346102 container start bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-03-18T14:57:08.648 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.459347213 +0000 UTC m=+0.462836837 container attach bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-18T14:57:08.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.648+0000 7f0e63fff700 1 -- 172.21.15.12:0/414603833 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0e6c04bed0 con 0x7f0e5806e000 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 msgr2=0x7f0e580704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 0x7f0e580704b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004600 tx=0x7f0e5400b040 comp rx=0 tx=0).stop 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 msgr2=0x7f0e6c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009b30 tx=0x7f0e5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 shutdown_connections 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0e5806e000 0x7f0e580704b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e6c102e00 0x7f0e6c0702c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 --2- 172.21.15.12:0/414603833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e6c1037a0 0x7f0e6c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.650+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 >> 172.21.15.12:0/414603833 conn(0x7f0e6c0f89b0 msgr2=0x7f0e6c1073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.651+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 shutdown_connections 2024-03-18T14:57:08.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.651+0000 7f0e71eea700 1 -- 172.21.15.12:0/414603833 wait complete. 2024-03-18T14:57:08.662 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:57:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 -- 172.21.15.12:0/3468002801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41026b0 msgr2=0x7f03e40fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/3468002801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41026b0 0x7f03e40fbd20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03e0004940 tx=0x7f03e0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 -- 172.21.15.12:0/3468002801 shutdown_connections 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/3468002801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e41026b0 0x7f03e40fbd20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/3468002801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4101d10 0x7f03e41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.758+0000 7f03ec0f2700 1 -- 172.21.15.12:0/3468002801 >> 172.21.15.12:0/3468002801 conn(0x7f03e40f7880 msgr2=0x7f03e40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.759+0000 7f03ec0f2700 1 -- 172.21.15.12:0/3468002801 shutdown_connections 2024-03-18T14:57:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.759+0000 7f03ec0f2700 1 -- 172.21.15.12:0/3468002801 wait complete. 2024-03-18T14:57:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.759+0000 7f03ec0f2700 1 Processor -- start 2024-03-18T14:57:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.759+0000 7f03ec0f2700 1 -- start start 2024-03-18T14:57:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03ec0f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03ec0f2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e41026b0 0x7f03e410f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03ec0f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e410f9d0 con 0x7f03e4101d10 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03ec0f2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e410fb10 con 0x7f03e41026b0 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40846/0 (socket says 172.21.15.12:40846) 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 -- 172.21.15.12:0/1018971518 learned_addr learned my addr 172.21.15.12:0/1018971518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e968d700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e41026b0 0x7f03e410f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 -- 172.21.15.12:0/1018971518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e41026b0 msgr2=0x7f03e410f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e41026b0 0x7f03e410f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 -- 172.21.15.12:0/1018971518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e0005040 con 0x7f03e4101d10 2024-03-18T14:57:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.760+0000 7f03e9e8e700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c830 tx=0x7f03d8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.762+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d80106f0 con 0x7f03e4101d10 2024-03-18T14:57:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.762+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03d8010d20 con 0x7f03e4101d10 2024-03-18T14:57:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.762+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4117eb0 con 0x7f03e4101d10 2024-03-18T14:57:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.762+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d80113a0 con 0x7f03e4101d10 2024-03-18T14:57:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.762+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e41183e0 con 0x7f03e4101d10 2024-03-18T14:57:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.763+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e40fd460 con 0x7f03e4101d10 2024-03-18T14:57:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.763+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f03d8012400 con 0x7f03e4101d10 2024-03-18T14:57:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.764+0000 7f03d67fc700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 0x7f03d00704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.764+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f03d808d690 con 0x7f03e4101d10 2024-03-18T14:57:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.764+0000 7f03e968d700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 0x7f03d00704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.764+0000 7f03e968d700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 0x7f03d00704b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03e4066890 tx=0x7f03e0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.769+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f03d8056c10 con 0x7f03e4101d10 2024-03-18T14:57:08.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-18T14:57:08.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 bash[123049]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate[123064]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-18T14:57:08.993 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:08 smithi012 podman[123049]: 2024-03-18 14:57:08.865550424 +0000 UTC m=+0.869040027 container died bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T14:57:09.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:08.999+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03e404bed0 con 0x7f03d006e000 2024-03-18T14:57:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.000+0000 7f03d67fc700 1 -- 172.21.15.12:0/1018971518 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f03e404bed0 con 0x7f03d006e000 2024-03-18T14:57:09.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 msgr2=0x7f03d00704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 0x7f03d00704b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03e4066890 tx=0x7f03e0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 msgr2=0x7f03e410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c830 tx=0x7f03d8004470 comp rx=0 tx=0).stop 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 shutdown_connections 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f03d006e000 0x7f03d00704b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03e4101d10 0x7f03e410ee40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 --2- 172.21.15.12:0/1018971518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e41026b0 0x7f03e410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.002+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 >> 172.21.15.12:0/1018971518 conn(0x7f03e40f7880 msgr2=0x7f03e40f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.003+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 shutdown_connections 2024-03-18T14:57:09.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.003+0000 7f03ec0f2700 1 -- 172.21.15.12:0/1018971518 wait complete. 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 -- 172.21.15.12:0/2605580014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c102fd0 msgr2=0x7fd77c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 --2- 172.21.15.12:0/2605580014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c102fd0 0x7fd77c1033a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd778004940 tx=0x7fd778009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 -- 172.21.15.12:0/2605580014 shutdown_connections 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 --2- 172.21.15.12:0/2605580014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 --2- 172.21.15.12:0/2605580014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c102fd0 0x7fd77c1033a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.105+0000 7fd783a08700 1 -- 172.21.15.12:0/2605580014 >> 172.21.15.12:0/2605580014 conn(0x7fd77c071530 msgr2=0x7fd77c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 -- 172.21.15.12:0/2605580014 shutdown_connections 2024-03-18T14:57:09.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 -- 172.21.15.12:0/2605580014 wait complete. 2024-03-18T14:57:09.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 Processor -- start 2024-03-18T14:57:09.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 -- start start 2024-03-18T14:57:09.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c102fd0 0x7fd77c0fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0fe670 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.106+0000 7fd783a08700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0fd1a0 con 0x7fd77c102fd0 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40864/0 (socket says 172.21.15.12:40864) 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 -- 172.21.15.12:0/1394424180 learned_addr learned my addr 172.21.15.12:0/1394424180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd780fa3700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c102fd0 0x7fd77c0fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 -- 172.21.15.12:0/1394424180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c102fd0 msgr2=0x7fd77c0fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c102fd0 0x7fd77c0fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 -- 172.21.15.12:0/1394424180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd778005040 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd7817a4700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b3b0 tx=0x7fd778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77800e040 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c0fd420 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd778013420 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77801c630 con 0x7fd77c0fb540 2024-03-18T14:57:09.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.107+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c11a070 con 0x7fd77c0fb540 2024-03-18T14:57:09.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.108+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c100270 con 0x7fd77c0fb540 2024-03-18T14:57:09.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.109+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fd778013590 con 0x7fd77c0fb540 2024-03-18T14:57:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.109+0000 7fd771ffb700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 0x7fd7680704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.109+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fd77808eaf0 con 0x7fd77c0fb540 2024-03-18T14:57:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.109+0000 7fd780fa3700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 0x7fd7680704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.110+0000 7fd780fa3700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 0x7fd7680704c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0fde50 tx=0x7fd76c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.114+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd778057ec0 con 0x7fd77c0fb540 2024-03-18T14:57:09.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:08 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T14:57:09.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.334+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd77c0fdc70 con 0x7fd76806e010 2024-03-18T14:57:09.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.345+0000 7fd771ffb700 1 -- 172.21.15.12:0/1394424180 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd77c0fdc70 con 0x7fd76806e010 2024-03-18T14:57:09.346 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:57:09.346 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (2m) 15s ago 13m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:57:09.346 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (26s) 15s ago 14m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:57:09.346 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (21s) 17s ago 13m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (2m) 15s ago 13m 77.8M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (8m) 15s ago 8m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (8m) 15s ago 8m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (8m) 17s ago 8m 16.7M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (8m) 17s ago 8m 13.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (4m) 15s ago 15m 495M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (3m) 17s ago 12m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (93s) 15s ago 15m 49.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (67s) 17s ago 12m 38.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (3m) 15s ago 14m 15.5M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (3m) 17s ago 13m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (12m) 15s ago 12m 257M 1559M 17.2.7-684-g6716c9f3 19857db3290c 7bd6dd299fdb 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (11m) 15s ago 11m 271M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (11m) 15s ago 11m 221M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:57:09.347 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (10m) 17s ago 10m 291M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:57:09.348 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (10m) 17s ago 10m 260M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:57:09.348 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (10m) 17s ago 9m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:57:09.348 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (3m) 15s ago 13m 61.9M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:57:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 msgr2=0x7fd7680704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 0x7fd7680704c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0fde50 tx=0x7fd76c00b040 comp rx=0 tx=0).stop 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 msgr2=0x7fd77c0fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b3b0 tx=0x7fd778009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 shutdown_connections 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd76806e010 0x7fd7680704c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd77c0fb540 0x7fd77c0fc720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 --2- 172.21.15.12:0/1394424180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd77c102fd0 0x7fd77c0fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 >> 172.21.15.12:0/1394424180 conn(0x7fd77c071530 msgr2=0x7fd77c0fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 shutdown_connections 2024-03-18T14:57:09.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.347+0000 7fd783a08700 1 -- 172.21.15.12:0/1394424180 wait complete. 2024-03-18T14:57:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 -- 172.21.15.12:0/614632360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 msgr2=0x7ff9801031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 --2- 172.21.15.12:0/614632360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9801031d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff97c004940 tx=0x7ff97c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 -- 172.21.15.12:0/614632360 shutdown_connections 2024-03-18T14:57:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 --2- 172.21.15.12:0/614632360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff9800fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 --2- 172.21.15.12:0/614632360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9801031d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.449+0000 7ff98775a700 1 -- 172.21.15.12:0/614632360 >> 172.21.15.12:0/614632360 conn(0x7ff9800f89b0 msgr2=0x7ff9800fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 -- 172.21.15.12:0/614632360 shutdown_connections 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 -- 172.21.15.12:0/614632360 wait complete. 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 Processor -- start 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 -- start start 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff980070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800721c0 con 0x7ff980102e00 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff98775a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980070d40 con 0x7ff9801037a0 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff9854f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff9854f6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40876/0 (socket says 172.21.15.12:40876) 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.450+0000 7ff9854f6700 1 -- 172.21.15.12:0/3931654174 learned_addr learned my addr 172.21.15.12:0/3931654174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:09.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff984cf5700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff980070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff9854f6700 1 -- 172.21.15.12:0/3931654174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 msgr2=0x7ff980070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff9854f6700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff980070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff9854f6700 1 -- 172.21.15.12:0/3931654174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c005040 con 0x7ff980102e00 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff984cf5700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff980070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff9854f6700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009b30 tx=0x7ff97c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97c00e070 con 0x7ff980102e00 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff97c013cd0 con 0x7ff980102e00 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff980070fc0 con 0x7ff980102e00 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97c00b720 con 0x7ff980102e00 2024-03-18T14:57:09.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.451+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff980119ff0 con 0x7ff980102e00 2024-03-18T14:57:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.452+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980100110 con 0x7ff980102e00 2024-03-18T14:57:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.453+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7ff97c01cab0 con 0x7ff980102e00 2024-03-18T14:57:09.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.453+0000 7ff975ffb700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 0x7ff96c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.453+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7ff97c08f1c0 con 0x7ff980102e00 2024-03-18T14:57:09.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.454+0000 7ff984cf5700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 0x7ff96c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.454+0000 7ff984cf5700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 0x7ff96c0704c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff970004600 tx=0x7ff97000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.458+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff97c058630 con 0x7ff980102e00 2024-03-18T14:57:09.741 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:09 smithi012 podman[123246]: 2024-03-18 14:57:09.581054628 +0000 UTC m=+0.706285392 container remove bcaa23a505179822baf4ecead6fb0ff6ef926bd38ccadaf386460bdf12705344 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T14:57:09.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.773+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff980109f90 con 0x7ff980102e00 2024-03-18T14:57:09.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.773+0000 7ff975ffb700 1 -- 172.21.15.12:0/3931654174 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7ff97c004490 con 0x7ff980102e00 2024-03-18T14:57:09.774 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 5 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:09.775 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 9, 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:57:09.776 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:57:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 msgr2=0x7ff96c0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 0x7ff96c0704c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff970004600 tx=0x7ff97000b040 comp rx=0 tx=0).stop 2024-03-18T14:57:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 msgr2=0x7ff9800702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009b30 tx=0x7ff97c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 shutdown_connections 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff96c06e010 0x7ff96c0704c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff980102e00 0x7ff9800702c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 --2- 172.21.15.12:0/3931654174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9801037a0 0x7ff980070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 >> 172.21.15.12:0/3931654174 conn(0x7ff9800f89b0 msgr2=0x7ff9800fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 shutdown_connections 2024-03-18T14:57:09.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.775+0000 7ff98775a700 1 -- 172.21.15.12:0/3931654174 wait complete. 2024-03-18T14:57:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 -- 172.21.15.12:0/1695096198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 msgr2=0x7fe6340fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/1695096198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fb990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe628004940 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 -- 172.21.15.12:0/1695096198 shutdown_connections 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/1695096198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fb990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/1695096198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 0x7fe6341033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 -- 172.21.15.12:0/1695096198 >> 172.21.15.12:0/1695096198 conn(0x7fe634071530 msgr2=0x7fe634071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 -- 172.21.15.12:0/1695096198 shutdown_connections 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.879+0000 7fe63a0e6700 1 -- 172.21.15.12:0/1695096198 wait complete. 2024-03-18T14:57:09.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 Processor -- start 2024-03-18T14:57:09.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 -- start start 2024-03-18T14:57:09.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 0x7fe6340fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340fe700 con 0x7fe6340fb540 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe63a0e6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340fd1a0 con 0x7fe634102fd0 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.880+0000 7fe6337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40896/0 (socket says 172.21.15.12:40896) 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 -- 172.21.15.12:0/3333170763 learned_addr learned my addr 172.21.15.12:0/3333170763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe632ffd700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 0x7fe6340fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 -- 172.21.15.12:0/3333170763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 msgr2=0x7fe6340fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 0x7fe6340fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 -- 172.21.15.12:0/3333170763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628005040 con 0x7fe6340fb540 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe6337fe700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b790 tx=0x7fe624009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62400f6f0 con 0x7fe6340fb540 2024-03-18T14:57:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe62400fd20 con 0x7fe6340fb540 2024-03-18T14:57:09.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6240103a0 con 0x7fe6340fb540 2024-03-18T14:57:09.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6340fd480 con 0x7fe6340fb540 2024-03-18T14:57:09.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.881+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63411a1b0 con 0x7fe6340fb540 2024-03-18T14:57:09.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.882+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634100270 con 0x7fe6340fb540 2024-03-18T14:57:09.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.883+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fe624011400 con 0x7fe6340fb540 2024-03-18T14:57:09.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.883+0000 7fe61bfff700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 0x7fe61c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:09.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.883+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fe62408d590 con 0x7fe6340fb540 2024-03-18T14:57:09.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.883+0000 7fe632ffd700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 0x7fe61c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:09.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.884+0000 7fe632ffd700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 0x7fe61c0704c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6340fde50 tx=0x7fe628009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:09.888+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe624056a50 con 0x7fe6340fb540 2024-03-18T14:57:10.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:09 smithi012 ceph-mon[111904]: pgmap v46: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:10.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:09 smithi012 ceph-mon[111904]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:10.041 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:09 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3931654174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:10.041 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:09 smithi012 podman[123354]: 2024-03-18 14:57:09.704313584 +0000 UTC m=+0.020954630 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:10.041 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:09 smithi012 podman[123354]: 2024-03-18 14:57:09.832194183 +0000 UTC m=+0.148835204 container create f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-18T14:57:10.041 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 podman[123354]: 2024-03-18 14:57:10.018181475 +0000 UTC m=+0.334822512 container init f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T14:57:10.041 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 podman[123354]: 2024-03-18 14:57:10.021311011 +0000 UTC m=+0.337952033 container start f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T14:57:10.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.143+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe63410a0c0 con 0x7fe6340fb540 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.144+0000 7fe61bfff700 1 -- 172.21.15.12:0/3333170763 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fe62405a110 con 0x7fe6340fb540 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:57:10.145 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:57:10.146 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:57:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:57:10.148 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:10.149 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:57:10.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:10.150 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:10.150 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 msgr2=0x7fe61c0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 0x7fe61c0704c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6340fde50 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 msgr2=0x7fe6340fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b790 tx=0x7fe624009250 comp rx=0 tx=0).stop 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 shutdown_connections 2024-03-18T14:57:10.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe61c06e010 0x7fe61c0704c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6340fb540 0x7fe6340fc720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 --2- 172.21.15.12:0/3333170763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe634102fd0 0x7fe6340fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 >> 172.21.15.12:0/3333170763 conn(0x7fe634071530 msgr2=0x7fe6340fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 shutdown_connections 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.146+0000 7fe63a0e6700 1 -- 172.21.15.12:0/3333170763 wait complete. 2024-03-18T14:57:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:57:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 -- 172.21.15.12:0/127956105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c0fb420 msgr2=0x7f6b6c0fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 --2- 172.21.15.12:0/127956105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c0fb870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c004940 tx=0x7f6b5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 -- 172.21.15.12:0/127956105 shutdown_connections 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 --2- 172.21.15.12:0/127956105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c0fb870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 --2- 172.21.15.12:0/127956105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c102f80 0x7f6b6c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.257+0000 7f6b74152700 1 -- 172.21.15.12:0/127956105 >> 172.21.15.12:0/127956105 conn(0x7f6b6c072380 msgr2=0x7f6b6c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.258+0000 7f6b74152700 1 -- 172.21.15.12:0/127956105 shutdown_connections 2024-03-18T14:57:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.258+0000 7f6b74152700 1 -- 172.21.15.12:0/127956105 wait complete. 2024-03-18T14:57:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.258+0000 7f6b74152700 1 Processor -- start 2024-03-18T14:57:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.258+0000 7f6b74152700 1 -- start start 2024-03-18T14:57:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.258+0000 7f6b74152700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b74152700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b74152700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c114500 con 0x7f6b6c102f80 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b74152700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c10db70 con 0x7f6b6c0fb420 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40922/0 (socket says 172.21.15.12:40922) 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 -- 172.21.15.12:0/787851611 learned_addr learned my addr 172.21.15.12:0/787851611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b71eee700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 -- 172.21.15.12:0/787851611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c0fb420 msgr2=0x7f6b6c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 -- 172.21.15.12:0/787851611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c005040 con 0x7f6b6c102f80 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b716ed700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b3b0 tx=0x7f6b5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c00e040 con 0x7f6b6c102f80 2024-03-18T14:57:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b5c013420 con 0x7f6b6c102f80 2024-03-18T14:57:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c01c630 con 0x7f6b6c102f80 2024-03-18T14:57:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.259+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c10ddf0 con 0x7f6b6c102f80 2024-03-18T14:57:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.260+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c10e330 con 0x7f6b6c102f80 2024-03-18T14:57:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.261+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7f6b5c013590 con 0x7f6b6c102f80 2024-03-18T14:57:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.261+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c105310 con 0x7f6b6c102f80 2024-03-18T14:57:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.261+0000 7f6b627fc700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 0x7f6b580704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.261+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6b5c08ea10 con 0x7f6b6c102f80 2024-03-18T14:57:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.261+0000 7f6b71eee700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 0x7f6b580704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.262+0000 7f6b71eee700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 0x7f6b580704c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b68004f40 tx=0x7f6b6800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.266+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b5c057e80 con 0x7f6b6c102f80 2024-03-18T14:57:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:09 smithi100 ceph-mon[84868]: pgmap v46: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:09 smithi100 ceph-mon[84868]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:10.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:09 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3931654174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:10.491 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-osd[123396]: -- 172.21.15.12:0/586508167 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ab183b680 con 0x563ab1805000 2024-03-18T14:57:10.492 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 bash[123354]: f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb 2024-03-18T14:57:10.492 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 systemd[1]: Started Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:57:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.496+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b6c10efc0 con 0x7f6b5806e010 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.497+0000 7f6b627fc700 1 -- 172.21.15.12:0/787851611 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6b6c10efc0 con 0x7f6b5806e010 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:57:10.498 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "6/21 daemons upgraded", 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:57:10.499 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:57:10.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 msgr2=0x7f6b580704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 0x7f6b580704c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b68004f40 tx=0x7f6b6800b040 comp rx=0 tx=0).stop 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 msgr2=0x7f6b6c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b3b0 tx=0x7f6b5c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 shutdown_connections 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b5806e010 0x7f6b580704c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b6c0fb420 0x7f6b6c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 --2- 172.21.15.12:0/787851611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b6c102f80 0x7f6b6c113eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 >> 172.21.15.12:0/787851611 conn(0x7f6b6c072380 msgr2=0x7f6b6c0f9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 shutdown_connections 2024-03-18T14:57:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.500+0000 7f6b74152700 1 -- 172.21.15.12:0/787851611 wait complete. 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 -- 172.21.15.12:0/798720133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 msgr2=0x7fad880fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 --2- 172.21.15.12:0/798720133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad880fbd80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad70004940 tx=0x7fad70009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 -- 172.21.15.12:0/798720133 shutdown_connections 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 --2- 172.21.15.12:0/798720133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad880fbd80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 --2- 172.21.15.12:0/798720133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 0x7fad88102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 -- 172.21.15.12:0/798720133 >> 172.21.15.12:0/798720133 conn(0x7fad88072bc0 msgr2=0x7fad88072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:10.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 -- 172.21.15.12:0/798720133 shutdown_connections 2024-03-18T14:57:10.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.606+0000 7fad8d650700 1 -- 172.21.15.12:0/798720133 wait complete. 2024-03-18T14:57:10.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 Processor -- start 2024-03-18T14:57:10.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 -- start start 2024-03-18T14:57:10.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 0x7fad8810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810f6d0 con 0x7fad88102710 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad8d650700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810f810 con 0x7fad88101d70 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40946/0 (socket says 172.21.15.12:40946) 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 -- 172.21.15.12:0/1467846892 learned_addr learned my addr 172.21.15.12:0/1467846892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad86ffd700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 0x7fad8810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 -- 172.21.15.12:0/1467846892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 msgr2=0x7fad8810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 0x7fad8810eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.607+0000 7fad867fc700 1 -- 172.21.15.12:0/1467846892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70005040 con 0x7fad88102710 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad867fc700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad70009100 tx=0x7fad70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7000e040 con 0x7fad88102710 2024-03-18T14:57:10.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad70009af0 con 0x7fad88102710 2024-03-18T14:57:10.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88117bb0 con 0x7fad88102710 2024-03-18T14:57:10.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad7001c3f0 con 0x7fad88102710 2024-03-18T14:57:10.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.608+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad881180c0 con 0x7fad88102710 2024-03-18T14:57:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.609+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88104240 con 0x7fad88102710 2024-03-18T14:57:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.610+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 39) v1 ==== 91943+0+0 (secure 0 0 0) 0x7fad7000baf0 con 0x7fad88102710 2024-03-18T14:57:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.610+0000 7fad7f7fe700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 0x7fad740704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.610+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fad7008eb20 con 0x7fad88102710 2024-03-18T14:57:10.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.610+0000 7fad86ffd700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 0x7fad740704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:10.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.611+0000 7fad86ffd700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 0x7fad740704b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad80004eb0 tx=0x7fad8000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.615+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad70057ef0 con 0x7fad88102710 2024-03-18T14:57:10.816 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T14:57:10.561+0000 7fe2373307c0 -1 Falling back to public interface 2024-03-18T14:57:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.932+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fad880fd4c0 con 0x7fad88102710 2024-03-18T14:57:10.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.932+0000 7fad7f7fe700 1 -- 172.21.15.12:0/1467846892 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1216 (secure 0 0 0) 0x7fad7005b5b0 con 0x7fad88102710 2024-03-18T14:57:10.933 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded 2024-03-18T14:57:10.933 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: osd.0 (root=default,host=smithi012) is down 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-18T14:57:10.934 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-18T14:57:10.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 msgr2=0x7fad740704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 0x7fad740704b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad80004eb0 tx=0x7fad8000b040 comp rx=0 tx=0).stop 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 msgr2=0x7fad8810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad70009100 tx=0x7fad70009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 shutdown_connections 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fad7406e000 0x7fad740704b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88101d70 0x7fad8810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 --2- 172.21.15.12:0/1467846892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad88102710 0x7fad8810f080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 >> 172.21.15.12:0/1467846892 conn(0x7fad88072bc0 msgr2=0x7fad880fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.934+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 shutdown_connections 2024-03-18T14:57:10.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:10.935+0000 7fad8d650700 1 -- 172.21.15.12:0/1467846892 wait complete. 2024-03-18T14:57:11.090 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:11.091 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:11.091 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3333170763' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:57:11.091 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:11.091 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:11.091 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:10 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:11.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3333170763' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:57:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:11.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:10 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:12.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:11 smithi012 ceph-mon[111904]: pgmap v47: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:12.150 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:11 smithi012 ceph-mon[111904]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:12.151 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:11 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1467846892' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:57:12.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:11 smithi100 ceph-mon[84868]: pgmap v47: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:11 smithi100 ceph-mon[84868]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:11 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1467846892' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:57:14.093 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:13 smithi012 ceph-mon[111904]: pgmap v48: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:14.097 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:13 smithi100 ceph-mon[84868]: pgmap v48: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:15.241 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T14:57:15.103+0000 7fe2373307c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-18T14:57:16.241 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T14:57:15.770+0000 7fe2373307c0 -1 osd.0 48 log_to_monitors true 2024-03-18T14:57:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-mon[111904]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:15 smithi012 ceph-mon[111904]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T14:57:16.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:15 smithi100 ceph-mon[84868]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:16.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:15 smithi100 ceph-mon[84868]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 991 B/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: osdmap e51: 6 total, 5 up, 6 in 2024-03-18T14:57:17.433 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:57:17.434 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.434 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 991 B/s rd, 1 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T14:57:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: osdmap e51: 6 total, 5 up, 6 in 2024-03-18T14:57:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:57:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:17.707 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 14:57:17 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T14:57:17.432+0000 7fe224780700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:57:18.665 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:18 smithi012 ceph-mon[111904]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' 2024-03-18T14:57:18.665 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:18 smithi012 ceph-mon[111904]: mgrmap e40: smithi012.tmscgt(active, since 91s), standbys: smithi100.rwnrcn 2024-03-18T14:57:18.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:18 smithi100 ceph-mon[84868]: from='osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564]' entity='osd.0' 2024-03-18T14:57:18.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:18 smithi100 ceph-mon[84868]: mgrmap e40: smithi012.tmscgt(active, since 91s), standbys: smithi100.rwnrcn 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564] boot 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: osdmap e52: 6 total, 6 up, 6 in 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:57:19.545 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:19 smithi012 ceph-mon[111904]: osdmap e53: 6 total, 6 up, 6 in 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: osd.0 [v2:172.21.15.12:6802/1970485564,v1:172.21.15.12:6803/1970485564] boot 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: osdmap e52: 6 total, 6 up, 6 in 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T14:57:19.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:19 smithi100 ceph-mon[84868]: osdmap e53: 6 total, 6 up, 6 in 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: pgmap v55: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:21.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:21.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:21.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:21 smithi012 ceph-mon[111904]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-18T14:57:22.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: pgmap v55: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s; 43/216 objects degraded (19.907%) 2024-03-18T14:57:22.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:22.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:22.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:22.047 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:22.048 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:21 smithi100 ceph-mon[84868]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-18T14:57:22.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:22 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded) 2024-03-18T14:57:22.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:22 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T14:57:23.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:22 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded) 2024-03-18T14:57:23.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:22 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T14:57:23.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:23 smithi012 ceph-mon[111904]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s; 46 B/s, 0 objects/s recovering 2024-03-18T14:57:24.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:23 smithi100 ceph-mon[84868]: pgmap v56: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s; 46 B/s, 0 objects/s recovering 2024-03-18T14:57:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:25 smithi012 ceph-mon[111904]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 404 B/s wr, 2 op/s; 37 B/s, 0 objects/s recovering 2024-03-18T14:57:26.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:25 smithi100 ceph-mon[84868]: pgmap v57: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 404 B/s wr, 2 op/s; 37 B/s, 0 objects/s recovering 2024-03-18T14:57:27.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:27 smithi012 ceph-mon[111904]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 0 op/s; 35 B/s, 0 objects/s recovering 2024-03-18T14:57:28.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:27 smithi100 ceph-mon[84868]: pgmap v58: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 0 op/s; 35 B/s, 0 objects/s recovering 2024-03-18T14:57:29.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:29 smithi012 ceph-mon[111904]: pgmap v59: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 848 B/s rd, 1 op/s; 29 B/s, 0 objects/s recovering 2024-03-18T14:57:30.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:29 smithi100 ceph-mon[84868]: pgmap v59: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 848 B/s rd, 1 op/s; 29 B/s, 0 objects/s recovering 2024-03-18T14:57:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:31 smithi012 ceph-mon[111904]: pgmap v60: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 768 B/s rd, 1 op/s; 26 B/s, 0 objects/s recovering 2024-03-18T14:57:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:32.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:31 smithi100 ceph-mon[84868]: pgmap v60: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 768 B/s rd, 1 op/s; 26 B/s, 0 objects/s recovering 2024-03-18T14:57:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:34.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:33 smithi012 ceph-mon[111904]: pgmap v61: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 23 B/s, 0 objects/s recovering 2024-03-18T14:57:34.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:33 smithi100 ceph-mon[84868]: pgmap v61: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 23 B/s, 0 objects/s recovering 2024-03-18T14:57:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:35 smithi012 ceph-mon[111904]: pgmap v62: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:36.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:36.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:35 smithi100 ceph-mon[84868]: pgmap v62: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:37.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:36 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:37.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:36 smithi012 ceph-mon[111904]: Upgrade: osd.1 is safe to restart 2024-03-18T14:57:37.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:36 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-18T14:57:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:36 smithi100 ceph-mon[84868]: Upgrade: osd.1 is safe to restart 2024-03-18T14:57:38.207 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:37 smithi012 ceph-mon[111904]: pgmap v63: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:38.207 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:38.207 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:57:38.207 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:37 smithi100 ceph-mon[84868]: pgmap v63: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T14:57:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:39.061 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:38 smithi012 ceph-mon[111904]: Upgrade: Updating osd.1 2024-03-18T14:57:39.061 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:38 smithi012 ceph-mon[111904]: Deploying daemon osd.1 on smithi012 2024-03-18T14:57:39.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:38 smithi100 ceph-mon[84868]: Upgrade: Updating osd.1 2024-03-18T14:57:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:38 smithi100 ceph-mon[84868]: Deploying daemon osd.1 on smithi012 2024-03-18T14:57:40.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:39 smithi012 ceph-mon[111904]: pgmap v64: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:57:40.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:39 smithi100 ceph-mon[84868]: pgmap v64: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:57:41.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- 172.21.15.12:0/925508267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c1037b0 msgr2=0x7fd37c0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 --2- 172.21.15.12:0/925508267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c1037b0 0x7fd37c0fce20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd370004940 tx=0x7fd370009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- 172.21.15.12:0/925508267 shutdown_connections 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 --2- 172.21.15.12:0/925508267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c1037b0 0x7fd37c0fce20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 --2- 172.21.15.12:0/925508267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c102e10 0x7fd37c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- 172.21.15.12:0/925508267 >> 172.21.15.12:0/925508267 conn(0x7fd37c0f8980 msgr2=0x7fd37c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- 172.21.15.12:0/925508267 shutdown_connections 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- 172.21.15.12:0/925508267 wait complete. 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 Processor -- start 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.051+0000 7fd380b6a700 1 -- start start 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd380b6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd380b6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c1037b0 0x7fd37c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd380b6a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c114710 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd380b6a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c10db00 con 0x7fd37c1037b0 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38404/0 (socket says 172.21.15.12:38404) 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 -- 172.21.15.12:0/3703654708 learned_addr learned my addr 172.21.15.12:0/3703654708 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd379d9b700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c1037b0 0x7fd37c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 -- 172.21.15.12:0/3703654708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c1037b0 msgr2=0x7fd37c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c1037b0 0x7fd37c1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 -- 172.21.15.12:0/3703654708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370005040 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd37a59c700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c830 tx=0x7fd36c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.052+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c0106f0 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.053+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c10dde0 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.053+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd36c010d20 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.053+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c0113a0 con 0x7fd37c102e10 2024-03-18T14:57:41.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.053+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c10e370 con 0x7fd37c102e10 2024-03-18T14:57:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.054+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c1052e0 con 0x7fd37c102e10 2024-03-18T14:57:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.054+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd36c012400 con 0x7fd37c102e10 2024-03-18T14:57:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.054+0000 7fd36affd700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 0x7fd364070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.055+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fd36c08d610 con 0x7fd37c102e10 2024-03-18T14:57:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.055+0000 7fd379d9b700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 0x7fd364070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.055+0000 7fd379d9b700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 0x7fd364070390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd370004890 tx=0x7fd370009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.059+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd36c05a730 con 0x7fd37c102e10 2024-03-18T14:57:41.148 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:40 smithi012 systemd[1]: Stopping Ceph osd.1 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:57:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.289+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37c10f240 con 0x7fd36406dee0 2024-03-18T14:57:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.290+0000 7fd36affd700 1 -- 172.21.15.12:0/3703654708 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd37c10f240 con 0x7fd36406dee0 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 msgr2=0x7fd364070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 0x7fd364070390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd370004890 tx=0x7fd370009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 msgr2=0x7fd37c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c830 tx=0x7fd36c004470 comp rx=0 tx=0).stop 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 shutdown_connections 2024-03-18T14:57:41.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd36406dee0 0x7fd364070390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd37c102e10 0x7fd37c113b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 --2- 172.21.15.12:0/3703654708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd37c1037b0 0x7fd37c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 >> 172.21.15.12:0/3703654708 conn(0x7fd37c0f8980 msgr2=0x7fd37c1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 shutdown_connections 2024-03-18T14:57:41.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.292+0000 7fd380b6a700 1 -- 172.21.15.12:0/3703654708 wait complete. 2024-03-18T14:57:41.303 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 -- 172.21.15.12:0/2874567952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 msgr2=0x7f06740fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 --2- 172.21.15.12:0/2874567952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f06740fce10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0664004940 tx=0x7f0664009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 -- 172.21.15.12:0/2874567952 shutdown_connections 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 --2- 172.21.15.12:0/2874567952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f06740fce10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 --2- 172.21.15.12:0/2874567952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 0x7f06741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.396+0000 7f067bf9a700 1 -- 172.21.15.12:0/2874567952 >> 172.21.15.12:0/2874567952 conn(0x7f06740f89b0 msgr2=0x7f06740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 -- 172.21.15.12:0/2874567952 shutdown_connections 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 -- 172.21.15.12:0/2874567952 wait complete. 2024-03-18T14:57:41.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 Processor -- start 2024-03-18T14:57:41.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 -- start start 2024-03-18T14:57:41.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 0x7f0674113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674114510 con 0x7f06741037a0 2024-03-18T14:57:41.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f067bf9a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067410db10 con 0x7f0674102e00 2024-03-18T14:57:41.399 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:41 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[54754]: 2024-03-18T14:57:41.146+0000 7f1bfd5eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:57:41.399 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:41 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[54754]: 2024-03-18T14:57:41.146+0000 7f1bfd5eb700 -1 osd.1 53 *** Got signal Terminated *** 2024-03-18T14:57:41.399 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:41 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[54754]: 2024-03-18T14:57:41.146+0000 7f1bfd5eb700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T14:57:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f0679535700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f0679535700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38424/0 (socket says 172.21.15.12:38424) 2024-03-18T14:57:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.397+0000 7f0679535700 1 -- 172.21.15.12:0/879449281 learned_addr learned my addr 172.21.15.12:0/879449281 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f0679d36700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 0x7f0674113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f0679535700 1 -- 172.21.15.12:0/879449281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 msgr2=0x7f0674113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f0679535700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 0x7f0674113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f0679535700 1 -- 172.21.15.12:0/879449281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664005040 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f0679535700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0664009180 tx=0x7f0664009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066400e040 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067410dd90 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066400bbc0 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066401c3f0 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.398+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067410e350 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.399+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06741052d0 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.400+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f066400bd30 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.400+0000 7f066a7fc700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 0x7f0660070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.400+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f066408e960 con 0x7f06741037a0 2024-03-18T14:57:41.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.400+0000 7f0679d36700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 0x7f0660070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.401+0000 7f0679d36700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 0x7f0660070390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0670007fd0 tx=0x7f0670009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.405+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f066405baa0 con 0x7f06741037a0 2024-03-18T14:57:41.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.631+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0674113650 con 0x7f066006dee0 2024-03-18T14:57:41.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.632+0000 7f066a7fc700 1 -- 172.21.15.12:0/879449281 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0674113650 con 0x7f066006dee0 2024-03-18T14:57:41.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 msgr2=0x7f0660070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 0x7f0660070390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0670007fd0 tx=0x7f0670009040 comp rx=0 tx=0).stop 2024-03-18T14:57:41.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 msgr2=0x7f0674113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0664009180 tx=0x7f0664009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 shutdown_connections 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f066006dee0 0x7f0660070390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0674102e00 0x7f0674113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 --2- 172.21.15.12:0/879449281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06741037a0 0x7f0674113ec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 >> 172.21.15.12:0/879449281 conn(0x7f06740f89b0 msgr2=0x7f06741021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 shutdown_connections 2024-03-18T14:57:41.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.635+0000 7f067bf9a700 1 -- 172.21.15.12:0/879449281 wait complete. 2024-03-18T14:57:41.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a00a1d700 1 -- 172.21.15.12:0/2068801092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99ec00bcb0 con 0x7f99fc102e10 2024-03-18T14:57:41.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 -- 172.21.15.12:0/2068801092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc102e10 msgr2=0x7f99fc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 --2- 172.21.15.12:0/2068801092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc102e10 0x7f99fc1031e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99ec004940 tx=0x7f99ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 -- 172.21.15.12:0/2068801092 shutdown_connections 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 --2- 172.21.15.12:0/2068801092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc1037b0 0x7f99fc0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 --2- 172.21.15.12:0/2068801092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc102e10 0x7f99fc1031e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.737+0000 7f9a03c83700 1 -- 172.21.15.12:0/2068801092 >> 172.21.15.12:0/2068801092 conn(0x7f99fc0f89a0 msgr2=0x7f99fc0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 -- 172.21.15.12:0/2068801092 shutdown_connections 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 -- 172.21.15.12:0/2068801092 wait complete. 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 Processor -- start 2024-03-18T14:57:41.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 -- start start 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc102e10 0x7f99fc115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc116820 con 0x7f99fc1037b0 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a03c83700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc10fbf0 con 0x7f99fc102e10 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a0121e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a0121e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38446/0 (socket says 172.21.15.12:38446) 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a0121e700 1 -- 172.21.15.12:0/3166646034 learned_addr learned my addr 172.21.15.12:0/3166646034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.738+0000 7f9a01a1f700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc102e10 0x7f99fc115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a0121e700 1 -- 172.21.15.12:0/3166646034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc102e10 msgr2=0x7f99fc115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a0121e700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc102e10 0x7f99fc115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a0121e700 1 -- 172.21.15.12:0/3166646034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec005040 con 0x7f99fc1037b0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a0121e700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99f8002a50 tx=0x7f99f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f8007cc0 con 0x7f99fc1037b0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc10fed0 con 0x7f99fc1037b0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99f800f040 con 0x7f99fc1037b0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f800db30 con 0x7f99fc1037b0 2024-03-18T14:57:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.739+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc110490 con 0x7f99fc1037b0 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.740+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc107310 con 0x7f99fc1037b0 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.741+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f99f800dc90 con 0x7f99fc1037b0 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.741+0000 7f99f27fc700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 0x7f99e8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.741+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f99f808d2c0 con 0x7f99fc1037b0 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.741+0000 7f9a01a1f700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 0x7f99e8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.742+0000 7f9a01a1f700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 0x7f99e8070390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99ec004f90 tx=0x7f99ec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:41.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.746+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99f805a360 con 0x7f99fc1037b0 2024-03-18T14:57:41.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.972+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99fc111350 con 0x7f99e806dee0 2024-03-18T14:57:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.982+0000 7f99f27fc700 1 -- 172.21.15.12:0/3166646034 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f99fc111350 con 0x7f99e806dee0 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (3m) 26s ago 14m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (59s) 26s ago 14m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (53s) 50s ago 13m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (3m) 26s ago 14m 78.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:57:41.984 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (8m) 26s ago 8m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:57:41.985 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (9m) 26s ago 9m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:57:41.985 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (8m) 50s ago 8m 16.7M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:57:41.985 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (8m) 50s ago 8m 13.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:57:41.985 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (5m) 26s ago 16m 497M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:57:41.985 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (4m) 50s ago 13m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:57:41.986 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (2m) 26s ago 16m 48.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:57:41.986 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (100s) 50s ago 13m 38.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:57:41.986 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (4m) 26s ago 14m 15.4M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:57:41.986 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (4m) 50s ago 13m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:57:41.986 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (32s) 26s ago 12m 13.0M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (12m) 26s ago 12m 271M 1559M 17.2.7-684-g6716c9f3 19857db3290c 2ad8804a9331 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (11m) 26s ago 11m 221M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (11m) 50s ago 11m 291M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (10m) 50s ago 10m 260M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (10m) 50s ago 10m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:57:41.987 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (3m) 26s ago 13m 64.2M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 msgr2=0x7f99e8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 0x7f99e8070390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99ec004f90 tx=0x7f99ec009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 msgr2=0x7f99fc1161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99f8002a50 tx=0x7f99f80046b0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 shutdown_connections 2024-03-18T14:57:41.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f99e806dee0 0x7f99e8070390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99fc102e10 0x7f99fc115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 --2- 172.21.15.12:0/3166646034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99fc1037b0 0x7f99fc1161d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:41.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 >> 172.21.15.12:0/3166646034 conn(0x7f99fc0f89a0 msgr2=0x7f99fc070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:41.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 shutdown_connections 2024-03-18T14:57:41.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:41.985+0000 7f9a03c83700 1 -- 172.21.15.12:0/3166646034 wait complete. 2024-03-18T14:57:41.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:41 smithi012 ceph-mon[111904]: pgmap v65: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:41.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:41 smithi012 ceph-mon[111904]: osd.1 marked itself down and dead 2024-03-18T14:57:42.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 -- 172.21.15.12:0/942418275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 msgr2=0x7fbe4c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 --2- 172.21.15.12:0/942418275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c1033a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c004940 tx=0x7fbe3c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 -- 172.21.15.12:0/942418275 shutdown_connections 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 --2- 172.21.15.12:0/942418275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 --2- 172.21.15.12:0/942418275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c1033a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 -- 172.21.15.12:0/942418275 >> 172.21.15.12:0/942418275 conn(0x7fbe4c071530 msgr2=0x7fbe4c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 -- 172.21.15.12:0/942418275 shutdown_connections 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.087+0000 7fbe51427700 1 -- 172.21.15.12:0/942418275 wait complete. 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 Processor -- start 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 -- start start 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1145c0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe51427700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c10dba0 con 0x7fbe4c0fb540 2024-03-18T14:57:42.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe4a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.088+0000 7fbe4a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38466/0 (socket says 172.21.15.12:38466) 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4a7fc700 1 -- 172.21.15.12:0/1124024154 learned_addr learned my addr 172.21.15.12:0/1124024154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4affd700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4a7fc700 1 -- 172.21.15.12:0/1124024154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 msgr2=0x7fbe4c1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4a7fc700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4a7fc700 1 -- 172.21.15.12:0/1124024154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c005040 con 0x7fbe4c102fd0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4affd700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe4a7fc700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b790 tx=0x7fbe40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4000f770 con 0x7fbe4c102fd0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c10de80 con 0x7fbe4c102fd0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe4000fda0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4000d310 con 0x7fbe4c102fd0 2024-03-18T14:57:42.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.089+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c10e3c0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.090+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c100270 con 0x7fbe4c102fd0 2024-03-18T14:57:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.091+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fbe40010420 con 0x7fbe4c102fd0 2024-03-18T14:57:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.091+0000 7fbe337fe700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 0x7fbe34070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.091+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fbe4008d6f0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.091+0000 7fbe4affd700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 0x7fbe34070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.092+0000 7fbe4affd700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 0x7fbe34070390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b3b0 tx=0x7fbe3c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.096+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe40056ce0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:41 smithi100 ceph-mon[84868]: pgmap v65: 65 pgs: 65 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:57:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:41 smithi100 ceph-mon[84868]: osd.1 marked itself down and dead 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.399+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe4c10eff0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.399+0000 7fbe337fe700 1 -- 172.21.15.12:0/1124024154 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fbe4005a3a0 con 0x7fbe4c102fd0 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:57:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4, 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 8, 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:57:42.403 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 msgr2=0x7fbe34070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 0x7fbe34070390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b3b0 tx=0x7fbe3c009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 msgr2=0x7fbe4c113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b790 tx=0x7fbe40009250 comp rx=0 tx=0).stop 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 shutdown_connections 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fbe3406dee0 0x7fbe34070390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c0fb540 0x7fbe4c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 --2- 172.21.15.12:0/1124024154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe4c102fd0 0x7fbe4c113ee0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 >> 172.21.15.12:0/1124024154 conn(0x7fbe4c071530 msgr2=0x7fbe4c0fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:42.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 shutdown_connections 2024-03-18T14:57:42.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.402+0000 7fbe51427700 1 -- 172.21.15.12:0/1124024154 wait complete. 2024-03-18T14:57:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- 172.21.15.12:0/279207629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 msgr2=0x7f68a00fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 --2- 172.21.15.12:0/279207629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fbd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6894004940 tx=0x7f6894009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- 172.21.15.12:0/279207629 shutdown_connections 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 --2- 172.21.15.12:0/279207629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fbd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 --2- 172.21.15.12:0/279207629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 0x7f68a01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- 172.21.15.12:0/279207629 >> 172.21.15.12:0/279207629 conn(0x7f68a00f7880 msgr2=0x7f68a00f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- 172.21.15.12:0/279207629 shutdown_connections 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- 172.21.15.12:0/279207629 wait complete. 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 Processor -- start 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.509+0000 7f68a496a700 1 -- start start 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 0x7f68a00fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00fe460 con 0x7f68a01026b0 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00fe5a0 con 0x7f68a0101d10 2024-03-18T14:57:42.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38498/0 (socket says 172.21.15.12:38498) 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 -- 172.21.15.12:0/1962377179 learned_addr learned my addr 172.21.15.12:0/1962377179 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689e59c700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 0x7f68a00fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 -- 172.21.15.12:0/1962377179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 msgr2=0x7f68a00fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 0x7f68a00fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 -- 172.21.15.12:0/1962377179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894005040 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f689dd9b700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6894009100 tx=0x7f6894009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689400e040 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6894013420 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a00fe7f0 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689401c610 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.510+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a011a5d0 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.511+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a00fd460 con 0x7f68a01026b0 2024-03-18T14:57:42.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.512+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6894013590 con 0x7f68a01026b0 2024-03-18T14:57:42.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.512+0000 7f688effd700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 0x7f6888070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.512+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f689408e8a0 con 0x7f68a01026b0 2024-03-18T14:57:42.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.512+0000 7f689e59c700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 0x7f6888070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.513+0000 7f689e59c700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 0x7f6888070390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6890009fd0 tx=0x7f6890009e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.517+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6894057e70 con 0x7f68a01026b0 2024-03-18T14:57:42.724 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:42 smithi012 podman[128452]: 2024-03-18 14:57:42.407027618 +0000 UTC m=+1.383640009 container died 2ad8804a9331e35249700eddd216a70a0b9c0ce1473c85b6bd0ef779b9d8fbc5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=quincy-6716c9f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-18T14:57:42.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.776+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f68a004bed0 con 0x7f68a01026b0 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.776+0000 7f688effd700 1 -- 172.21.15.12:0/1962377179 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f689405b530 con 0x7f68a01026b0 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:57:42.777 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:57:42.778 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:57:42.779 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:42.780 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 msgr2=0x7f6888070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 0x7f6888070390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6890009fd0 tx=0x7f6890009e50 comp rx=0 tx=0).stop 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 msgr2=0x7f68a00fded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6894009100 tx=0x7f6894009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 shutdown_connections 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f688806dee0 0x7f6888070390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68a0101d10 0x7f68a00fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 --2- 172.21.15.12:0/1962377179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a01026b0 0x7f68a00fded0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 >> 172.21.15.12:0/1962377179 conn(0x7f68a00f7880 msgr2=0x7f68a00f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:42.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 shutdown_connections 2024-03-18T14:57:42.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.778+0000 7f68a496a700 1 -- 172.21.15.12:0/1962377179 wait complete. 2024-03-18T14:57:42.782 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- 172.21.15.12:0/2292407346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fdcb0 msgr2=0x7f6bb00fe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/2292407346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb00fe080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0004940 tx=0x7f6ba0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- 172.21.15.12:0/2292407346 shutdown_connections 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/2292407346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb00fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/2292407346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb00fe080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- 172.21.15.12:0/2292407346 >> 172.21.15.12:0/2292407346 conn(0x7f6bb0072310 msgr2=0x7f6bb0072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- 172.21.15.12:0/2292407346 shutdown_connections 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- 172.21.15.12:0/2292407346 wait complete. 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 Processor -- start 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.882+0000 7f6bb62a9700 1 -- start start 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb0116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0116670 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb010fca0 con 0x7f6bb00fdcb0 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38528/0 (socket says 172.21.15.12:38528) 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 -- 172.21.15.12:0/1595492492 learned_addr learned my addr 172.21.15.12:0/1595492492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baf7fe700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb0116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 -- 172.21.15.12:0/1595492492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fdcb0 msgr2=0x7f6bb0116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb0116020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 -- 172.21.15.12:0/1595492492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0005040 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6baffff700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009380 tx=0x7f6ba0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba000e040 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb010ff80 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ba0009af0 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba001c3f0 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.883+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb01104c0 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.884+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0107260 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.885+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6ba0013420 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.885+0000 7f6bacff9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 0x7f6b98070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.885+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6ba008ea00 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.885+0000 7f6baf7fe700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 0x7f6b98070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:42.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.886+0000 7f6baf7fe700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 0x7f6b98070390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0111380 tx=0x7f6ba400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:42.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:42.890+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ba005baa0 con 0x7f6bb00fb3b0 2024-03-18T14:57:42.991 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:42 smithi012 podman[128452]: 2024-03-18 14:57:42.923423878 +0000 UTC m=+1.900036257 container cleanup 2ad8804a9331e35249700eddd216a70a0b9c0ce1473c85b6bd0ef779b9d8fbc5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-6716c9f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-18T14:57:42.992 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:42 smithi012 bash[128452]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: osdmap e54: 6 total, 5 up, 6 in 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1124024154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1962377179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:57:42.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:42 smithi012 ceph-mon[111904]: osdmap e55: 6 total, 5 up, 6 in 2024-03-18T14:57:43.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.139+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bb010c140 con 0x7f6b9806dee0 2024-03-18T14:57:43.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.140+0000 7f6bacff9700 1 -- 172.21.15.12:0/1595492492 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6bb010c140 con 0x7f6b9806dee0 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "7/21 daemons upgraded", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:57:43.141 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 msgr2=0x7f6b98070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 0x7f6b98070390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0111380 tx=0x7f6ba400b040 comp rx=0 tx=0).stop 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 msgr2=0x7f6bb0115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009380 tx=0x7f6ba0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 shutdown_connections 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b9806dee0 0x7f6b98070390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6bb00fb3b0 0x7f6bb0115ae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 --2- 172.21.15.12:0/1595492492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb00fdcb0 0x7f6bb0116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 >> 172.21.15.12:0/1595492492 conn(0x7f6bb0072310 msgr2=0x7f6bb00fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 shutdown_connections 2024-03-18T14:57:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.142+0000 7f6bb62a9700 1 -- 172.21.15.12:0/1595492492 wait complete. 2024-03-18T14:57:43.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- 172.21.15.12:0/3919804344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0fb4d0 msgr2=0x7f6fbc0fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/3919804344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fb940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004940 tx=0x7f6fb0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:43.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- 172.21.15.12:0/3919804344 shutdown_connections 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/3919804344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fb940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/3919804344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc102f50 0x7f6fbc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- 172.21.15.12:0/3919804344 >> 172.21.15.12:0/3919804344 conn(0x7f6fbc0714c0 msgr2=0x7f6fbc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- 172.21.15.12:0/3919804344 shutdown_connections 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- 172.21.15.12:0/3919804344 wait complete. 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 Processor -- start 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.255+0000 7f6fc19ac700 1 -- start start 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fc19ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fc19ac700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fc19ac700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0fe600 con 0x7f6fbc102f50 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fc19ac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0fd130 con 0x7f6fbc0fb4d0 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38546/0 (socket says 172.21.15.12:38546) 2024-03-18T14:57:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 -- 172.21.15.12:0/1337879356 learned_addr learned my addr 172.21.15.12:0/1337879356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:57:43.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fbaffd700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 -- 172.21.15.12:0/1337879356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc0fb4d0 msgr2=0x7f6fbc0fc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 -- 172.21.15.12:0/1337879356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0005040 con 0x7f6fbc102f50 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fba7fc700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b3b0 tx=0x7f6fb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb000e040 con 0x7f6fbc102f50 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fb0013420 con 0x7f6fbc102f50 2024-03-18T14:57:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb001c630 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.256+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc0fd3b0 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.257+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc11a0b0 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.258+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc105330 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.258+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6fb0013590 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.258+0000 7f6fa37fe700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 0x7f6fa4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.259+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6fb008e880 con 0x7f6fbc102f50 2024-03-18T14:57:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.259+0000 7f6fbaffd700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 0x7f6fa4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:57:43.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.259+0000 7f6fbaffd700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 0x7f6fa4070390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc065c10 tx=0x7f6fac00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:57:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.263+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fb0057e70 con 0x7f6fbc102f50 2024-03-18T14:57:43.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: osdmap e54: 6 total, 5 up, 6 in 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1124024154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1962377179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:57:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:42 smithi100 ceph-mon[84868]: osdmap e55: 6 total, 5 up, 6 in 2024-03-18T14:57:43.491 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:43 smithi012 podman[128560]: 2024-03-18 14:57:43.18383255 +0000 UTC m=+0.780568493 container remove 2ad8804a9331e35249700eddd216a70a0b9c0ce1473c85b6bd0ef779b9d8fbc5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6716c9f, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.581+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6fbc0fe7f0 con 0x7f6fbc102f50 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.581+0000 7f6fa37fe700 1 -- 172.21.15.12:0/1337879356 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+901 (secure 0 0 0) 0x7f6fb005b530 con 0x7f6fbc102f50 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 9 pgs peering 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: osd.1 (root=default,host=smithi012) is down 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.6 is stuck peering for 4m, current state peering, last acting [3,4] 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.9 is stuck peering for 4m, current state peering, last acting [4,0] 2024-03-18T14:57:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.a is stuck peering for 4m, current state peering, last acting [4,3] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.d is stuck peering for 4m, current state peering, last acting [3,2] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 2.15 is stuck peering for 4m, current state peering, last acting [3,0] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 3.0 is stuck peering for 4m, current state peering, last acting [2,4] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 3.2 is stuck peering for 4m, current state peering, last acting [5,3] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 3.4 is stuck peering for 4m, current state peering, last acting [2,3] 2024-03-18T14:57:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: pg 3.19 is stuck peering for 4m, current state peering, last acting [4,3] 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 msgr2=0x7f6fa4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 0x7f6fa4070390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc065c10 tx=0x7f6fac00b040 comp rx=0 tx=0).stop 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 msgr2=0x7f6fbc0fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b3b0 tx=0x7f6fb0009e30 comp rx=0 tx=0).stop 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 shutdown_connections 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fa406dee0 0x7f6fa4070390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fbc0fb4d0 0x7f6fbc0fc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 --2- 172.21.15.12:0/1337879356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102f50 0x7f6fbc0fcbf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 >> 172.21.15.12:0/1337879356 conn(0x7f6fbc0714c0 msgr2=0x7f6fbc0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 shutdown_connections 2024-03-18T14:57:43.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:57:43.583+0000 7f6fc19ac700 1 -- 172.21.15.12:0/1337879356 wait complete. 2024-03-18T14:57:43.904 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:43 smithi012 podman[128719]: 2024-03-18 14:57:43.517876895 +0000 UTC m=+0.018232991 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:43.904 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:43 smithi012 podman[128719]: 2024-03-18 14:57:43.676705919 +0000 UTC m=+0.177062001 container create e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-18T14:57:43.904 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:43 smithi012 podman[128719]: 2024-03-18 14:57:43.904102949 +0000 UTC m=+0.404459046 container init e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T14:57:44.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:43 smithi100 ceph-mon[84868]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:57:44.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:43 smithi100 ceph-mon[84868]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-18T14:57:44.196 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:43 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1337879356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:57:44.207 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:43 smithi012 ceph-mon[111904]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:57:44.208 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:43 smithi012 ceph-mon[111904]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-18T14:57:44.208 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:43 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1337879356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:57:44.208 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:43 smithi012 podman[128719]: 2024-03-18 14:57:43.907356776 +0000 UTC m=+0.407712858 container start e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-18T14:57:44.208 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 podman[128719]: 2024-03-18 14:57:44.01251287 +0000 UTC m=+0.512868969 container attach e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:57:44.491 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 podman[128719]: 2024-03-18 14:57:44.2081075 +0000 UTC m=+0.708463597 container died e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-18T14:57:45.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:44 smithi012 ceph-mon[111904]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:45.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 podman[128748]: 2024-03-18 14:57:44.824786735 +0000 UTC m=+0.611218924 container remove e93c915780c6a20d25f710d14e503c6fdb83d7d0f66f77ee759ea2a4340e7be0 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-18T14:57:45.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-18T14:57:45.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 systemd[1]: Stopped Ceph osd.1 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:57:45.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:44 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.1.service: Consumed 1min 8.736s CPU time. 2024-03-18T14:57:45.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 systemd[1]: Starting Ceph osd.1 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:57:45.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:44 smithi100 ceph-mon[84868]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:57:45.561 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 podman[128853]: 2024-03-18 14:57:45.166927221 +0000 UTC m=+0.018216714 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:45.561 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 podman[128853]: 2024-03-18 14:57:45.284107531 +0000 UTC m=+0.135397046 container create 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:57:45.561 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 podman[128853]: 2024-03-18 14:57:45.561591106 +0000 UTC m=+0.412880580 container init 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T14:57:45.944 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 podman[128853]: 2024-03-18 14:57:45.565085415 +0000 UTC m=+0.416374892 container start 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:57:45.944 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:45 smithi012 podman[128853]: 2024-03-18 14:57:45.661583819 +0000 UTC m=+0.512873317 container attach 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-18T14:57:46.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:45 smithi012 ceph-mon[111904]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-18T14:57:46.242 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 bash[128853]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate[128867]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-18T14:57:46.243 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 podman[128853]: 2024-03-18 14:57:46.041660837 +0000 UTC m=+0.892950311 container died 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-18T14:57:46.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:45 smithi100 ceph-mon[84868]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T14:57:46.991 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 podman[128996]: 2024-03-18 14:57:46.73287444 +0000 UTC m=+0.685942691 container remove 4d59c9f4a41d6094aa3cc6cf6c5c210023500e20be3647e929b34bc549a95b90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T14:57:46.992 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 podman[129061]: 2024-03-18 14:57:46.848671262 +0000 UTC m=+0.018214360 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:57:46.992 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:46 smithi012 podman[129061]: 2024-03-18 14:57:46.958738985 +0000 UTC m=+0.128282083 container create a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:57:47.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:47 smithi012 ceph-mon[111904]: pgmap v70: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-03-18T14:57:47.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:47.370 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:47.370 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 podman[129061]: 2024-03-18 14:57:47.269895913 +0000 UTC m=+0.439439011 container init a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:57:47.370 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 podman[129061]: 2024-03-18 14:57:47.272907764 +0000 UTC m=+0.442450868 container start a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:57:47.370 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 ceph-osd[129079]: -- 172.21.15.12:0/1673272542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a9bfa3680 con 0x555a9bf6d000 2024-03-18T14:57:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:47 smithi100 ceph-mon[84868]: pgmap v70: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-03-18T14:57:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:57:47.741 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 bash[129061]: a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b 2024-03-18T14:57:47.742 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 systemd[1]: Started Ceph osd.1 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:57:48.241 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:47 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T14:57:47.818+0000 7fc71a3967c0 -1 Falling back to public interface 2024-03-18T14:57:48.506 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:48.506 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:48.506 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:49.703 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:49 smithi012 ceph-mon[111904]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:49.703 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:49 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 31/216 objects degraded (14.352%), 13 pgs degraded (PG_DEGRADED) 2024-03-18T14:57:49.703 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:49 smithi012 ceph-mon[111904]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-18T14:57:49.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:49 smithi100 ceph-mon[84868]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:49 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 31/216 objects degraded (14.352%), 13 pgs degraded (PG_DEGRADED) 2024-03-18T14:57:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:49 smithi100 ceph-mon[84868]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-18T14:57:51.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:51 smithi012 ceph-mon[111904]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:51.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:51 smithi100 ceph-mon[84868]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:52.491 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:52 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T14:57:52.401+0000 7fc71a3967c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-18T14:57:53.491 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:53 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T14:57:53.035+0000 7fc71a3967c0 -1 osd.1 53 log_to_monitors true 2024-03-18T14:57:53.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:53 smithi012 ceph-mon[111904]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:53 smithi012 ceph-mon[111904]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T14:57:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:53 smithi100 ceph-mon[84868]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:54.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:53 smithi100 ceph-mon[84868]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T14:57:54.951 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T14:57:54.682+0000 7fc7077e6700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:57:54.951 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T14:57:54.951 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: osdmap e56: 6 total, 5 up, 6 in 2024-03-18T14:57:54.951 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:57:54.952 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:54.952 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:54.952 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:54 smithi012 ceph-mon[111904]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' 2024-03-18T14:57:55.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T14:57:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: osdmap e56: 6 total, 5 up, 6 in 2024-03-18T14:57:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:57:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:55.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:54 smithi100 ceph-mon[84868]: from='osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107]' entity='osd.1' 2024-03-18T14:57:56.001 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:55 smithi012 ceph-mon[111904]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:56.001 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:55 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:57:56.001 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:55 smithi012 ceph-mon[111904]: osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107] boot 2024-03-18T14:57:56.001 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:55 smithi012 ceph-mon[111904]: osdmap e57: 6 total, 6 up, 6 in 2024-03-18T14:57:56.002 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:55 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:57:56.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:55 smithi100 ceph-mon[84868]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:56.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:55 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:57:56.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:55 smithi100 ceph-mon[84868]: osd.1 [v2:172.21.15.12:6810/4045714107,v1:172.21.15.12:6811/4045714107] boot 2024-03-18T14:57:56.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:55 smithi100 ceph-mon[84868]: osdmap e57: 6 total, 6 up, 6 in 2024-03-18T14:57:56.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:55 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T14:57:57.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:57 smithi012 ceph-mon[111904]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:57.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:57 smithi012 ceph-mon[111904]: osdmap e58: 6 total, 6 up, 6 in 2024-03-18T14:57:58.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:57 smithi100 ceph-mon[84868]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 31/216 objects degraded (14.352%) 2024-03-18T14:57:58.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:57 smithi100 ceph-mon[84868]: osdmap e58: 6 total, 6 up, 6 in 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:57:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:58 smithi012 ceph-mon[111904]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-18T14:57:59.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:57:59.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:58 smithi100 ceph-mon[84868]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-18T14:57:59.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:59 smithi012 ceph-mon[111904]: pgmap v79: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 16/216 objects degraded (7.407%) 2024-03-18T14:57:59.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:57:59 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 16/216 objects degraded (7.407%), 7 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:00.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:59 smithi100 ceph-mon[84868]: pgmap v79: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 16/216 objects degraded (7.407%) 2024-03-18T14:58:00.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:57:59 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 16/216 objects degraded (7.407%), 7 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:01.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:00 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/216 objects degraded (7.407%), 7 pgs degraded) 2024-03-18T14:58:01.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:00 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T14:58:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:00 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/216 objects degraded (7.407%), 7 pgs degraded) 2024-03-18T14:58:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:00 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T14:58:02.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:01 smithi100 ceph-mon[84868]: pgmap v80: 65 pgs: 2 peering, 63 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 645 B/s rd, 0 op/s 2024-03-18T14:58:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:02.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:01 smithi012 ceph-mon[111904]: pgmap v80: 65 pgs: 2 peering, 63 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 645 B/s rd, 0 op/s 2024-03-18T14:58:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:04.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:03 smithi100 ceph-mon[84868]: pgmap v81: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:58:04.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:03 smithi012 ceph-mon[111904]: pgmap v81: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:58:06.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:05 smithi100 ceph-mon[84868]: pgmap v82: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:58:06.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:05 smithi012 ceph-mon[111904]: pgmap v82: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:58:08.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:07 smithi100 ceph-mon[84868]: pgmap v83: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-18T14:58:08.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:07 smithi012 ceph-mon[111904]: pgmap v83: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-18T14:58:10.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:09 smithi100 ceph-mon[84868]: pgmap v84: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:09 smithi012 ceph-mon[111904]: pgmap v84: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:12.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:11 smithi100 ceph-mon[84868]: pgmap v85: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:12.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:11 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:12.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:11 smithi012 ceph-mon[111904]: pgmap v85: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:12.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:11 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:13.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- 172.21.15.12:0/173834632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 msgr2=0x7f0a90106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/173834632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90106f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c004940 tx=0x7f0a7c009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- 172.21.15.12:0/173834632 shutdown_connections 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/173834632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90106f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/173834632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 0x7f0a900fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- 172.21.15.12:0/173834632 >> 172.21.15.12:0/173834632 conn(0x7f0a90072bc0 msgr2=0x7f0a90072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:13.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- 172.21.15.12:0/173834632 shutdown_connections 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- 172.21.15.12:0/173834632 wait complete. 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 Processor -- start 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.691+0000 7f0a96eb2700 1 -- start start 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a96eb2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 0x7f0a90113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a96eb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a96eb2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a901146b0 con 0x7f0a900fe830 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a96eb2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9010daa0 con 0x7f0a900fde90 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:13.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39068/0 (socket says 172.21.15.12:39068) 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 -- 172.21.15.12:0/1213246261 learned_addr learned my addr 172.21.15.12:0/1213246261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a94c4e700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 0x7f0a90113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 -- 172.21.15.12:0/1213246261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 msgr2=0x7f0a90113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 0x7f0a90113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 -- 172.21.15.12:0/1213246261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c005040 con 0x7f0a900fe830 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.692+0000 7f0a8ffff700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c000c00 tx=0x7f0a7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.693+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7c021070 con 0x7f0a900fe830 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.693+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9010dd20 con 0x7f0a900fe830 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.693+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c013cd0 con 0x7f0a900fe830 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.693+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7c00e070 con 0x7f0a900fe830 2024-03-18T14:58:13.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.693+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9010e260 con 0x7f0a900fe830 2024-03-18T14:58:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.694+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90065e30 con 0x7f0a900fe830 2024-03-18T14:58:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.694+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f0a7c01ca60 con 0x7f0a900fe830 2024-03-18T14:58:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.694+0000 7f0a8d7fa700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 0x7f0a800703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.695+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f0a7c08ea50 con 0x7f0a900fe830 2024-03-18T14:58:13.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.695+0000 7f0a94c4e700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 0x7f0a800703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:13.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.695+0000 7f0a94c4e700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 0x7f0a800703a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a84004600 tx=0x7f0a8400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:13.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.699+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a7c05bb70 con 0x7f0a900fe830 2024-03-18T14:58:13.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.929+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a90109f10 con 0x7f0a8006def0 2024-03-18T14:58:13.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.931+0000 7f0a8d7fa700 1 -- 172.21.15.12:0/1213246261 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0a90109f10 con 0x7f0a8006def0 2024-03-18T14:58:13.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 msgr2=0x7f0a800703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:13.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 0x7f0a800703a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0a84004600 tx=0x7f0a8400b040 comp rx=0 tx=0).stop 2024-03-18T14:58:13.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 msgr2=0x7f0a90114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:13.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c000c00 tx=0x7f0a7c009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:13.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 shutdown_connections 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0a8006def0 0x7f0a800703a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a900fde90 0x7f0a90113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 --2- 172.21.15.12:0/1213246261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a900fe830 0x7f0a90114060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 >> 172.21.15.12:0/1213246261 conn(0x7f0a90072bc0 msgr2=0x7f0a90103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 shutdown_connections 2024-03-18T14:58:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:13.933+0000 7f0a96eb2700 1 -- 172.21.15.12:0/1213246261 wait complete. 2024-03-18T14:58:13.944 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 -- 172.21.15.12:0/2714505432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 msgr2=0x7f737c0fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 --2- 172.21.15.12:0/2714505432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fe1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f736c004940 tx=0x7f736c009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 -- 172.21.15.12:0/2714505432 shutdown_connections 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 --2- 172.21.15.12:0/2714505432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 0x7f737c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 --2- 172.21.15.12:0/2714505432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fe1f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 -- 172.21.15.12:0/2714505432 >> 172.21.15.12:0/2714505432 conn(0x7f737c0f7870 msgr2=0x7f737c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 -- 172.21.15.12:0/2714505432 shutdown_connections 2024-03-18T14:58:14.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.037+0000 7f7380817700 1 -- 172.21.15.12:0/2714505432 wait complete. 2024-03-18T14:58:14.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:13 smithi012 ceph-mon[111904]: pgmap v86: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:14.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:58:14.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:13 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:58:14.040 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:13 smithi012 ceph-mon[111904]: Upgrade: osd.2 is safe to restart 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 Processor -- start 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 -- start start 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 0x7f737c0fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0fdba0 con 0x7f737c0fde20 2024-03-18T14:58:14.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f7380817700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0fc6d0 con 0x7f737c0fe7c0 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f737a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f737a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39082/0 (socket says 172.21.15.12:39082) 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.038+0000 7f737a59c700 1 -- 172.21.15.12:0/2078878745 learned_addr learned my addr 172.21.15.12:0/2078878745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7373fff700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 0x7f737c0fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f737a59c700 1 -- 172.21.15.12:0/2078878745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 msgr2=0x7f737c0fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f737a59c700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 0x7f737c0fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f737a59c700 1 -- 172.21.15.12:0/2078878745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c005040 con 0x7f737c0fde20 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f737a59c700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f736c009380 tx=0x7f736c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c00e040 con 0x7f737c0fde20 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f736c009af0 con 0x7f737c0fde20 2024-03-18T14:58:14.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c0fc950 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c01c630 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.039+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c11a100 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.040+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c066b70 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.041+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f736c00bbc0 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.041+0000 7f7372ffd700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 0x7f7368070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.041+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f736c08e960 con 0x7f737c0fde20 2024-03-18T14:58:14.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.041+0000 7f7373fff700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 0x7f7368070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:13 smithi100 ceph-mon[84868]: pgmap v86: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:14.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:58:14.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:13 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-18T14:58:14.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:13 smithi100 ceph-mon[84868]: Upgrade: osd.2 is safe to restart 2024-03-18T14:58:14.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.042+0000 7f7373fff700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 0x7f7368070390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7364007fd0 tx=0x7f7364009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.046+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f736c057f50 con 0x7f737c0fde20 2024-03-18T14:58:14.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.273+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f737c102720 con 0x7f736806dee0 2024-03-18T14:58:14.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.275+0000 7f7372ffd700 1 -- 172.21.15.12:0/2078878745 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f737c102720 con 0x7f736806dee0 2024-03-18T14:58:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 msgr2=0x7f7368070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 0x7f7368070390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7364007fd0 tx=0x7f7364009040 comp rx=0 tx=0).stop 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 msgr2=0x7f737c0fbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f736c009380 tx=0x7f736c009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 shutdown_connections 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f736806dee0 0x7f7368070390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f737c0fde20 0x7f737c0fbc50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 --2- 172.21.15.12:0/2078878745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f737c0fe7c0 0x7f737c0fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 >> 172.21.15.12:0/2078878745 conn(0x7f737c0f7870 msgr2=0x7f737c101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 shutdown_connections 2024-03-18T14:58:14.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.277+0000 7f7380817700 1 -- 172.21.15.12:0/2078878745 wait complete. 2024-03-18T14:58:14.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.379+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3357538466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 msgr2=0x7f3ac00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.379+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3357538466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac00fce10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4004940 tx=0x7f3ab4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3357538466 shutdown_connections 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3357538466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac00fce10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3357538466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 0x7f3ac01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3357538466 >> 172.21.15.12:0/3357538466 conn(0x7f3ac00f89b0 msgr2=0x7f3ac00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3357538466 shutdown_connections 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3357538466 wait complete. 2024-03-18T14:58:14.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 Processor -- start 2024-03-18T14:58:14.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.380+0000 7f3ac67d9700 1 -- start start 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3ac67d9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 0x7f3ac0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3ac67d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3ac67d9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0114510 con 0x7f3ac01037a0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3ac67d9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac010db10 con 0x7f3ac0102e00 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39114/0 (socket says 172.21.15.12:39114) 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 -- 172.21.15.12:0/3877653468 learned_addr learned my addr 172.21.15.12:0/3877653468 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abffff700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 0x7f3ac0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 -- 172.21.15.12:0/3877653468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 msgr2=0x7f3ac0113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 0x7f3ac0113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 -- 172.21.15.12:0/3877653468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4005040 con 0x7f3ac01037a0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abf7fe700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009100 tx=0x7f3ab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab400e040 con 0x7f3ac01037a0 2024-03-18T14:58:14.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac010dd90 con 0x7f3ac01037a0 2024-03-18T14:58:14.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ab4004290 con 0x7f3ac01037a0 2024-03-18T14:58:14.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.381+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab401c3f0 con 0x7f3ac01037a0 2024-03-18T14:58:14.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.382+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac010e350 con 0x7f3ac01037a0 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.382+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac01052d0 con 0x7f3ac01037a0 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.383+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3ab4013690 con 0x7f3ac01037a0 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.383+0000 7f3abcff9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 0x7f3aa8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.383+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f3ab408e870 con 0x7f3ac01037a0 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.384+0000 7f3abffff700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 0x7f3aa8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.384+0000 7f3abffff700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 0x7f3aa8070390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0004f40 tx=0x7f3ab000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.388+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ab405b990 con 0x7f3ac01037a0 2024-03-18T14:58:14.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.607+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ac004c7b0 con 0x7f3aa806dee0 2024-03-18T14:58:14.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.617+0000 7f3abcff9700 1 -- 172.21.15.12:0/3877653468 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3ac004c7b0 con 0x7f3aa806dee0 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (4m) 21s ago 15m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (91s) 21s ago 15m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (86s) 82s ago 14m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (3m) 21s ago 14m 79.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (9m) 21s ago 9m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (9m) 21s ago 9m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (9m) 82s ago 9m 16.7M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (9m) 82s ago 9m 13.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:58:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (5m) 21s ago 16m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (4m) 82s ago 14m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (2m) 21s ago 16m 50.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (2m) 82s ago 13m 38.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (4m) 21s ago 15m 15.3M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (4m) 82s ago 14m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (64s) 21s ago 13m 101M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (27s) 21s ago 12m 12.6M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (12m) 21s ago 12m 222M 1559M 17.2.7-684-g6716c9f3 19857db3290c f87ceb735d2b 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (11m) 82s ago 11m 291M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (11m) 82s ago 11m 260M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (11m) 82s ago 11m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:58:14.619 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (4m) 21s ago 14m 66.3M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 msgr2=0x7f3aa8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 0x7f3aa8070390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0004f40 tx=0x7f3ab000b040 comp rx=0 tx=0).stop 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 msgr2=0x7f3ac0113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009100 tx=0x7f3ab4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 shutdown_connections 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3aa806dee0 0x7f3aa8070390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac0102e00 0x7f3ac0113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 --2- 172.21.15.12:0/3877653468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ac01037a0 0x7f3ac0113ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 >> 172.21.15.12:0/3877653468 conn(0x7f3ac00f89b0 msgr2=0x7f3ac01021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 shutdown_connections 2024-03-18T14:58:14.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.619+0000 7f3ac67d9700 1 -- 172.21.15.12:0/3877653468 wait complete. 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 -- 172.21.15.12:0/325983062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e0102e70 msgr2=0x7f80e0103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 --2- 172.21.15.12:0/325983062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e0102e70 0x7f80e0103240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80d4004940 tx=0x7f80d4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 -- 172.21.15.12:0/325983062 shutdown_connections 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 --2- 172.21.15.12:0/325983062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e00fb4a0 0x7f80e00fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 --2- 172.21.15.12:0/325983062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e0102e70 0x7f80e0103240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 -- 172.21.15.12:0/325983062 >> 172.21.15.12:0/325983062 conn(0x7f80e00714c0 msgr2=0x7f80e00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 -- 172.21.15.12:0/325983062 shutdown_connections 2024-03-18T14:58:14.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.724+0000 7f80e6d07700 1 -- 172.21.15.12:0/325983062 wait complete. 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 Processor -- start 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 -- start start 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e0102e70 0x7f80e0114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01147a0 con 0x7f80e00fb4a0 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e6d07700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e010db90 con 0x7f80e0102e70 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e4aa3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e4aa3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39120/0 (socket says 172.21.15.12:39120) 2024-03-18T14:58:14.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.725+0000 7f80e4aa3700 1 -- 172.21.15.12:0/174854237 learned_addr learned my addr 172.21.15.12:0/174854237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80dffff700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e0102e70 0x7f80e0114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e4aa3700 1 -- 172.21.15.12:0/174854237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e0102e70 msgr2=0x7f80e0114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e4aa3700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e0102e70 0x7f80e0114150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e4aa3700 1 -- 172.21.15.12:0/174854237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4005040 con 0x7f80e00fb4a0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e4aa3700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80d400b3b0 tx=0x7f80d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d400e040 con 0x7f80e00fb4a0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e010de10 con 0x7f80e00fb4a0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e010e350 con 0x7f80e00fb4a0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80d4013420 con 0x7f80e00fb4a0 2024-03-18T14:58:14.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.726+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d401c630 con 0x7f80e00fb4a0 2024-03-18T14:58:14.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.727+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00653a0 con 0x7f80e00fb4a0 2024-03-18T14:58:14.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.728+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f80d400baf0 con 0x7f80e00fb4a0 2024-03-18T14:58:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.729+0000 7f80dd7fa700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 0x7f80d0070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.729+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f80d408e780 con 0x7f80e00fb4a0 2024-03-18T14:58:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.729+0000 7f80dffff700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 0x7f80d0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:14.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.730+0000 7f80dffff700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 0x7f80d0070390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80e010f440 tx=0x7f80cc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:14.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:14.734+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80d405b940 con 0x7f80e00fb4a0 2024-03-18T14:58:14.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:14 smithi012 ceph-mon[111904]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:14.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:14.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:58:14.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.031+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80e004bed0 con 0x7f80e00fb4a0 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.031+0000 7f80dd7fa700 1 -- 172.21.15.12:0/174854237 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f80d405b4d0 con 0x7f80e00fb4a0 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4, 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 8, 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:58:15.033 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 msgr2=0x7f80d0070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 0x7f80d0070390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80e010f440 tx=0x7f80cc00b040 comp rx=0 tx=0).stop 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 msgr2=0x7f80e0113c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f80d400b3b0 tx=0x7f80d4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 shutdown_connections 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f80d006dee0 0x7f80d0070390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00fb4a0 0x7f80e0113c10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 --2- 172.21.15.12:0/174854237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e0102e70 0x7f80e0114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.033+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 >> 172.21.15.12:0/174854237 conn(0x7f80e00714c0 msgr2=0x7f80e00fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.034+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 shutdown_connections 2024-03-18T14:58:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.034+0000 7f80e6d07700 1 -- 172.21.15.12:0/174854237 wait complete. 2024-03-18T14:58:15.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:14 smithi100 ceph-mon[84868]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:15.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:15.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T14:58:15.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 -- 172.21.15.12:0/4055639930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc102e00 msgr2=0x7f3adc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/4055639930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc102e00 0x7f3adc1031d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009ce0 tx=0x7f3acc004d10 comp rx=0 tx=0).stop 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 -- 172.21.15.12:0/4055639930 shutdown_connections 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/4055639930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc1037a0 0x7f3adc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/4055639930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc102e00 0x7f3adc1031d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 -- 172.21.15.12:0/4055639930 >> 172.21.15.12:0/4055639930 conn(0x7f3adc0f89b0 msgr2=0x7f3adc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 -- 172.21.15.12:0/4055639930 shutdown_connections 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.147+0000 7f3ae47a7700 1 -- 172.21.15.12:0/4055639930 wait complete. 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 Processor -- start 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 -- start start 2024-03-18T14:58:15.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc102e00 0x7f3adc113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc114890 con 0x7f3adc1037a0 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae47a7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc10dcb0 con 0x7f3adc102e00 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39138/0 (socket says 172.21.15.12:39138) 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 -- 172.21.15.12:0/2130885141 learned_addr learned my addr 172.21.15.12:0/2130885141 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae2543700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc102e00 0x7f3adc113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 -- 172.21.15.12:0/2130885141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc102e00 msgr2=0x7f3adc113d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc102e00 0x7f3adc113d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.148+0000 7f3ae1d42700 1 -- 172.21.15.12:0/2130885141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc009940 con 0x7f3adc1037a0 2024-03-18T14:58:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3ae1d42700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c800 tx=0x7f3ad0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad00106f0 con 0x7f3adc1037a0 2024-03-18T14:58:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc10df90 con 0x7f3adc1037a0 2024-03-18T14:58:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ad0010d20 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad00113a0 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.149+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc10e550 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.150+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc100110 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.150+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3ad0010850 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.151+0000 7f3adaffd700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 0x7f3ac4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.151+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f3ad008d550 con 0x7f3adc1037a0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.151+0000 7f3ae2543700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 0x7f3ac4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.151+0000 7f3ae2543700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 0x7f3ac4070390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b740 tx=0x7f3acc004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.155+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ad005a710 con 0x7f3adc1037a0 2024-03-18T14:58:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.411+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3adc10f350 con 0x7f3adc1037a0 2024-03-18T14:58:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.412+0000 7f3adaffd700 1 -- 172.21.15.12:0/2130885141 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f3ad005a2a0 con 0x7f3adc1037a0 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:58:15.413 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:58:15.414 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:58:15.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 msgr2=0x7f3ac4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 0x7f3ac4070390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b740 tx=0x7f3acc004d10 comp rx=0 tx=0).stop 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 msgr2=0x7f3adc114240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c800 tx=0x7f3ad0004470 comp rx=0 tx=0).stop 2024-03-18T14:58:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 shutdown_connections 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ac406dee0 0x7f3ac4070390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3adc102e00 0x7f3adc113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 --2- 172.21.15.12:0/2130885141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3adc1037a0 0x7f3adc114240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 >> 172.21.15.12:0/2130885141 conn(0x7f3adc0f89b0 msgr2=0x7f3adc1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 shutdown_connections 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.414+0000 7f3ae47a7700 1 -- 172.21.15.12:0/2130885141 wait complete. 2024-03-18T14:58:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:58:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- 172.21.15.12:0/31368669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fe7c0 msgr2=0x7f2c00106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/31368669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fe7c0 0x7f2c00106e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4004940 tx=0x7f2bf4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- 172.21.15.12:0/31368669 shutdown_connections 2024-03-18T14:58:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/31368669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fe7c0 0x7f2c00106e00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/31368669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fde20 0x7f2c000fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- 172.21.15.12:0/31368669 >> 172.21.15.12:0/31368669 conn(0x7f2c000f7870 msgr2=0x7f2c000f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- 172.21.15.12:0/31368669 shutdown_connections 2024-03-18T14:58:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- 172.21.15.12:0/31368669 wait complete. 2024-03-18T14:58:15.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 Processor -- start 2024-03-18T14:58:15.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.521+0000 7f2c07cdb700 1 -- start start 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fe7c0 0x7f2c000fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000fdc30 con 0x7f2c000fde20 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000fc6d0 con 0x7f2c000fe7c0 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39154/0 (socket says 172.21.15.12:39154) 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 -- 172.21.15.12:0/1531032186 learned_addr learned my addr 172.21.15.12:0/1531032186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05276700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fe7c0 0x7f2c000fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 -- 172.21.15.12:0/1531032186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fe7c0 msgr2=0x7f2c000fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fe7c0 0x7f2c000fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 -- 172.21.15.12:0/1531032186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf4005040 con 0x7f2c000fde20 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c05a77700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b790 tx=0x7f2bfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bfc00f770 con 0x7f2c000fde20 2024-03-18T14:58:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bfc00fda0 con 0x7f2c000fde20 2024-03-18T14:58:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bfc00d310 con 0x7f2c000fde20 2024-03-18T14:58:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c000fc9b0 con 0x7f2c000fde20 2024-03-18T14:58:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.522+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0011a150 con 0x7f2c000fde20 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.523+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00066b50 con 0x7f2c000fde20 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.524+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2bfc010420 con 0x7f2c000fde20 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.524+0000 7f2bf27fc700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 0x7f2bec070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.524+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f2bfc08d760 con 0x7f2c000fde20 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.525+0000 7f2c05276700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 0x7f2bec070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.525+0000 7f2c05276700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 0x7f2bec070390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400b3b0 tx=0x7f2bf4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.529+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2bfc056cb0 con 0x7f2c000fde20 2024-03-18T14:58:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.756+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c0005e400 con 0x7f2bec06dee0 2024-03-18T14:58:15.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.757+0000 7f2bf27fc700 1 -- 172.21.15.12:0/1531032186 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2c0005e400 con 0x7f2bec06dee0 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:58:15.758 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:58:15.759 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:58:15.759 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "8/21 daemons upgraded", 2024-03-18T14:58:15.759 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:58:15.759 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:58:15.759 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 msgr2=0x7f2bec070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 0x7f2bec070390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400b3b0 tx=0x7f2bf4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 msgr2=0x7f2c000fbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b790 tx=0x7f2bfc009250 comp rx=0 tx=0).stop 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 shutdown_connections 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2bec06dee0 0x7f2bec070390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c000fde20 0x7f2c000fbc50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 --2- 172.21.15.12:0/1531032186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c000fe7c0 0x7f2c000fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 >> 172.21.15.12:0/1531032186 conn(0x7f2c000f7870 msgr2=0x7f2c000ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 shutdown_connections 2024-03-18T14:58:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.759+0000 7f2c07cdb700 1 -- 172.21.15.12:0/1531032186 wait complete. 2024-03-18T14:58:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 -- 172.21.15.12:0/1987633702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e81037a0 msgr2=0x7f52e80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 --2- 172.21.15.12:0/1987633702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e81037a0 0x7f52e80fce10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004940 tx=0x7f52d0009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 -- 172.21.15.12:0/1987633702 shutdown_connections 2024-03-18T14:58:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 --2- 172.21.15.12:0/1987633702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e81037a0 0x7f52e80fce10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 --2- 172.21.15.12:0/1987633702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e8102e00 0x7f52e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 -- 172.21.15.12:0/1987633702 >> 172.21.15.12:0/1987633702 conn(0x7f52e80f89b0 msgr2=0x7f52e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 -- 172.21.15.12:0/1987633702 shutdown_connections 2024-03-18T14:58:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.867+0000 7f52ec937700 1 -- 172.21.15.12:0/1987633702 wait complete. 2024-03-18T14:58:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 Processor -- start 2024-03-18T14:58:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 -- start start 2024-03-18T14:58:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e81037a0 0x7f52e8113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81145b0 con 0x7f52e8102e00 2024-03-18T14:58:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52ec937700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e810db40 con 0x7f52e81037a0 2024-03-18T14:58:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39178/0 (socket says 172.21.15.12:39178) 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 -- 172.21.15.12:0/1270191602 learned_addr learned my addr 172.21.15.12:0/1270191602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e5d9b700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e81037a0 0x7f52e8113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 -- 172.21.15.12:0/1270191602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e81037a0 msgr2=0x7f52e8113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e81037a0 0x7f52e8113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.868+0000 7f52e659c700 1 -- 172.21.15.12:0/1270191602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0005040 con 0x7f52e8102e00 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52e659c700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52e000bbb0 tx=0x7f52e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52e00092e0 con 0x7f52e8102e00 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52e000cb20 con 0x7f52e8102e00 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52e0015d70 con 0x7f52e8102e00 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e810de20 con 0x7f52e8102e00 2024-03-18T14:58:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.869+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e810e360 con 0x7f52e8102e00 2024-03-18T14:58:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.870+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e8100110 con 0x7f52e8102e00 2024-03-18T14:58:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.871+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f52e0012070 con 0x7f52e8102e00 2024-03-18T14:58:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.871+0000 7f52deffd700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 0x7f52d4070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.871+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f52e008d770 con 0x7f52e8102e00 2024-03-18T14:58:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.871+0000 7f52e5d9b700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 0x7f52d4070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.872+0000 7f52e5d9b700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 0x7f52d4070180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004890 tx=0x7f52d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:15.876+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52e005a810 con 0x7f52e8102e00 2024-03-18T14:58:16.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: pgmap v87: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: Upgrade: Updating osd.2 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: Deploying daemon osd.2 on smithi012 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/174854237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:16.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:15 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2130885141' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:58:16.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.174+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f52e8109f50 con 0x7f52e8102e00 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: pgmap v87: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: Upgrade: Updating osd.2 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: Deploying daemon osd.2 on smithi012 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/174854237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:16.176 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:15 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2130885141' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:58:16.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.175+0000 7f52deffd700 1 -- 172.21.15.12:0/1270191602 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f52e005a3a0 con 0x7f52e8102e00 2024-03-18T14:58:16.177 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:58:16.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 msgr2=0x7f52d4070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:16.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 0x7f52d4070180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004890 tx=0x7f52d0009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:16.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 msgr2=0x7f52e8113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52e000bbb0 tx=0x7f52e0009840 comp rx=0 tx=0).stop 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 shutdown_connections 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52d406dcd0 0x7f52d4070180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52e8102e00 0x7f52e8113990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 --2- 172.21.15.12:0/1270191602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52e81037a0 0x7f52e8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 >> 172.21.15.12:0/1270191602 conn(0x7f52e80f89b0 msgr2=0x7f52e80fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 shutdown_connections 2024-03-18T14:58:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:16.178+0000 7f52ec937700 1 -- 172.21.15.12:0/1270191602 wait complete. 2024-03-18T14:58:17.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:16 smithi100 ceph-mon[84868]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:17.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:16 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1270191602' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:58:17.220 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:16 smithi012 ceph-mon[111904]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:17.220 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:17.221 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:16 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1270191602' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:58:17.940 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:17 smithi012 ceph-mon[111904]: pgmap v88: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:18.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:17 smithi100 ceph-mon[84868]: pgmap v88: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:18.192 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:17 smithi012 systemd[1]: Stopping Ceph osd.2 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:58:18.491 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:18 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[62266]: 2024-03-18T14:58:18.191+0000 7f8d3648b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:58:18.492 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:18 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[62266]: 2024-03-18T14:58:18.191+0000 7f8d3648b700 -1 osd.2 58 *** Got signal Terminated *** 2024-03-18T14:58:18.492 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:18 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[62266]: 2024-03-18T14:58:18.191+0000 7f8d3648b700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T14:58:19.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:18 smithi100 ceph-mon[84868]: osd.2 marked itself down and dead 2024-03-18T14:58:19.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:18 smithi012 ceph-mon[111904]: osd.2 marked itself down and dead 2024-03-18T14:58:19.585 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:19 smithi012 podman[134268]: 2024-03-18 14:58:19.28446047 +0000 UTC m=+1.203185872 container died f87ceb735d2bceb4276cfff62aa44a8a77049ff46f2647bc04136cceefd8a172 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, RELEASE=quincy-6716c9f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T14:58:19.943 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:19 smithi012 ceph-mon[111904]: pgmap v89: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:19.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:19 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:58:19.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:19 smithi012 ceph-mon[111904]: osdmap e59: 6 total, 5 up, 6 in 2024-03-18T14:58:19.944 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:19 smithi012 podman[134268]: 2024-03-18 14:58:19.684094365 +0000 UTC m=+1.602819764 container cleanup f87ceb735d2bceb4276cfff62aa44a8a77049ff46f2647bc04136cceefd8a172 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, RELEASE=quincy-6716c9f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-18T14:58:19.944 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:19 smithi012 bash[134268]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2 2024-03-18T14:58:19.944 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:19 smithi012 podman[134287]: 2024-03-18 14:58:19.943936726 +0000 UTC m=+0.658401541 container remove f87ceb735d2bceb4276cfff62aa44a8a77049ff46f2647bc04136cceefd8a172 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-6716c9f, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T14:58:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:19 smithi100 ceph-mon[84868]: pgmap v89: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:19 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:58:20.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:19 smithi100 ceph-mon[84868]: osdmap e59: 6 total, 5 up, 6 in 2024-03-18T14:58:20.539 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.182360382 +0000 UTC m=+0.019198818 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:20.539 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.295049415 +0000 UTC m=+0.131887845 container create 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T14:58:20.539 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.539128136 +0000 UTC m=+0.375966574 container init 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True) 2024-03-18T14:58:20.793 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.542293348 +0000 UTC m=+0.379131788 container start 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T14:58:20.793 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.622473763 +0000 UTC m=+0.459312212 container attach 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:58:21.210 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:20 smithi012 ceph-mon[111904]: osdmap e60: 6 total, 5 up, 6 in 2024-03-18T14:58:21.210 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:20 smithi012 podman[134386]: 2024-03-18 14:58:20.845522348 +0000 UTC m=+0.682360779 container died 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-18T14:58:21.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:20 smithi100 ceph-mon[84868]: osdmap e60: 6 total, 5 up, 6 in 2024-03-18T14:58:21.800 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 podman[134416]: 2024-03-18 14:58:21.493234118 +0000 UTC m=+0.639397931 container remove 7439c161bd56edfaf8b57bfbe58582fbc4a57ce3c83d9437804da17dabd70a95 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:21.800 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-18T14:58:21.800 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 systemd[1]: Stopped Ceph osd.2 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:58:21.800 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.2.service: Consumed 55.365s CPU time. 2024-03-18T14:58:21.801 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 systemd[1]: Starting Ceph osd.2 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:58:22.071 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:21 smithi012 ceph-mon[111904]: pgmap v92: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:58:22.071 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 podman[134519]: 2024-03-18 14:58:21.84334938 +0000 UTC m=+0.018118205 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:22.071 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:21 smithi012 podman[134519]: 2024-03-18 14:58:21.960837506 +0000 UTC m=+0.135606307 container create 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-18T14:58:22.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:21 smithi100 ceph-mon[84868]: pgmap v92: 65 pgs: 9 stale+active+clean, 56 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:58:22.491 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 podman[134519]: 2024-03-18 14:58:22.18027339 +0000 UTC m=+0.355042196 container init 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-18T14:58:22.492 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 podman[134519]: 2024-03-18 14:58:22.183475059 +0000 UTC m=+0.358243865 container start 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:58:22.492 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 podman[134519]: 2024-03-18 14:58:22.280149154 +0000 UTC m=+0.454917968 container attach 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:58:22.941 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 13/216 objects degraded (6.019%), 6 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 bash[134519]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-18T14:58:22.942 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-18T14:58:22.943 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-18T14:58:22.943 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-18T14:58:22.943 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-18T14:58:22.943 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate[134534]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-18T14:58:22.943 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:22 smithi012 podman[134519]: 2024-03-18 14:58:22.661274965 +0000 UTC m=+0.836043757 container died 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-18T14:58:23.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:22 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 13/216 objects degraded (6.019%), 6 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:23.520 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 podman[134658]: 2024-03-18 14:58:23.261375177 +0000 UTC m=+0.595499860 container remove 7acc37f161abf895dc237638f91be2805084641770819ca0f20d2fcfff59ff6b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:23.520 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 podman[134724]: 2024-03-18 14:58:23.378008009 +0000 UTC m=+0.018251836 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:23.520 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 podman[134724]: 2024-03-18 14:58:23.520598563 +0000 UTC m=+0.160842378 container create f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:23.771 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 podman[134724]: 2024-03-18 14:58:23.748068785 +0000 UTC m=+0.388312607 container init f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-18T14:58:23.771 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 podman[134724]: 2024-03-18 14:58:23.751236704 +0000 UTC m=+0.391480529 container start f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-18T14:58:24.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:23 smithi012 ceph-mon[111904]: pgmap v93: 65 pgs: 6 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 44 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 13/216 objects degraded (6.019%) 2024-03-18T14:58:24.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 ceph-osd[134743]: -- 172.21.15.12:0/1063993300 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b972487680 con 0x55b972451000 2024-03-18T14:58:24.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 bash[134724]: f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a 2024-03-18T14:58:24.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:23 smithi012 systemd[1]: Started Ceph osd.2 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:58:24.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:23 smithi100 ceph-mon[84868]: pgmap v93: 65 pgs: 6 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 44 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 13/216 objects degraded (6.019%) 2024-03-18T14:58:24.686 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:24 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T14:58:24.303+0000 7f9b5ed8f7c0 -1 Falling back to public interface 2024-03-18T14:58:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:58:25.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:25.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:25.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:58:26.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:25 smithi012 ceph-mon[111904]: pgmap v94: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:26.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:25 smithi100 ceph-mon[84868]: pgmap v94: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:27.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:27 smithi012 ceph-mon[111904]: pgmap v95: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:28.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:27 smithi100 ceph-mon[84868]: pgmap v95: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:28.769 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:28 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T14:58:28.505+0000 7f9b5ed8f7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-18T14:58:29.241 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:28 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T14:58:28.988+0000 7f9b5ed8f7c0 -1 osd.2 58 log_to_monitors true 2024-03-18T14:58:30.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:29 smithi012 ceph-mon[111904]: pgmap v96: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:30.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:29 smithi012 ceph-mon[111904]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T14:58:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:30.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:30.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:29 smithi100 ceph-mon[84868]: pgmap v96: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:29 smithi100 ceph-mon[84868]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T14:58:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:30.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:31.088 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T14:58:30.824+0000 7f9b4c1df700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: osdmap e61: 6 total, 5 up, 6 in 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' 2024-03-18T14:58:31.089 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:30 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 29/216 objects degraded (13.426%), 11 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: osdmap e61: 6 total, 5 up, 6 in 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: from='osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396]' entity='osd.2' 2024-03-18T14:58:31.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:30 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 29/216 objects degraded (13.426%), 11 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:32.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: pgmap v98: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:32.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:58:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396] boot 2024-03-18T14:58:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: osdmap e62: 6 total, 6 up, 6 in 2024-03-18T14:58:32.179 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: pgmap v98: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/216 objects degraded (13.426%) 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: osd.2 [v2:172.21.15.12:6818/3733153396,v1:172.21.15.12:6819/3733153396] boot 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: osdmap e62: 6 total, 6 up, 6 in 2024-03-18T14:58:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T14:58:33.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:33 smithi012 ceph-mon[111904]: pgmap v100: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 21/216 objects degraded (9.722%) 2024-03-18T14:58:33.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:33 smithi012 ceph-mon[111904]: osdmap e63: 6 total, 6 up, 6 in 2024-03-18T14:58:34.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:33 smithi100 ceph-mon[84868]: pgmap v100: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 21/216 objects degraded (9.722%) 2024-03-18T14:58:34.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:33 smithi100 ceph-mon[84868]: osdmap e63: 6 total, 6 up, 6 in 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: pgmap v102: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/216 objects degraded (8.333%) 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:35.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:35.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:35 smithi012 ceph-mon[111904]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: pgmap v102: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/216 objects degraded (8.333%) 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:58:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:35.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:35.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:35.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:35 smithi100 ceph-mon[84868]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-18T14:58:36.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:36 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 18/216 objects degraded (8.333%), 6 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:36.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:36 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 18/216 objects degraded (8.333%), 6 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:37.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:37 smithi012 ceph-mon[111904]: pgmap v103: 65 pgs: 10 peering, 1 active+undersized+degraded, 4 active+undersized, 50 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2/216 objects degraded (0.926%) 2024-03-18T14:58:37.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:37 smithi100 ceph-mon[84868]: pgmap v103: 65 pgs: 10 peering, 1 active+undersized+degraded, 4 active+undersized, 50 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2/216 objects degraded (0.926%) 2024-03-18T14:58:38.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:38 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/216 objects degraded (0.926%), 1 pg degraded) 2024-03-18T14:58:38.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:38 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T14:58:38.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:38 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/216 objects degraded (0.926%), 1 pg degraded) 2024-03-18T14:58:38.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:38 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T14:58:39.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:39 smithi012 ceph-mon[111904]: pgmap v104: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-18T14:58:39.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:39 smithi100 ceph-mon[84868]: pgmap v104: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-18T14:58:41.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:41 smithi012 ceph-mon[111904]: pgmap v105: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T14:58:41.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:41 smithi100 ceph-mon[84868]: pgmap v105: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T14:58:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:43 smithi012 ceph-mon[111904]: pgmap v106: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-03-18T14:58:43.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:43 smithi100 ceph-mon[84868]: pgmap v106: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-03-18T14:58:45.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:45 smithi012 ceph-mon[111904]: pgmap v107: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 907 B/s rd, 0 op/s 2024-03-18T14:58:45.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:45 smithi100 ceph-mon[84868]: pgmap v107: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 907 B/s rd, 0 op/s 2024-03-18T14:58:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.292+0000 7f045a9bc700 1 -- 172.21.15.12:0/531157761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 msgr2=0x7f0454102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.292+0000 7f045a9bc700 1 --2- 172.21.15.12:0/531157761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f0454102070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0444004940 tx=0x7f0444009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 -- 172.21.15.12:0/531157761 shutdown_connections 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 --2- 172.21.15.12:0/531157761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 0x7f04540fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 --2- 172.21.15.12:0/531157761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f0454102070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 -- 172.21.15.12:0/531157761 >> 172.21.15.12:0/531157761 conn(0x7f04540f7850 msgr2=0x7f04540f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 -- 172.21.15.12:0/531157761 shutdown_connections 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 -- 172.21.15.12:0/531157761 wait complete. 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 Processor -- start 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.293+0000 7f045a9bc700 1 -- start start 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f045a9bc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f045a9bc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 0x7f045410ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f045a9bc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045410d350 con 0x7f0454101ca0 2024-03-18T14:58:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f045a9bc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045410d4c0 con 0x7f0454102640 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42878/0 (socket says 172.21.15.12:42878) 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 -- 172.21.15.12:0/3490921103 learned_addr learned my addr 172.21.15.12:0/3490921103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f04537fe700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 0x7f045410ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 -- 172.21.15.12:0/3490921103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 msgr2=0x7f045410ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 0x7f045410ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 -- 172.21.15.12:0/3490921103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444005040 con 0x7f0454101ca0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0453fff700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0444009150 tx=0x7f0444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044400e040 con 0x7f0454101ca0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0444013420 con 0x7f0454101ca0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.294+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045410d740 con 0x7f0454101ca0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.295+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044401c610 con 0x7f0454101ca0 2024-03-18T14:58:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.295+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045411a2c0 con 0x7f0454101ca0 2024-03-18T14:58:46.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.296+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04540fd3f0 con 0x7f0454101ca0 2024-03-18T14:58:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.296+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f044400baf0 con 0x7f0454101ca0 2024-03-18T14:58:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.296+0000 7f0450ff9700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 0x7f043c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.297+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f044408e670 con 0x7f0454101ca0 2024-03-18T14:58:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.297+0000 7f04537fe700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 0x7f043c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.297+0000 7f04537fe700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 0x7f043c070390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0454066bb0 tx=0x7f044800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.301+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f044405b790 con 0x7f0454101ca0 2024-03-18T14:58:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.529+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f045410d740 con 0x7f043c06dee0 2024-03-18T14:58:46.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.531+0000 7f0450ff9700 1 -- 172.21.15.12:0/3490921103 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f045410d740 con 0x7f043c06dee0 2024-03-18T14:58:46.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 msgr2=0x7f043c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 0x7f043c070390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0454066bb0 tx=0x7f044800b040 comp rx=0 tx=0).stop 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 msgr2=0x7f045410c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0444009150 tx=0x7f0444009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 shutdown_connections 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f043c06dee0 0x7f043c070390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0454101ca0 0x7f045410c8d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 --2- 172.21.15.12:0/3490921103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0454102640 0x7f045410ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 >> 172.21.15.12:0/3490921103 conn(0x7f04540f7850 msgr2=0x7f04540f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 shutdown_connections 2024-03-18T14:58:46.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.533+0000 7f045a9bc700 1 -- 172.21.15.12:0/3490921103 wait complete. 2024-03-18T14:58:46.544 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 -- 172.21.15.12:0/2191833077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf81026c0 msgr2=0x7fadf80fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 --2- 172.21.15.12:0/2191833077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf81026c0 0x7fadf80fbd30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004940 tx=0x7fadf4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 -- 172.21.15.12:0/2191833077 shutdown_connections 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 --2- 172.21.15.12:0/2191833077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf81026c0 0x7fadf80fbd30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 --2- 172.21.15.12:0/2191833077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf8101d20 0x7fadf81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 -- 172.21.15.12:0/2191833077 >> 172.21.15.12:0/2191833077 conn(0x7fadf80f7870 msgr2=0x7fadf80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:46.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 -- 172.21.15.12:0/2191833077 shutdown_connections 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.637+0000 7fadff4b5700 1 -- 172.21.15.12:0/2191833077 wait complete. 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 Processor -- start 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 -- start start 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf81026c0 0x7fadf80fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80ff8f0 con 0x7fadf8101d20 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadff4b5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80fe420 con 0x7fadf81026c0 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadfd251700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadfd251700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42894/0 (socket says 172.21.15.12:42894) 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadfd251700 1 -- 172.21.15.12:0/3888414582 learned_addr learned my addr 172.21.15.12:0/3888414582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.638+0000 7fadfca50700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf81026c0 0x7fadf80fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadfd251700 1 -- 172.21.15.12:0/3888414582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf81026c0 msgr2=0x7fadf80fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadfd251700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf81026c0 0x7fadf80fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadfd251700 1 -- 172.21.15.12:0/3888414582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4005040 con 0x7fadf8101d20 2024-03-18T14:58:46.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadfd251700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fade8002a50 tx=0x7fade80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fade8007cc0 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fade800f040 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf80fe700 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fade800da70 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.639+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf811a100 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.640+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf80fd470 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.640+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fade800dc10 con 0x7fadf8101d20 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.641+0000 7fadedffb700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 0x7fade4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.641+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fade808ccd0 con 0x7fadf8101d20 2024-03-18T14:58:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.641+0000 7fadfca50700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 0x7fade4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.641+0000 7fadfca50700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 0x7fade4070390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004890 tx=0x7fadf4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.646+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fade8056240 con 0x7fadf8101d20 2024-03-18T14:58:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.871+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf80669d0 con 0x7fade406dee0 2024-03-18T14:58:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.872+0000 7fadedffb700 1 -- 172.21.15.12:0/3888414582 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fadf80669d0 con 0x7fade406dee0 2024-03-18T14:58:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.874+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 msgr2=0x7fade4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.874+0000 7fadff4b5700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 0x7fade4070390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004890 tx=0x7fadf4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:46.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.874+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 msgr2=0x7fadf80fd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.874+0000 7fadff4b5700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fade8002a50 tx=0x7fade80046b0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 shutdown_connections 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fade406dee0 0x7fade4070390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf8101d20 0x7fadf80fd9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 --2- 172.21.15.12:0/3888414582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf81026c0 0x7fadf80fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 >> 172.21.15.12:0/3888414582 conn(0x7fadf80f7870 msgr2=0x7fadf80662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 shutdown_connections 2024-03-18T14:58:46.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.875+0000 7fadff4b5700 1 -- 172.21.15.12:0/3888414582 wait complete. 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 -- 172.21.15.12:0/1814823115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fe830 msgr2=0x7f4924106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 --2- 172.21.15.12:0/1814823115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fe830 0x7f4924106f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4918004940 tx=0x7f4918009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 -- 172.21.15.12:0/1814823115 shutdown_connections 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 --2- 172.21.15.12:0/1814823115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fe830 0x7f4924106f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 --2- 172.21.15.12:0/1814823115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fde90 0x7f49240fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 -- 172.21.15.12:0/1814823115 >> 172.21.15.12:0/1814823115 conn(0x7f4924072bc0 msgr2=0x7f4924072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 -- 172.21.15.12:0/1814823115 shutdown_connections 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 -- 172.21.15.12:0/1814823115 wait complete. 2024-03-18T14:58:46.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.978+0000 7f492a43a700 1 Processor -- start 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f492a43a700 1 -- start start 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f492a43a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f492a43a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fe830 0x7f492411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f492a43a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492411aca0 con 0x7f49240fde90 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f492a43a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924114070 con 0x7f49240fe830 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42904/0 (socket says 172.21.15.12:42904) 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 -- 172.21.15.12:0/4253124915 learned_addr learned my addr 172.21.15.12:0/4253124915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:46.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f49237fe700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fe830 0x7f492411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 -- 172.21.15.12:0/4253124915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fe830 msgr2=0x7f492411a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fe830 0x7f492411a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 -- 172.21.15.12:0/4253124915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918005040 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.979+0000 7f4923fff700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4914002a80 tx=0x7f49140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.980+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4914007cc0 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.980+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f491400f040 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.980+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4924114350 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.980+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4924114890 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.980+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491400dac0 con 0x7f49240fde90 2024-03-18T14:58:46.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.981+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4924065e30 con 0x7f49240fde90 2024-03-18T14:58:46.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.981+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f491400dcc0 con 0x7f49240fde90 2024-03-18T14:58:46.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.982+0000 7f4920ff9700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 0x7f490c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:46.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.982+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f491408d330 con 0x7f49240fde90 2024-03-18T14:58:46.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.982+0000 7f49237fe700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 0x7f490c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:46.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.982+0000 7f49237fe700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 0x7f490c070390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49241159b0 tx=0x7f4918009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:46.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:46.986+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f491405a3d0 con 0x7f49240fde90 2024-03-18T14:58:47.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.205+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f492404bf40 con 0x7f490c06dee0 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.215+0000 7f4920ff9700 1 -- 172.21.15.12:0/4253124915 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f492404bf40 con 0x7f490c06dee0 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (4m) 18s ago 15m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (2m) 18s ago 15m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (118s) 115s ago 14m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (4m) 18s ago 15m 83.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (10m) 18s ago 10m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:58:47.216 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (10m) 18s ago 10m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (10m) 115s ago 10m 16.7M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (9m) 115s ago 9m 13.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (6m) 18s ago 17m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (5m) 115s ago 14m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (3m) 18s ago 17m 51.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (2m) 115s ago 14m 38.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (5m) 18s ago 15m 15.4M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (5m) 115s ago 14m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (97s) 18s ago 13m 103M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (60s) 18s ago 13m 107M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (23s) 18s ago 12m 14.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (12m) 115s ago 12m 291M 2581M 17.2.7-684-g6716c9f3 19857db3290c b567de619d79 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (12m) 115s ago 12m 260M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:58:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (11m) 115s ago 11m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:58:47.218 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (4m) 18s ago 15m 66.3M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 msgr2=0x7f490c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 0x7f490c070390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49241159b0 tx=0x7f4918009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 msgr2=0x7f492411a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4914002a80 tx=0x7f49140046b0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 shutdown_connections 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f490c06dee0 0x7f490c070390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49240fde90 0x7f492411a110 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 --2- 172.21.15.12:0/4253124915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49240fe830 0x7f492411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 >> 172.21.15.12:0/4253124915 conn(0x7f4924072bc0 msgr2=0x7f49240fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 shutdown_connections 2024-03-18T14:58:47.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.218+0000 7f492a43a700 1 -- 172.21.15.12:0/4253124915 wait complete. 2024-03-18T14:58:47.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.319+0000 7f882e7dd700 1 -- 172.21.15.12:0/786689989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fc510 msgr2=0x7f8828109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.319+0000 7f882e7dd700 1 --2- 172.21.15.12:0/786689989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fc510 0x7f8828109270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8818004940 tx=0x7f8818009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- 172.21.15.12:0/786689989 shutdown_connections 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 --2- 172.21.15.12:0/786689989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fc510 0x7f8828109270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 --2- 172.21.15.12:0/786689989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fbc00 0x7f88280fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- 172.21.15.12:0/786689989 >> 172.21.15.12:0/786689989 conn(0x7f88280f7870 msgr2=0x7f88280f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- 172.21.15.12:0/786689989 shutdown_connections 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- 172.21.15.12:0/786689989 wait complete. 2024-03-18T14:58:47.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 Processor -- start 2024-03-18T14:58:47.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- start start 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fc510 0x7f8828116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281166a0 con 0x7f88280fbc00 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.320+0000 7f882e7dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882810fc80 con 0x7f88280fc510 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42914/0 (socket says 172.21.15.12:42914) 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 -- 172.21.15.12:0/2779474345 learned_addr learned my addr 172.21.15.12:0/2779474345 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f88277fe700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fc510 0x7f8828116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 -- 172.21.15.12:0/2779474345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fc510 msgr2=0x7f8828116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fc510 0x7f8828116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 -- 172.21.15.12:0/2779474345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8818005040 con 0x7f88280fbc00 2024-03-18T14:58:47.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8827fff700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8810002a80 tx=0x7f88100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8810007cc0 con 0x7f88280fbc00 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f881000f040 con 0x7f88280fbc00 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882810ff60 con 0x7f88280fbc00 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881000dac0 con 0x7f88280fbc00 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.321+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8828110520 con 0x7f88280fbc00 2024-03-18T14:58:47.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.322+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8828066af0 con 0x7f88280fbc00 2024-03-18T14:58:47.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.323+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f881000dc20 con 0x7f88280fbc00 2024-03-18T14:58:47.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.323+0000 7f8824ff9700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 0x7f8814070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.323+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f881008d460 con 0x7f88280fbc00 2024-03-18T14:58:47.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.323+0000 7f88277fe700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 0x7f8814070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.324+0000 7f88277fe700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 0x7f8814070390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88281113b0 tx=0x7f8818009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.328+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f881005a500 con 0x7f88280fbc00 2024-03-18T14:58:47.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: pgmap v108: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:47.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:47.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:47.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:47.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:47.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:47 smithi012 ceph-mon[111904]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: pgmap v108: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:47.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:47 smithi100 ceph-mon[84868]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:47.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.621+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f882804c710 con 0x7f88280fbc00 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.621+0000 7f8824ff9700 1 -- 172.21.15.12:0/2779474345 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f881005a090 con 0x7f88280fbc00 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:58:47.622 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 3, 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 7, 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 7 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:58:47.623 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 msgr2=0x7f8814070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 0x7f8814070390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88281113b0 tx=0x7f8818009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 msgr2=0x7f8828115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8810002a80 tx=0x7f88100046b0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 shutdown_connections 2024-03-18T14:58:47.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f881406dee0 0x7f8814070390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88280fbc00 0x7f8828115b10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 --2- 172.21.15.12:0/2779474345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88280fc510 0x7f8828116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.623+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 >> 172.21.15.12:0/2779474345 conn(0x7f88280f7870 msgr2=0x7f88280f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:47.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.624+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 shutdown_connections 2024-03-18T14:58:47.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.624+0000 7f882e7dd700 1 -- 172.21.15.12:0/2779474345 wait complete. 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- 172.21.15.12:0/1785589068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 msgr2=0x7fa01c0fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 --2- 172.21.15.12:0/1785589068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c0fbac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa010004940 tx=0x7fa010009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- 172.21.15.12:0/1785589068 shutdown_connections 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 --2- 172.21.15.12:0/1785589068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c0fbac0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 --2- 172.21.15.12:0/1785589068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 0x7fa01c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- 172.21.15.12:0/1785589068 >> 172.21.15.12:0/1785589068 conn(0x7fa01c071530 msgr2=0x7fa01c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- 172.21.15.12:0/1785589068 shutdown_connections 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- 172.21.15.12:0/1785589068 wait complete. 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 Processor -- start 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.726+0000 7fa020fba700 1 -- start start 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa020fba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa020fba700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 0x7fa01c113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa020fba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1144f0 con 0x7fa01c0fb650 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa020fba700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c10daf0 con 0x7fa01c1015c0 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42942/0 (socket says 172.21.15.12:42942) 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 -- 172.21.15.12:0/2257628869 learned_addr learned my addr 172.21.15.12:0/2257628869 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa019d9b700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 0x7fa01c113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 -- 172.21.15.12:0/2257628869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 msgr2=0x7fa01c113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 0x7fa01c113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 -- 172.21.15.12:0/2257628869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010005040 con 0x7fa01c0fb650 2024-03-18T14:58:47.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa01a59c700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00eb40 tx=0x7fa00c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c007cc0 con 0x7fa01c0fb650 2024-03-18T14:58:47.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c10dd70 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.727+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa00c00f040 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.728+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c00ca10 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.728+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c10e280 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.728+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c103a70 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.729+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa00c00cc80 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.729+0000 7fa00affd700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 0x7fa004070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.729+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fa00c08d260 con 0x7fa01c0fb650 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.729+0000 7fa019d9b700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 0x7fa004070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:47.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.730+0000 7fa019d9b700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 0x7fa004070390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa010009b30 tx=0x7fa010009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:47.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.734+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa00c05a3b0 con 0x7fa01c0fb650 2024-03-18T14:58:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.988+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa01c04c740 con 0x7fa01c0fb650 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.988+0000 7fa00affd700 1 -- 172.21.15.12:0/2257628869 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fa00c059f40 con 0x7fa01c0fb650 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:58:47.989 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:58:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:58:47.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 msgr2=0x7fa004070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 0x7fa004070390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa010009b30 tx=0x7fa010009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 msgr2=0x7fa01c113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00eb40 tx=0x7fa00c00b040 comp rx=0 tx=0).stop 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 shutdown_connections 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa00406dee0 0x7fa004070390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01c0fb650 0x7fa01c113960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 --2- 172.21.15.12:0/2257628869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa01c1015c0 0x7fa01c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 >> 172.21.15.12:0/2257628869 conn(0x7fa01c071530 msgr2=0x7fa01c0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 shutdown_connections 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:47.990+0000 7fa020fba700 1 -- 172.21.15.12:0/2257628869 wait complete. 2024-03-18T14:58:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 -- 172.21.15.12:0/2875221707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac101530 msgr2=0x7f63ac101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 --2- 172.21.15.12:0/2875221707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac101530 0x7f63ac101900 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f63a4004940 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 -- 172.21.15.12:0/2875221707 shutdown_connections 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 --2- 172.21.15.12:0/2875221707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac0fb570 0x7f63ac0fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 --2- 172.21.15.12:0/2875221707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac101530 0x7f63ac101900 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 -- 172.21.15.12:0/2875221707 >> 172.21.15.12:0/2875221707 conn(0x7f63ac0714c0 msgr2=0x7f63ac0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 -- 172.21.15.12:0/2875221707 shutdown_connections 2024-03-18T14:58:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 -- 172.21.15.12:0/2875221707 wait complete. 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.094+0000 7f63b4651700 1 Processor -- start 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 -- start start 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac101530 0x7f63ac113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac114500 con 0x7f63ac0fb570 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac10db20 con 0x7f63ac101530 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42964/0 (socket says 172.21.15.12:42964) 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 -- 172.21.15.12:0/4221541811 learned_addr learned my addr 172.21.15.12:0/4221541811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b1bec700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac101530 0x7f63ac113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 -- 172.21.15.12:0/4221541811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac101530 msgr2=0x7f63ac113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac101530 0x7f63ac113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 -- 172.21.15.12:0/4221541811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4005040 con 0x7f63ac0fb570 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b23ed700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009150 tx=0x7f63a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a400e040 con 0x7f63ac0fb570 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63a4013420 con 0x7f63ac0fb570 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac10dda0 con 0x7f63ac0fb570 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.095+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a401c630 con 0x7f63ac0fb570 2024-03-18T14:58:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.096+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac10e360 con 0x7f63ac0fb570 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.097+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac1039e0 con 0x7f63ac0fb570 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.097+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f63a4013590 con 0x7f63ac0fb570 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.097+0000 7f63a2ffd700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 0x7f6398070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.098+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f63a408e930 con 0x7f63ac0fb570 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.098+0000 7f63b1bec700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 0x7f6398070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.098+0000 7f63b1bec700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 0x7f6398070390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63ac10f210 tx=0x7f639c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:48.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.102+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63a4057f20 con 0x7f63ac0fb570 2024-03-18T14:58:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:48 smithi012 ceph-mon[111904]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:48 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2779474345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:48 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2257628869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:58:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.332+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63ac109ff0 con 0x7f639806dee0 2024-03-18T14:58:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.333+0000 7f63a2ffd700 1 -- 172.21.15.12:0/4221541811 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f63ac109ff0 con 0x7f639806dee0 2024-03-18T14:58:48.334 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:58:48.334 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:58:48.337 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:58:48.337 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:58:48.337 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "9/21 daemons upgraded", 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:58:48.338 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 msgr2=0x7f6398070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 0x7f6398070390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63ac10f210 tx=0x7f639c00b040 comp rx=0 tx=0).stop 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 msgr2=0x7f63ac113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009150 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 shutdown_connections 2024-03-18T14:58:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f639806dee0 0x7f6398070390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ac0fb570 0x7f63ac113970 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 --2- 172.21.15.12:0/4221541811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac101530 0x7f63ac113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 >> 172.21.15.12:0/4221541811 conn(0x7f63ac0714c0 msgr2=0x7f63ac0fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 shutdown_connections 2024-03-18T14:58:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.336+0000 7f63b4651700 1 -- 172.21.15.12:0/4221541811 wait complete. 2024-03-18T14:58:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 -- 172.21.15.12:0/2681372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 msgr2=0x7f6ffc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 --2- 172.21.15.12:0/2681372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc0fce10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4004940 tx=0x7f6fe4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 -- 172.21.15.12:0/2681372898 shutdown_connections 2024-03-18T14:58:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 --2- 172.21.15.12:0/2681372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc0fce10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 --2- 172.21.15.12:0/2681372898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 -- 172.21.15.12:0/2681372898 >> 172.21.15.12:0/2681372898 conn(0x7f6ffc0f89b0 msgr2=0x7f6ffc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 -- 172.21.15.12:0/2681372898 shutdown_connections 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.441+0000 7f700218f700 1 -- 172.21.15.12:0/2681372898 wait complete. 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 Processor -- start 2024-03-18T14:58:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 -- start start 2024-03-18T14:58:48.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc114470 con 0x7f6ffc1037a0 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f700218f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc10da70 con 0x7f6ffc102e00 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f6ff2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f6ff2ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40798/0 (socket says 172.21.15.12:40798) 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f6ff2ffd700 1 -- 172.21.15.12:0/3212417958 learned_addr learned my addr 172.21.15.12:0/3212417958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.442+0000 7f6ffb7fe700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff2ffd700 1 -- 172.21.15.12:0/3212417958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 msgr2=0x7f6ffc1138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff2ffd700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff2ffd700 1 -- 172.21.15.12:0/3212417958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4005040 con 0x7f6ffc1037a0 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ffb7fe700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:58:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff2ffd700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009180 tx=0x7f6fe4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe400e040 con 0x7f6ffc1037a0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fe400bbc0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe401c3f0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc10dcf0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.443+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc10e2b0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.444+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc1052d0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.445+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6fe400bd30 con 0x7f6ffc1037a0 2024-03-18T14:58:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.445+0000 7f6ff8ff9700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 0x7f6fe8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:58:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.445+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6fe408e8e0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.445+0000 7f6ffb7fe700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 0x7f6fe8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:58:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.445+0000 7f6ffb7fe700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 0x7f6fe8070390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6fec007fd0 tx=0x7f6fec009f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:58:48.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.450+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fe405baa0 con 0x7f6ffc1037a0 2024-03-18T14:58:48.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:48 smithi100 ceph-mon[84868]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:48.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:48 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2779474345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:58:48.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:48 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2257628869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:58:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.745+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6ffc0fe550 con 0x7f6ffc1037a0 2024-03-18T14:58:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.746+0000 7f6ff8ff9700 1 -- 172.21.15.12:0/3212417958 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6fe405b630 con 0x7f6ffc1037a0 2024-03-18T14:58:48.747 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 msgr2=0x7f6fe8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 0x7f6fe8070390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6fec007fd0 tx=0x7f6fec009f60 comp rx=0 tx=0).stop 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 msgr2=0x7f6ffc113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009180 tx=0x7f6fe4009e30 comp rx=0 tx=0).stop 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 shutdown_connections 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6fe806dee0 0x7f6fe8070390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc102e00 0x7f6ffc1138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 --2- 172.21.15.12:0/3212417958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ffc1037a0 0x7f6ffc113e20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 >> 172.21.15.12:0/3212417958 conn(0x7f6ffc0f89b0 msgr2=0x7f6ffc102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 shutdown_connections 2024-03-18T14:58:48.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:58:48.750+0000 7f700218f700 1 -- 172.21.15.12:0/3212417958 wait complete. 2024-03-18T14:58:49.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:49 smithi012 ceph-mon[111904]: pgmap v109: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:49.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:49 smithi012 ceph-mon[111904]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:49.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:49 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3212417958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:58:49.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:49 smithi100 ceph-mon[84868]: pgmap v109: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:49.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:49 smithi100 ceph-mon[84868]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:58:49.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:49 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3212417958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:58:50.363 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:50 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:50.363 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:50 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:50.363 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:50 smithi100 ceph-mon[84868]: Upgrade: osd.3 is safe to restart 2024-03-18T14:58:50.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:50 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:50 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-18T14:58:50.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:50 smithi012 ceph-mon[111904]: Upgrade: osd.3 is safe to restart 2024-03-18T14:58:51.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:51 smithi100 ceph-mon[84868]: pgmap v110: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T14:58:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:51 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:51 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:58:51.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:51 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:51.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:51 smithi012 ceph-mon[111904]: pgmap v110: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T14:58:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:58:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T14:58:51.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:58:52.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:52 smithi012 ceph-mon[111904]: Upgrade: Updating osd.3 2024-03-18T14:58:52.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:52 smithi012 ceph-mon[111904]: Deploying daemon osd.3 on smithi100 2024-03-18T14:58:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:52 smithi100 ceph-mon[84868]: Upgrade: Updating osd.3 2024-03-18T14:58:52.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:52 smithi100 ceph-mon[84868]: Deploying daemon osd.3 on smithi100 2024-03-18T14:58:53.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:53 smithi012 ceph-mon[111904]: pgmap v111: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:53.537 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:53 smithi100 ceph-mon[84868]: pgmap v111: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:58:54.452 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:54 smithi100 systemd[1]: Stopping Ceph osd.3 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:58:54.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:54 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[43219]: 2024-03-18T14:58:54.451+0000 7f2e1740a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:58:54.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:54 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[43219]: 2024-03-18T14:58:54.451+0000 7f2e1740a700 -1 osd.3 63 *** Got signal Terminated *** 2024-03-18T14:58:54.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:54 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[43219]: 2024-03-18T14:58:54.451+0000 7f2e1740a700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T14:58:55.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:55 smithi012 ceph-mon[111904]: pgmap v112: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:55 smithi012 ceph-mon[111904]: osd.3 marked itself down and dead 2024-03-18T14:58:55.525 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:55 smithi100 ceph-mon[84868]: pgmap v112: 65 pgs: 65 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:58:55.525 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:55 smithi100 ceph-mon[84868]: osd.3 marked itself down and dead 2024-03-18T14:58:55.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:55 smithi100 podman[93612]: 2024-03-18 14:58:55.524930367 +0000 UTC m=+1.188333311 container died b567de619d795c633713df0b79256e6456c464cf1f0e750dca9fde29cce0e185 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-6716c9f, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-18T14:58:56.162 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:55 smithi100 podman[93612]: 2024-03-18 14:58:55.858671513 +0000 UTC m=+1.522074458 container cleanup b567de619d795c633713df0b79256e6456c464cf1f0e750dca9fde29cce0e185 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-6716c9f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:56.163 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:55 smithi100 bash[93612]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3 2024-03-18T14:58:56.163 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93629]: 2024-03-18 14:58:56.084982509 +0000 UTC m=+0.558869636 container remove b567de619d795c633713df0b79256e6456c464cf1f0e750dca9fde29cce0e185 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=quincy-6716c9f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T14:58:56.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:56 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:58:56.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:56 smithi012 ceph-mon[111904]: osdmap e64: 6 total, 5 up, 6 in 2024-03-18T14:58:56.533 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:56 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:58:56.533 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:56 smithi100 ceph-mon[84868]: osdmap e64: 6 total, 5 up, 6 in 2024-03-18T14:58:56.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93719]: 2024-03-18 14:58:56.435543244 +0000 UTC m=+0.018301005 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:56.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93719]: 2024-03-18 14:58:56.586038238 +0000 UTC m=+0.168795994 container create 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:58:56.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93719]: 2024-03-18 14:58:56.772421937 +0000 UTC m=+0.355179701 container init 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-18T14:58:56.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93719]: 2024-03-18 14:58:56.775943655 +0000 UTC m=+0.358701416 container start 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-18T14:58:57.084 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:56 smithi100 podman[93719]: 2024-03-18 14:58:56.830153028 +0000 UTC m=+0.412910785 container attach 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-18T14:58:57.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:57 smithi012 ceph-mon[111904]: pgmap v114: 65 pgs: 16 stale+active+clean, 49 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:58:57.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:57 smithi012 ceph-mon[111904]: osdmap e65: 6 total, 5 up, 6 in 2024-03-18T14:58:57.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:57 smithi100 ceph-mon[84868]: pgmap v114: 65 pgs: 16 stale+active+clean, 49 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:58:57.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:57 smithi100 ceph-mon[84868]: osdmap e65: 6 total, 5 up, 6 in 2024-03-18T14:58:57.545 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 podman[93745]: 2024-03-18 14:58:57.084821778 +0000 UTC m=+0.019348673 container died 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:58:58.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 podman[93745]: 2024-03-18 14:58:57.65026621 +0000 UTC m=+0.584793105 container remove 569cbf5b673ddccf52e02ce0c98f908750ff901ee09c30c4b3c9886ccc239cff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-18T14:58:58.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-18T14:58:58.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 systemd[1]: Stopped Ceph osd.3 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:58:58.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.3.service: Consumed 1min 15.046s CPU time. 2024-03-18T14:58:58.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:57 smithi100 systemd[1]: Starting Ceph osd.3 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:58:58.395 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 32/216 objects degraded (14.815%), 10 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:58.395 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:57.997148978 +0000 UTC m=+0.019013803 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:58.395 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:58.101264743 +0000 UTC m=+0.123129560 container create a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:58:58.395 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:58.320316947 +0000 UTC m=+0.342181763 container init a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:58:58.395 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:58.323744853 +0000 UTC m=+0.345609670 container start a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-18T14:58:58.395 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:58.395326185 +0000 UTC m=+0.417191012 container attach a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-18T14:58:58.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:58 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 32/216 objects degraded (14.815%), 10 pgs degraded (PG_DEGRADED) 2024-03-18T14:58:59.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-18T14:58:59.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-18T14:58:59.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-18T14:58:59.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 bash[93853]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-18T14:58:59.046 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-18T14:58:59.047 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate[93868]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-18T14:58:59.047 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:58 smithi100 podman[93853]: 2024-03-18 14:58:58.78258655 +0000 UTC m=+0.804451367 container died a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-18T14:58:59.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:58:59 smithi012 ceph-mon[111904]: pgmap v116: 65 pgs: 10 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 37 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32/216 objects degraded (14.815%) 2024-03-18T14:58:59.524 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:58:59 smithi100 ceph-mon[84868]: pgmap v116: 65 pgs: 10 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 37 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32/216 objects degraded (14.815%) 2024-03-18T14:58:59.525 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 podman[93993]: 2024-03-18 14:58:59.315819218 +0000 UTC m=+0.522660875 container remove a84e53b6c5c3541be815ec4a32c3fbc59c77160bd22102c54f4b46b72583ad45 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:59.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 podman[94055]: 2024-03-18 14:58:59.525115224 +0000 UTC m=+0.111649310 container create 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:58:59.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 podman[94055]: 2024-03-18 14:58:59.431235838 +0000 UTC m=+0.017769888 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:58:59.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 podman[94055]: 2024-03-18 14:58:59.727334811 +0000 UTC m=+0.313868853 container init 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-18T14:58:59.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 podman[94055]: 2024-03-18 14:58:59.730192018 +0000 UTC m=+0.316726062 container start 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:58:59.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 ceph-osd[94073]: -- 172.21.15.100:0/3546817835 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602b7b2f680 con 0x5602b7af9000 2024-03-18T14:59:00.271 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 bash[94055]: 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb 2024-03-18T14:59:00.272 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:58:59 smithi100 systemd[1]: Started Ceph osd.3 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:59:00.544 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:59:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T14:59:00.271+0000 7fb5ecb6a7c0 -1 Falling back to public interface 2024-03-18T14:59:01.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:00 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:01.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:00 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:01.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:00 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:01.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:00 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:00 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:00 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:02.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:01 smithi100 ceph-mon[84868]: pgmap v117: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:02.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:02.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:02.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:01 smithi012 ceph-mon[111904]: pgmap v117: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:02.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:04.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:03 smithi012 ceph-mon[111904]: pgmap v118: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:04.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:03 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:03 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:03 smithi100 ceph-mon[84868]: pgmap v118: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:03 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:04.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:03 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:05.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:05.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:04 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:04 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:05.295 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:59:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T14:59:04.879+0000 7fb5ecb6a7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-18T14:59:05.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:59:05 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T14:59:05.468+0000 7fb5ecb6a7c0 -1 osd.3 63 log_to_monitors true 2024-03-18T14:59:06.123 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:05 smithi100 ceph-mon[84868]: pgmap v119: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:06.123 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:05 smithi100 ceph-mon[84868]: from='osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:59:06.124 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:05 smithi100 ceph-mon[84868]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:59:06.124 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:05 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 48/216 objects degraded (22.222%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:06.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:05 smithi012 ceph-mon[111904]: pgmap v119: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:06.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:05 smithi012 ceph-mon[111904]: from='osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:59:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:05 smithi012 ceph-mon[111904]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T14:59:06.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:05 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 48/216 objects degraded (22.222%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:07.193 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T14:59:06.896+0000 7fb5d9fba700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:59:07.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-mon[84868]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T14:59:07.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-mon[84868]: osdmap e66: 6 total, 5 up, 6 in 2024-03-18T14:59:07.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-mon[84868]: from='osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:07.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-mon[84868]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:07.193 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:06 smithi100 ceph-mon[84868]: from='osd.3 ' entity='osd.3' 2024-03-18T14:59:07.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:06 smithi012 ceph-mon[111904]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T14:59:07.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:06 smithi012 ceph-mon[111904]: osdmap e66: 6 total, 5 up, 6 in 2024-03-18T14:59:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:06 smithi012 ceph-mon[111904]: from='osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:06 smithi012 ceph-mon[111904]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:07.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:06 smithi012 ceph-mon[111904]: from='osd.3 ' entity='osd.3' 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: pgmap v121: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:08.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:59:08.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834] boot 2024-03-18T14:59:08.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: osdmap e67: 6 total, 6 up, 6 in 2024-03-18T14:59:08.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:07 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: pgmap v121: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/216 objects degraded (22.222%) 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:08.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:08.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:59:08.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: osd.3 [v2:172.21.15.100:6800/1440116834,v1:172.21.15.100:6801/1440116834] boot 2024-03-18T14:59:08.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: osdmap e67: 6 total, 6 up, 6 in 2024-03-18T14:59:08.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:07 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T14:59:09.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:08 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:09.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:08 smithi100 ceph-mon[84868]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-18T14:59:09.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:08 smithi100 ceph-mon[84868]: osdmap e68: 6 total, 6 up, 6 in 2024-03-18T14:59:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:08 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:08 smithi012 ceph-mon[111904]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-18T14:59:09.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:08 smithi012 ceph-mon[111904]: osdmap e68: 6 total, 6 up, 6 in 2024-03-18T14:59:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:09 smithi100 ceph-mon[84868]: pgmap v123: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 48/216 objects degraded (22.222%) 2024-03-18T14:59:10.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:09 smithi012 ceph-mon[111904]: pgmap v123: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 48/216 objects degraded (22.222%) 2024-03-18T14:59:11.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:11 smithi012 ceph-mon[111904]: pgmap v125: 65 pgs: 7 peering, 11 active+undersized+degraded, 17 active+undersized, 30 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 33/216 objects degraded (15.278%) 2024-03-18T14:59:11.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:11 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 33/216 objects degraded (15.278%), 11 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:11.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:11 smithi100 ceph-mon[84868]: pgmap v125: 65 pgs: 7 peering, 11 active+undersized+degraded, 17 active+undersized, 30 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 33/216 objects degraded (15.278%) 2024-03-18T14:59:11.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:11 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 33/216 objects degraded (15.278%), 11 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:12.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:12 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/216 objects degraded (15.278%), 11 pgs degraded) 2024-03-18T14:59:12.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:12 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T14:59:12.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:12 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/216 objects degraded (15.278%), 11 pgs degraded) 2024-03-18T14:59:12.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:12 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T14:59:13.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:13 smithi012 ceph-mon[111904]: pgmap v126: 65 pgs: 7 peering, 58 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:59:13.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:13 smithi100 ceph-mon[84868]: pgmap v126: 65 pgs: 7 peering, 58 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-18T14:59:15.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:15 smithi012 ceph-mon[111904]: pgmap v127: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-03-18T14:59:15.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:15 smithi100 ceph-mon[84868]: pgmap v127: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-03-18T14:59:17.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:17 smithi012 ceph-mon[111904]: pgmap v128: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 872 B/s rd, 2 op/s 2024-03-18T14:59:17.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:17.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:17 smithi100 ceph-mon[84868]: pgmap v128: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 872 B/s rd, 2 op/s 2024-03-18T14:59:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:17.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 -- 172.21.15.12:0/2919598109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 msgr2=0x7fd550109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 --2- 172.21.15.12:0/2919598109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550109170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd544004970 tx=0x7fd544009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 -- 172.21.15.12:0/2919598109 shutdown_connections 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 --2- 172.21.15.12:0/2919598109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550109170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 --2- 172.21.15.12:0/2919598109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 0x7fd5500706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 -- 172.21.15.12:0/2919598109 >> 172.21.15.12:0/2919598109 conn(0x7fd5500f89b0 msgr2=0x7fd5500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 -- 172.21.15.12:0/2919598109 shutdown_connections 2024-03-18T14:59:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 -- 172.21.15.12:0/2919598109 wait complete. 2024-03-18T14:59:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.873+0000 7fd555ffd700 1 Processor -- start 2024-03-18T14:59:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd555ffd700 1 -- start start 2024-03-18T14:59:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd555ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 0x7fd550115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd555ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd555ffd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5501167e0 con 0x7fd550070c10 2024-03-18T14:59:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd555ffd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55010fd30 con 0x7fd550070300 2024-03-18T14:59:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48948/0 (socket says 172.21.15.12:48948) 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 -- 172.21.15.12:0/1067030746 learned_addr learned my addr 172.21.15.12:0/1067030746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54f7fe700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 0x7fd550115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 -- 172.21.15.12:0/1067030746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 msgr2=0x7fd550115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 0x7fd550115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 -- 172.21.15.12:0/1067030746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd544005040 con 0x7fd550070c10 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.874+0000 7fd54effd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd544004f50 tx=0x7fd544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.875+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54400e040 con 0x7fd550070c10 2024-03-18T14:59:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.875+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55010ff50 con 0x7fd550070c10 2024-03-18T14:59:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.875+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd544009400 con 0x7fd550070c10 2024-03-18T14:59:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.875+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54401c3f0 con 0x7fd550070c10 2024-03-18T14:59:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.875+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd550110490 con 0x7fd550070c10 2024-03-18T14:59:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.876+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550071a40 con 0x7fd550070c10 2024-03-18T14:59:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.876+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd544013420 con 0x7fd550070c10 2024-03-18T14:59:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.877+0000 7fd537fff700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 0x7fd538070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.877+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fd54408e7e0 con 0x7fd550070c10 2024-03-18T14:59:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.877+0000 7fd54f7fe700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 0x7fd538070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.877+0000 7fd54f7fe700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 0x7fd538070390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd540009bf0 tx=0x7fd5400091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:18.882+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd54405b9a0 con 0x7fd550070c10 2024-03-18T14:59:19.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.110+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd550110700 con 0x7fd53806dee0 2024-03-18T14:59:19.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.111+0000 7fd537fff700 1 -- 172.21.15.12:0/1067030746 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd550110700 con 0x7fd53806dee0 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 msgr2=0x7fd538070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 0x7fd538070390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd540009bf0 tx=0x7fd5400091a0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 msgr2=0x7fd550116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd544004f50 tx=0x7fd544009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 shutdown_connections 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd53806dee0 0x7fd538070390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd550070300 0x7fd550115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 --2- 172.21.15.12:0/1067030746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd550070c10 0x7fd550116100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 >> 172.21.15.12:0/1067030746 conn(0x7fd5500f89b0 msgr2=0x7fd5500f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 shutdown_connections 2024-03-18T14:59:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.113+0000 7fd555ffd700 1 -- 172.21.15.12:0/1067030746 wait complete. 2024-03-18T14:59:19.124 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 -- 172.21.15.12:0/2582041024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 msgr2=0x7f65180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 --2- 172.21.15.12:0/2582041024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f65180fce10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6508004940 tx=0x7f6508009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 -- 172.21.15.12:0/2582041024 shutdown_connections 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 --2- 172.21.15.12:0/2582041024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f65180fce10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 --2- 172.21.15.12:0/2582041024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f65181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.217+0000 7f651f6a8700 1 -- 172.21.15.12:0/2582041024 >> 172.21.15.12:0/2582041024 conn(0x7f65180f89b0 msgr2=0x7f65180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 -- 172.21.15.12:0/2582041024 shutdown_connections 2024-03-18T14:59:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 -- 172.21.15.12:0/2582041024 wait complete. 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 Processor -- start 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 -- start start 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f6518113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518114510 con 0x7f65181037a0 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651f6a8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651810db10 con 0x7f6518102e00 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651cc43700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651cc43700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48964/0 (socket says 172.21.15.12:48964) 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.218+0000 7f651cc43700 1 -- 172.21.15.12:0/605244681 learned_addr learned my addr 172.21.15.12:0/605244681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651d444700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f6518113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651cc43700 1 -- 172.21.15.12:0/605244681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 msgr2=0x7f6518113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651cc43700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f6518113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651cc43700 1 -- 172.21.15.12:0/605244681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6508005040 con 0x7f65181037a0 2024-03-18T14:59:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651d444700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f6518113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651cc43700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6508009180 tx=0x7f6508009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650800e040 con 0x7f65181037a0 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650800bbc0 con 0x7f65181037a0 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651810dd90 con 0x7f65181037a0 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650801c3f0 con 0x7f65181037a0 2024-03-18T14:59:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.219+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651810e350 con 0x7f65181037a0 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.220+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65181052d0 con 0x7f65181037a0 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.221+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f650800bd30 con 0x7f65181037a0 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.221+0000 7f650dffb700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 0x7f6504070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.221+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f650808e9d0 con 0x7f65181037a0 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.221+0000 7f651d444700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 0x7f6504070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.222+0000 7f651d444700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 0x7f6504070390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6514007fd0 tx=0x7f6514009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.226+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f650805baf0 con 0x7f65181037a0 2024-03-18T14:59:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.455+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f651804c7b0 con 0x7f650406dee0 2024-03-18T14:59:19.456 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:19 smithi012 ceph-mon[111904]: pgmap v129: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s 2024-03-18T14:59:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.456+0000 7f650dffb700 1 -- 172.21.15.12:0/605244681 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f651804c7b0 con 0x7f650406dee0 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 msgr2=0x7f6504070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 0x7f6504070390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6514007fd0 tx=0x7f6514009040 comp rx=0 tx=0).stop 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 msgr2=0x7f6518113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6508009180 tx=0x7f6508009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 shutdown_connections 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f650406dee0 0x7f6504070390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518102e00 0x7f6518113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 --2- 172.21.15.12:0/605244681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65181037a0 0x7f6518113ec0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 >> 172.21.15.12:0/605244681 conn(0x7f65180f89b0 msgr2=0x7f65181021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 shutdown_connections 2024-03-18T14:59:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.458+0000 7f651f6a8700 1 -- 172.21.15.12:0/605244681 wait complete. 2024-03-18T14:59:19.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:19 smithi100 ceph-mon[84868]: pgmap v129: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 1 op/s 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 -- 172.21.15.12:0/1919942773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 msgr2=0x7faf44101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 --2- 172.21.15.12:0/1919942773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf44101990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf34004940 tx=0x7faf34009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 -- 172.21.15.12:0/1919942773 shutdown_connections 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 --2- 172.21.15.12:0/1919942773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 0x7faf440fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 --2- 172.21.15.12:0/1919942773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf44101990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.561+0000 7faf4949c700 1 -- 172.21.15.12:0/1919942773 >> 172.21.15.12:0/1919942773 conn(0x7faf44071530 msgr2=0x7faf44071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 -- 172.21.15.12:0/1919942773 shutdown_connections 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 -- 172.21.15.12:0/1919942773 wait complete. 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 Processor -- start 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 -- start start 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 0x7faf4410eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4410f5d0 con 0x7faf441015c0 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.562+0000 7faf4949c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4410f710 con 0x7faf440fb650 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48980/0 (socket says 172.21.15.12:48980) 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 -- 172.21.15.12:0/1207675679 learned_addr learned my addr 172.21.15.12:0/1207675679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf42ffd700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 0x7faf4410eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 -- 172.21.15.12:0/1207675679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 msgr2=0x7faf4410eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 0x7faf4410eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 -- 172.21.15.12:0/1207675679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34005040 con 0x7faf441015c0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf427fc700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf38002a50 tx=0x7faf380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf38007cc0 con 0x7faf441015c0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf3800f040 con 0x7faf441015c0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf44117bc0 con 0x7faf441015c0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3800da30 con 0x7faf441015c0 2024-03-18T14:59:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.563+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf44117f90 con 0x7faf441015c0 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.564+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf440fcd20 con 0x7faf441015c0 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.565+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7faf3800db90 con 0x7faf441015c0 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.565+0000 7faf2b7fe700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 0x7faf2c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.565+0000 7faf42ffd700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 0x7faf2c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.565+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7faf3808d430 con 0x7faf441015c0 2024-03-18T14:59:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.566+0000 7faf42ffd700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 0x7faf2c070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf34004f90 tx=0x7faf34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.570+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf380569a0 con 0x7faf441015c0 2024-03-18T14:59:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.785+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf440fa170 con 0x7faf2c06dee0 2024-03-18T14:59:19.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.795+0000 7faf2b7fe700 1 -- 172.21.15.12:0/1207675679 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faf440fa170 con 0x7faf2c06dee0 2024-03-18T14:59:19.796 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:59:19.797 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (5m) 50s ago 16m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:59:19.797 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (2m) 50s ago 16m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:59:19.797 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (2m) 16s ago 15m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:59:19.797 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (4m) 50s ago 15m 83.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:59:19.798 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (10m) 50s ago 10m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:59:19.798 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (10m) 50s ago 10m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:59:19.798 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (10m) 16s ago 10m 17.2M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:59:19.798 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (10m) 16s ago 10m 14.1M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:59:19.798 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (6m) 50s ago 17m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:59:19.799 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (6m) 16s ago 15m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:59:19.799 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (3m) 50s ago 17m 51.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:59:19.799 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (3m) 16s ago 15m 40.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:59:19.799 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (5m) 50s ago 16m 15.4M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:59:19.800 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (5m) 16s ago 15m 16.1M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:59:19.800 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (2m) 50s ago 14m 103M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T14:59:19.800 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (92s) 50s ago 13m 107M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T14:59:19.800 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (56s) 50s ago 13m 14.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T14:59:19.800 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (20s) 16s ago 13m 11.6M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T14:59:19.801 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (12m) 16s ago 12m 261M 2581M 17.2.7-684-g6716c9f3 19857db3290c 2e73a5239141 2024-03-18T14:59:19.801 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (12m) 16s ago 12m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:59:19.801 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (5m) 50s ago 15m 66.3M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:59:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 msgr2=0x7faf2c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 0x7faf2c070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf34004f90 tx=0x7faf34009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 msgr2=0x7faf4410f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf38002a50 tx=0x7faf380046b0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 shutdown_connections 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faf2c06dee0 0x7faf2c070390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf440fb650 0x7faf4410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 --2- 172.21.15.12:0/1207675679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf441015c0 0x7faf4410f090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 >> 172.21.15.12:0/1207675679 conn(0x7faf44071530 msgr2=0x7faf440f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 shutdown_connections 2024-03-18T14:59:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.797+0000 7faf4949c700 1 -- 172.21.15.12:0/1207675679 wait complete. 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/3933349052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b380fb540 msgr2=0x7f6b380fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/3933349052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b380fb540 0x7f6b380fb990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b20004940 tx=0x7f6b20009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/3933349052 shutdown_connections 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/3933349052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b380fb540 0x7f6b380fb990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/3933349052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b38102fd0 0x7f6b381033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/3933349052 >> 172.21.15.12:0/3933349052 conn(0x7f6b38071530 msgr2=0x7f6b38071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/3933349052 shutdown_connections 2024-03-18T14:59:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.899+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/3933349052 wait complete. 2024-03-18T14:59:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 Processor -- start 2024-03-18T14:59:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 -- start start 2024-03-18T14:59:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b380fb540 0x7f6b380fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380fe700 con 0x7f6b38102fd0 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b3d9c7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380fd1a0 con 0x7f6b380fb540 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48996/0 (socket says 172.21.15.12:48996) 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 -- 172.21.15.12:0/1020425263 learned_addr learned my addr 172.21.15.12:0/1020425263 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b36ffd700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b380fb540 0x7f6b380fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 -- 172.21.15.12:0/1020425263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b380fb540 msgr2=0x7f6b380fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b380fb540 0x7f6b380fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.900+0000 7f6b367fc700 1 -- 172.21.15.12:0/1020425263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20005040 con 0x7f6b38102fd0 2024-03-18T14:59:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b367fc700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b8b0 tx=0x7f6b20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b2000e040 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b20004290 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b380fd420 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b2001c3f0 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.901+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3811a150 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.902+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b38100270 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.902+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6b2000baf0 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.903+0000 7f6b2f7fe700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 0x7f6b24070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.903+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6b2008e9e0 con 0x7f6b38102fd0 2024-03-18T14:59:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.903+0000 7f6b36ffd700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 0x7f6b24070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.903+0000 7f6b36ffd700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 0x7f6b24070390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b30006fd0 tx=0x7f6b30008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:19.907+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b20057fd0 con 0x7f6b38102fd0 2024-03-18T14:59:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.205+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b380fe860 con 0x7f6b38102fd0 2024-03-18T14:59:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.205+0000 7f6b2f7fe700 1 -- 172.21.15.12:0/1020425263 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f6b2005b690 con 0x7f6b38102fd0 2024-03-18T14:59:20.206 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:59:20.206 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:59:20.206 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2, 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:20.207 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:59:20.208 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 6, 2024-03-18T14:59:20.208 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 8 2024-03-18T14:59:20.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:59:20.208 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 msgr2=0x7f6b24070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 0x7f6b24070390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b30006fd0 tx=0x7f6b30008040 comp rx=0 tx=0).stop 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 msgr2=0x7f6b380fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b8b0 tx=0x7f6b20009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 shutdown_connections 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6b2406dee0 0x7f6b24070390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b380fb540 0x7f6b380fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 --2- 172.21.15.12:0/1020425263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b38102fd0 0x7f6b380fcc60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.207+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 >> 172.21.15.12:0/1020425263 conn(0x7f6b38071530 msgr2=0x7f6b380fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.208+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 shutdown_connections 2024-03-18T14:59:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.208+0000 7f6b3d9c7700 1 -- 172.21.15.12:0/1020425263 wait complete. 2024-03-18T14:59:20.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.311+0000 7f3c3e917700 1 -- 172.21.15.12:0/3024155341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 msgr2=0x7f3c380fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.311+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3024155341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c380fe1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c28004940 tx=0x7f3c28009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.311+0000 7f3c3e917700 1 -- 172.21.15.12:0/3024155341 shutdown_connections 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.311+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3024155341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 0x7f3c38106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3024155341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c380fe1f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- 172.21.15.12:0/3024155341 >> 172.21.15.12:0/3024155341 conn(0x7f3c380f7870 msgr2=0x7f3c380f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- 172.21.15.12:0/3024155341 shutdown_connections 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- 172.21.15.12:0/3024155341 wait complete. 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 Processor -- start 2024-03-18T14:59:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- start start 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 0x7f3c38113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381145c0 con 0x7f3c380fde20 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.312+0000 7f3c3e917700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3810dba0 con 0x7f3c380fe7c0 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49004/0 (socket says 172.21.15.12:49004) 2024-03-18T14:59:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 -- 172.21.15.12:0/3301368740 learned_addr learned my addr 172.21.15.12:0/3301368740 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c377fe700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 0x7f3c38113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 -- 172.21.15.12:0/3301368740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 msgr2=0x7f3c38113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 0x7f3c38113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 -- 172.21.15.12:0/3301368740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28005040 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c37fff700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009380 tx=0x7f3c28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2800e040 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c28009af0 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3810de20 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2801c630 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.313+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3810e360 con 0x7f3c380fde20 2024-03-18T14:59:20.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.314+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c38066b70 con 0x7f3c380fde20 2024-03-18T14:59:20.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.315+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3c2800bbc0 con 0x7f3c380fde20 2024-03-18T14:59:20.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.315+0000 7f3c34ff9700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 0x7f3c20070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.315+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f3c2808e8c0 con 0x7f3c380fde20 2024-03-18T14:59:20.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.315+0000 7f3c377fe700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 0x7f3c20070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.316+0000 7f3c377fe700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 0x7f3c20070390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c3810f2d0 tx=0x7f3c2c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:20.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.320+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c2805ba80 con 0x7f3c380fde20 2024-03-18T14:59:20.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:20 smithi012 ceph-mon[111904]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:20 smithi012 ceph-mon[111904]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:20 smithi012 ceph-mon[111904]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:20 smithi100 ceph-mon[84868]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:20 smithi100 ceph-mon[84868]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:20 smithi100 ceph-mon[84868]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.573+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3c3810f080 con 0x7f3c380fde20 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.573+0000 7f3c34ff9700 1 -- 172.21.15.12:0/3301368740 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f3c2805b610 con 0x7f3c380fde20 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:59:20.574 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:59:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:59:20.576 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:20.577 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:20.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:20.578 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:59:20.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:20.578 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:20.578 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 msgr2=0x7f3c20070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 0x7f3c20070390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c3810f2d0 tx=0x7f3c2c00b040 comp rx=0 tx=0).stop 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 msgr2=0x7f3c38113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009380 tx=0x7f3c28009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 shutdown_connections 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3c2006dee0 0x7f3c20070390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c380fde20 0x7f3c38113a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 --2- 172.21.15.12:0/3301368740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c380fe7c0 0x7f3c38113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 >> 172.21.15.12:0/3301368740 conn(0x7f3c380f7870 msgr2=0x7f3c38101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.575+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 shutdown_connections 2024-03-18T14:59:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.576+0000 7f3c3e917700 1 -- 172.21.15.12:0/3301368740 wait complete. 2024-03-18T14:59:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 -- 172.21.15.12:0/381078855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6090101d20 msgr2=0x7f60901020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 --2- 172.21.15.12:0/381078855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6090101d20 0x7f60901020f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6080004940 tx=0x7f6080009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 -- 172.21.15.12:0/381078855 shutdown_connections 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 --2- 172.21.15.12:0/381078855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60901026c0 0x7f60900fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 --2- 172.21.15.12:0/381078855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6090101d20 0x7f60901020f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 -- 172.21.15.12:0/381078855 >> 172.21.15.12:0/381078855 conn(0x7f60900f7870 msgr2=0x7f60900f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 -- 172.21.15.12:0/381078855 shutdown_connections 2024-03-18T14:59:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.679+0000 7f6097935700 1 -- 172.21.15.12:0/381078855 wait complete. 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 Processor -- start 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 -- start start 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6090101d20 0x7f6090102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6090104b60 con 0x7f60901026c0 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6097935700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6090103600 con 0x7f6090101d20 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49022/0 (socket says 172.21.15.12:49022) 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 -- 172.21.15.12:0/3607578230 learned_addr learned my addr 172.21.15.12:0/3607578230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f60956d1700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6090101d20 0x7f6090102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 -- 172.21.15.12:0/3607578230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6090101d20 msgr2=0x7f6090102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6090101d20 0x7f6090102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.680+0000 7f6094ed0700 1 -- 172.21.15.12:0/3607578230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080005040 con 0x7f60901026c0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6094ed0700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b790 tx=0x7f608c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f608c00f6f0 con 0x7f60901026c0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f608c00fd20 con 0x7f60901026c0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60901038e0 con 0x7f60901026c0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f608c0103a0 con 0x7f60901026c0 2024-03-18T14:59:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.681+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609011a200 con 0x7f60901026c0 2024-03-18T14:59:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.682+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60900fd470 con 0x7f60901026c0 2024-03-18T14:59:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.682+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f608c011400 con 0x7f60901026c0 2024-03-18T14:59:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.683+0000 7f6085ffb700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 0x7f607c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.683+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f608c08d430 con 0x7f60901026c0 2024-03-18T14:59:20.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.683+0000 7f60956d1700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 0x7f607c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:20.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.683+0000 7f60956d1700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 0x7f607c070390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6080004f90 tx=0x7f6080009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.687+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f608c056a20 con 0x7f60901026c0 2024-03-18T14:59:20.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.917+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6090108f00 con 0x7f607c06dee0 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.918+0000 7f6085ffb700 1 -- 172.21.15.12:0/3607578230 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6090108f00 con 0x7f607c06dee0 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:59:20.919 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "10/21 daemons upgraded", 2024-03-18T14:59:20.920 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:59:20.920 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:59:20.920 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 msgr2=0x7f607c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 0x7f607c070390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6080004f90 tx=0x7f6080009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 msgr2=0x7f60901030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b790 tx=0x7f608c009220 comp rx=0 tx=0).stop 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 shutdown_connections 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f607c06dee0 0x7f607c070390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6090101d20 0x7f6090102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.921+0000 7f6097935700 1 --2- 172.21.15.12:0/3607578230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60901026c0 0x7f60901030c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.922+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 >> 172.21.15.12:0/3607578230 conn(0x7f60900f7870 msgr2=0x7f60900f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.922+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 shutdown_connections 2024-03-18T14:59:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:20.922+0000 7f6097935700 1 -- 172.21.15.12:0/3607578230 wait complete. 2024-03-18T14:59:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- 172.21.15.12:0/1030559382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 msgr2=0x7f5ac40fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1030559382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac40fbd80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0004940 tx=0x7f5ab0009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- 172.21.15.12:0/1030559382 shutdown_connections 2024-03-18T14:59:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1030559382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac40fbd80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1030559382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac4101d70 0x7f5ac4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- 172.21.15.12:0/1030559382 >> 172.21.15.12:0/1030559382 conn(0x7f5ac4072bc0 msgr2=0x7f5ac4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- 172.21.15.12:0/1030559382 shutdown_connections 2024-03-18T14:59:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- 172.21.15.12:0/1030559382 wait complete. 2024-03-18T14:59:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 Processor -- start 2024-03-18T14:59:21.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.026+0000 7f5acabb6700 1 -- start start 2024-03-18T14:59:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5acabb6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac4101d70 0x7f5ac4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5acabb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:21.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5acabb6700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4114710 con 0x7f5ac4102710 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5acabb6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac410db00 con 0x7f5ac4101d70 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49030/0 (socket says 172.21.15.12:49030) 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 -- 172.21.15.12:0/1309234923 learned_addr learned my addr 172.21.15.12:0/1309234923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 -- 172.21.15.12:0/1309234923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac4101d70 msgr2=0x7f5ac4113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac4101d70 0x7f5ac4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 -- 172.21.15.12:0/1309234923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0005040 con 0x7f5ac4102710 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac3fff700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009100 tx=0x7f5ab0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab000e040 con 0x7f5ac4102710 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.027+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ab0004290 con 0x7f5ac4102710 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.028+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac410dd80 con 0x7f5ac4102710 2024-03-18T14:59:21.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.028+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab001c3f0 con 0x7f5ac4102710 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.028+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac410e340 con 0x7f5ac4102710 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.029+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac40fd4c0 con 0x7f5ac4102710 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.029+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f5ab0013790 con 0x7f5ac4102710 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.029+0000 7f5ac17fa700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 0x7f5ab40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.030+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f5ab008e840 con 0x7f5ac4102710 2024-03-18T14:59:21.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.030+0000 7f5ac8952700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 0x7f5ab40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:21.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.030+0000 7f5ac8952700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 0x7f5ab40703a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8004b70 tx=0x7f5ab8007cc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:21.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.034+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ab005bab0 con 0x7f5ac4102710 2024-03-18T14:59:21.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:21 smithi012 ceph-mon[111904]: pgmap v130: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 638 B/s rd, 1 op/s 2024-03-18T14:59:21.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:21 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1020425263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:21.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:21 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3301368740' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:59:21.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:21 smithi012 ceph-mon[111904]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.328+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5ac4108e50 con 0x7f5ac4102710 2024-03-18T14:59:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.329+0000 7f5ac17fa700 1 -- 172.21.15.12:0/1309234923 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5ab0013420 con 0x7f5ac4102710 2024-03-18T14:59:21.330 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:59:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 msgr2=0x7f5ab40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 0x7f5ab40703a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8004b70 tx=0x7f5ab8007cc0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 msgr2=0x7f5ac41140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009100 tx=0x7f5ab0009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 shutdown_connections 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f5ab406def0 0x7f5ab40703a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ac4101d70 0x7f5ac4113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 --2- 172.21.15.12:0/1309234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4102710 0x7f5ac41140c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 >> 172.21.15.12:0/1309234923 conn(0x7f5ac4072bc0 msgr2=0x7f5ac40659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 shutdown_connections 2024-03-18T14:59:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:21.331+0000 7f5acabb6700 1 -- 172.21.15.12:0/1309234923 wait complete. 2024-03-18T14:59:21.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:21 smithi100 ceph-mon[84868]: pgmap v130: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 638 B/s rd, 1 op/s 2024-03-18T14:59:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:21 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1020425263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:21 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3301368740' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:59:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:21 smithi100 ceph-mon[84868]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:22.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:22 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:22.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:22 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1309234923' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:59:22.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:22 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:22.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:22 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1309234923' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:59:23.479 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:23 smithi100 ceph-mon[84868]: pgmap v131: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:59:23.479 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:23 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:23.479 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:23 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:23.479 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:23 smithi100 ceph-mon[84868]: Upgrade: osd.4 is safe to restart 2024-03-18T14:59:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:23 smithi012 ceph-mon[111904]: pgmap v131: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:59:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:23 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:23 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-18T14:59:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:23 smithi012 ceph-mon[111904]: Upgrade: osd.4 is safe to restart 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: pgmap v132: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: Upgrade: Updating osd.4 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:25.790 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:25 smithi100 ceph-mon[84868]: Deploying daemon osd.4 on smithi100 2024-03-18T14:59:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: pgmap v132: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: Upgrade: Updating osd.4 2024-03-18T14:59:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T14:59:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:25 smithi012 ceph-mon[111904]: Deploying daemon osd.4 on smithi100 2024-03-18T14:59:27.726 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:27 smithi100 ceph-mon[84868]: pgmap v133: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:27.983 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:27 smithi100 systemd[1]: Stopping Ceph osd.4 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:59:27.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:27 smithi012 ceph-mon[111904]: pgmap v133: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:28.294 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[49653]: 2024-03-18T14:59:28.001+0000 7f192101c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T14:59:28.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[49653]: 2024-03-18T14:59:28.001+0000 7f192101c700 -1 osd.4 68 *** Got signal Terminated *** 2024-03-18T14:59:28.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[49653]: 2024-03-18T14:59:28.001+0000 7f192101c700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T14:59:28.912 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:28 smithi100 ceph-mon[84868]: osd.4 marked itself down and dead 2024-03-18T14:59:28.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:28 smithi012 ceph-mon[111904]: osd.4 marked itself down and dead 2024-03-18T14:59:29.286 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 podman[98778]: 2024-03-18 14:59:29.008935159 +0000 UTC m=+1.135728878 container died 2e73a5239141e515f29fa3c580ab406b0f138fc93f573e6e69c56519443015f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, RELEASE=quincy-6716c9f, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T14:59:29.544 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 podman[98778]: 2024-03-18 14:59:29.385059888 +0000 UTC m=+1.511853615 container cleanup 2e73a5239141e515f29fa3c580ab406b0f138fc93f573e6e69c56519443015f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=quincy-6716c9f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-18T14:59:29.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 bash[98778]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4 2024-03-18T14:59:29.941 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:29 smithi100 ceph-mon[84868]: pgmap v134: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:29.942 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:29 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:59:29.942 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:29 smithi100 ceph-mon[84868]: osdmap e69: 6 total, 5 up, 6 in 2024-03-18T14:59:29.942 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 podman[98806]: 2024-03-18 14:59:29.627879456 +0000 UTC m=+0.620738726 container remove 2e73a5239141e515f29fa3c580ab406b0f138fc93f573e6e69c56519443015f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-6716c9f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T14:59:29.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:29 smithi012 ceph-mon[111904]: pgmap v134: 65 pgs: 65 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:29.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:29 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T14:59:29.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:29 smithi012 ceph-mon[111904]: osdmap e69: 6 total, 5 up, 6 in 2024-03-18T14:59:30.206 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 podman[98898]: 2024-03-18 14:59:29.844651389 +0000 UTC m=+0.017611779 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:59:30.206 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:29 smithi100 podman[98898]: 2024-03-18 14:59:29.96209045 +0000 UTC m=+0.135050840 container create a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:59:30.206 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:30 smithi100 podman[98898]: 2024-03-18 14:59:30.131154667 +0000 UTC m=+0.304115058 container init a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-18T14:59:30.206 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:30 smithi100 podman[98898]: 2024-03-18 14:59:30.13441311 +0000 UTC m=+0.307373507 container start a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T14:59:30.206 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:30 smithi100 podman[98898]: 2024-03-18 14:59:30.206032168 +0000 UTC m=+0.378992566 container attach a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T14:59:30.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:30 smithi100 podman[98898]: 2024-03-18 14:59:30.424624007 +0000 UTC m=+0.597584392 container died a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T14:59:30.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:30 smithi012 ceph-mon[111904]: osdmap e70: 6 total, 5 up, 6 in 2024-03-18T14:59:31.009 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:30 smithi100 ceph-mon[84868]: osdmap e70: 6 total, 5 up, 6 in 2024-03-18T14:59:31.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[98924]: 2024-03-18 14:59:31.009879816 +0000 UTC m=+0.580237844 container remove a6715d134d768449c0e538dd4aae4dc4694873f6a16dd76df3db42e4bd4d0f31 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:59:31.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-18T14:59:31.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 systemd[1]: Stopped Ceph osd.4 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:59:31.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.4.service: Consumed 1min 2.419s CPU time. 2024-03-18T14:59:31.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 systemd[1]: Starting Ceph osd.4 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T14:59:31.738 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:31 smithi100 ceph-mon[84868]: pgmap v137: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:59:31.739 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:31 smithi100 ceph-mon[84868]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T14:59:31.739 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:31.739 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:31.739 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[99024]: 2024-03-18 14:59:31.344507087 +0000 UTC m=+0.018007674 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:59:31.739 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[99024]: 2024-03-18 14:59:31.460992205 +0000 UTC m=+0.134492780 container create a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T14:59:31.739 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[99024]: 2024-03-18 14:59:31.663682638 +0000 UTC m=+0.337183223 container init a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:59:31.739 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[99024]: 2024-03-18 14:59:31.666903527 +0000 UTC m=+0.340404110 container start a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T14:59:31.739 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:31 smithi100 podman[99024]: 2024-03-18 14:59:31.73834025 +0000 UTC m=+0.411840842 container attach a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-18T14:59:31.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:31 smithi012 ceph-mon[111904]: pgmap v137: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-18T14:59:31.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:31 smithi012 ceph-mon[111904]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T14:59:31.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:31.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-18T14:59:32.416 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 bash[99024]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate[99039]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-18T14:59:32.417 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 podman[99024]: 2024-03-18 14:59:32.134028986 +0000 UTC m=+0.807529565 container died a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T14:59:32.700 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:32 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 29/216 objects degraded (13.426%), 10 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:32.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:32 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 29/216 objects degraded (13.426%), 10 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:33.044 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 podman[99174]: 2024-03-18 14:59:32.700416011 +0000 UTC m=+0.561501108 container remove a9b32beaa39e71d50caff286f4de279d04f18b6a2acb951a818bed6682b6dd9f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-18T14:59:33.044 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 podman[99241]: 2024-03-18 14:59:32.81321523 +0000 UTC m=+0.017962426 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T14:59:33.045 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:32 smithi100 podman[99241]: 2024-03-18 14:59:32.935048876 +0000 UTC m=+0.139796099 container create 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T14:59:33.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 podman[99241]: 2024-03-18 14:59:33.162512615 +0000 UTC m=+0.367259810 container init 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-18T14:59:33.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 podman[99241]: 2024-03-18 14:59:33.16556954 +0000 UTC m=+0.370316725 container start 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T14:59:33.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-osd[99260]: -- 172.21.15.100:0/150409146 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55611f1df680 con 0x55611f1a9000 2024-03-18T14:59:33.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 bash[99241]: 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 2024-03-18T14:59:33.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 systemd[1]: Started Ceph osd.4 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T14:59:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:33 smithi012 ceph-mon[111904]: pgmap v138: 65 pgs: 10 active+undersized+degraded, 8 peering, 16 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 29/216 objects degraded (13.426%) 2024-03-18T14:59:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:33 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:33 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:33.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:33 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:34.044 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T14:59:33.709+0000 7f94410e97c0 -1 Falling back to public interface 2024-03-18T14:59:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-mon[84868]: pgmap v138: 65 pgs: 10 active+undersized+degraded, 8 peering, 16 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 29/216 objects degraded (13.426%) 2024-03-18T14:59:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:34.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:33 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:34.971 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:34 smithi100 ceph-mon[84868]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-18T14:59:34.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:34 smithi012 ceph-mon[111904]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-18T14:59:35.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:35 smithi012 ceph-mon[111904]: pgmap v139: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:35 smithi100 ceph-mon[84868]: pgmap v139: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:37.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:37 smithi012 ceph-mon[111904]: pgmap v140: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:37.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:37.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:37 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:37.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:37 smithi100 ceph-mon[84868]: pgmap v140: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:37.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:37.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:37 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:38.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:38.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:38 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:38.545 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:38 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T14:59:38.308+0000 7f94410e97c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-18T14:59:38.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:38.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:38 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:39.203 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:38 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T14:59:38.871+0000 7f94410e97c0 -1 osd.4 68 log_to_monitors true 2024-03-18T14:59:39.492 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 14:59:39 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T14:59:39.386+0000 7f9432541700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T14:59:39.492 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:39 smithi100 ceph-mon[84868]: pgmap v141: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 537 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:39.492 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:39 smithi100 ceph-mon[84868]: from='osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:59:39.492 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:39 smithi100 ceph-mon[84868]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:59:39.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:39 smithi012 ceph-mon[111904]: pgmap v141: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 537 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:39.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:39 smithi012 ceph-mon[111904]: from='osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:59:39.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:39 smithi012 ceph-mon[111904]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T14:59:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:40 smithi012 ceph-mon[111904]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T14:59:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:40 smithi012 ceph-mon[111904]: osdmap e71: 6 total, 5 up, 6 in 2024-03-18T14:59:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:40 smithi012 ceph-mon[111904]: from='osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:40.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:40 smithi012 ceph-mon[111904]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:40.745 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:40 smithi100 ceph-mon[84868]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T14:59:40.745 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:40 smithi100 ceph-mon[84868]: osdmap e71: 6 total, 5 up, 6 in 2024-03-18T14:59:40.746 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:40 smithi100 ceph-mon[84868]: from='osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:40.746 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:40 smithi100 ceph-mon[84868]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: pgmap v143: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735] boot 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: osdmap e72: 6 total, 6 up, 6 in 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 35/216 objects degraded (16.204%), 14 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-18T14:59:41.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:41 smithi012 ceph-mon[111904]: osdmap e73: 6 total, 6 up, 6 in 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: pgmap v143: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/216 objects degraded (16.204%) 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: osd.4 [v2:172.21.15.100:6808/2146066735,v1:172.21.15.100:6809/2146066735] boot 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: osdmap e72: 6 total, 6 up, 6 in 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T14:59:41.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 35/216 objects degraded (16.204%), 14 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-18T14:59:41.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:41 smithi100 ceph-mon[84868]: osdmap e73: 6 total, 6 up, 6 in 2024-03-18T14:59:43.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:43 smithi012 ceph-mon[111904]: pgmap v146: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 9/216 objects degraded (4.167%) 2024-03-18T14:59:43.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:43 smithi100 ceph-mon[84868]: pgmap v146: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 9/216 objects degraded (4.167%) 2024-03-18T14:59:45.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:45 smithi012 ceph-mon[111904]: pgmap v147: 65 pgs: 10 peering, 2 active+undersized+degraded, 5 active+undersized, 48 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 3/216 objects degraded (1.389%) 2024-03-18T14:59:45.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:45 smithi100 ceph-mon[84868]: pgmap v147: 65 pgs: 10 peering, 2 active+undersized+degraded, 5 active+undersized, 48 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 3/216 objects degraded (1.389%) 2024-03-18T14:59:46.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:46 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 3/216 objects degraded (1.389%), 2 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:46.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:46.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:46.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:46 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 3/216 objects degraded (1.389%), 2 pgs degraded (PG_DEGRADED) 2024-03-18T14:59:46.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:46.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T14:59:47.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:47 smithi012 ceph-mon[111904]: pgmap v148: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:47.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:47 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/216 objects degraded (1.389%), 2 pgs degraded) 2024-03-18T14:59:47.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:47 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T14:59:47.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:47 smithi100 ceph-mon[84868]: pgmap v148: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:47.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:47 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/216 objects degraded (1.389%), 2 pgs degraded) 2024-03-18T14:59:47.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:47 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T14:59:49.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:49 smithi012 ceph-mon[111904]: pgmap v149: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:59:49.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:49 smithi100 ceph-mon[84868]: pgmap v149: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-18T14:59:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 -- 172.21.15.12:0/1245702760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fde10 msgr2=0x7f6a400fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 --2- 172.21.15.12:0/1245702760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fde10 0x7f6a400fe1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004940 tx=0x7f6a30009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 -- 172.21.15.12:0/1245702760 shutdown_connections 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 --2- 172.21.15.12:0/1245702760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fe7b0 0x7f6a40106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 --2- 172.21.15.12:0/1245702760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fde10 0x7f6a400fe1e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 -- 172.21.15.12:0/1245702760 >> 172.21.15.12:0/1245702760 conn(0x7f6a400f7870 msgr2=0x7f6a400f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 -- 172.21.15.12:0/1245702760 shutdown_connections 2024-03-18T14:59:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.449+0000 7f6a468c8700 1 -- 172.21.15.12:0/1245702760 wait complete. 2024-03-18T14:59:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 Processor -- start 2024-03-18T14:59:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 -- start start 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 0x7f6a40113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40114520 con 0x7f6a400fe7b0 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a468c8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4010db40 con 0x7f6a400fde10 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53322/0 (socket says 172.21.15.12:53322) 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.450+0000 7f6a3f7fe700 1 -- 172.21.15.12:0/133629712 learned_addr learned my addr 172.21.15.12:0/133629712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3ffff700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 0x7f6a40113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3f7fe700 1 -- 172.21.15.12:0/133629712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 msgr2=0x7f6a40113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3f7fe700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 0x7f6a40113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3f7fe700 1 -- 172.21.15.12:0/133629712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30005040 con 0x7f6a400fe7b0 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3ffff700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 0x7f6a40113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3f7fe700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c830 tx=0x7f6a34004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3400a760 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a3400ad90 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4010de20 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a340113a0 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.451+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4010e360 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.452+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40066af0 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.453+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f6a34012400 con 0x7f6a400fe7b0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.453+0000 7f6a3cff9700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 0x7f6a28070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.454+0000 7f6a3ffff700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 0x7f6a28070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.454+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6a3408d700 con 0x7f6a400fe7b0 2024-03-18T14:59:51.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.454+0000 7f6a3ffff700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 0x7f6a28070390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b3b0 tx=0x7f6a30009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:51.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.458+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a34056cf0 con 0x7f6a400fe7b0 2024-03-18T14:59:51.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.689+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a40109ea0 con 0x7f6a2806dee0 2024-03-18T14:59:51.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:51 smithi012 ceph-mon[111904]: pgmap v150: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 840 B/s rd, 1 op/s 2024-03-18T14:59:51.690 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:51.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.690+0000 7f6a3cff9700 1 -- 172.21.15.12:0/133629712 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6a40109ea0 con 0x7f6a2806dee0 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 msgr2=0x7f6a28070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 0x7f6a28070390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b3b0 tx=0x7f6a30009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 msgr2=0x7f6a40113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c830 tx=0x7f6a34004470 comp rx=0 tx=0).stop 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 shutdown_connections 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f6a2806dee0 0x7f6a28070390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a400fde10 0x7f6a40113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 --2- 172.21.15.12:0/133629712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a400fe7b0 0x7f6a40113ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 >> 172.21.15.12:0/133629712 conn(0x7f6a400f7870 msgr2=0x7f6a400f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 shutdown_connections 2024-03-18T14:59:51.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.692+0000 7f6a468c8700 1 -- 172.21.15.12:0/133629712 wait complete. 2024-03-18T14:59:51.703 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T14:59:51.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:51 smithi100 ceph-mon[84868]: pgmap v150: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 840 B/s rd, 1 op/s 2024-03-18T14:59:51.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:51 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 -- 172.21.15.12:0/3362669155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc50102e00 msgr2=0x7fcc501031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 --2- 172.21.15.12:0/3362669155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc50102e00 0x7fcc501031d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc44004940 tx=0x7fcc44009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 -- 172.21.15.12:0/3362669155 shutdown_connections 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 --2- 172.21.15.12:0/3362669155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc501037a0 0x7fcc500fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 --2- 172.21.15.12:0/3362669155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc50102e00 0x7fcc501031d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 -- 172.21.15.12:0/3362669155 >> 172.21.15.12:0/3362669155 conn(0x7fcc500f89b0 msgr2=0x7fcc500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 -- 172.21.15.12:0/3362669155 shutdown_connections 2024-03-18T14:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 -- 172.21.15.12:0/3362669155 wait complete. 2024-03-18T14:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.796+0000 7fcc549d4700 1 Processor -- start 2024-03-18T14:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc549d4700 1 -- start start 2024-03-18T14:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc549d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc50102e00 0x7fcc50113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc549d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc549d4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50114510 con 0x7fcc501037a0 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc549d4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5010db30 con 0x7fcc50102e00 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53336/0 (socket says 172.21.15.12:53336) 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 -- 172.21.15.12:0/2072311625 learned_addr learned my addr 172.21.15.12:0/2072311625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4e59c700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc50102e00 0x7fcc50113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 -- 172.21.15.12:0/2072311625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc50102e00 msgr2=0x7fcc50113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc50102e00 0x7fcc50113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 -- 172.21.15.12:0/2072311625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44005040 con 0x7fcc501037a0 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.797+0000 7fcc4dd9b700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc40002a80 tx=0x7fcc40004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.798+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc40004e80 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.798+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5010de10 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.798+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc4000f040 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.798+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc4000daa0 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.798+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5010e3d0 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.799+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc50100110 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.799+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcc40007cc0 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.800+0000 7fcc3effd700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 0x7fcc380703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.800+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fcc4008cd50 con 0x7fcc501037a0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.800+0000 7fcc4e59c700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 0x7fcc380703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.800+0000 7fcc4e59c700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 0x7fcc380703e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc44004f90 tx=0x7fcc44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:51.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:51.804+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc40056220 con 0x7fcc501037a0 2024-03-18T14:59:52.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.031+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc5010efd0 con 0x7fcc3806df30 2024-03-18T14:59:52.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.032+0000 7fcc3effd700 1 -- 172.21.15.12:0/2072311625 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcc5010efd0 con 0x7fcc3806df30 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 msgr2=0x7fcc380703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 0x7fcc380703e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc44004f90 tx=0x7fcc44009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 msgr2=0x7fcc50113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc40002a80 tx=0x7fcc40004630 comp rx=0 tx=0).stop 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 shutdown_connections 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcc3806df30 0x7fcc380703e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc50102e00 0x7fcc50113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 --2- 172.21.15.12:0/2072311625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc501037a0 0x7fcc50113ec0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 >> 172.21.15.12:0/2072311625 conn(0x7fcc500f89b0 msgr2=0x7fcc501073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 shutdown_connections 2024-03-18T14:59:52.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.034+0000 7fcc549d4700 1 -- 172.21.15.12:0/2072311625 wait complete. 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 -- 172.21.15.12:0/2595297190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41881037a0 msgr2=0x7f41880fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 --2- 172.21.15.12:0/2595297190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41881037a0 0x7f41880fce10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4178004940 tx=0x7f4178009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 -- 172.21.15.12:0/2595297190 shutdown_connections 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 --2- 172.21.15.12:0/2595297190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41881037a0 0x7f41880fce10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 --2- 172.21.15.12:0/2595297190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4188102e00 0x7f41881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 -- 172.21.15.12:0/2595297190 >> 172.21.15.12:0/2595297190 conn(0x7f41880f89b0 msgr2=0x7f41880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 -- 172.21.15.12:0/2595297190 shutdown_connections 2024-03-18T14:59:52.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.137+0000 7f418d8ae700 1 -- 172.21.15.12:0/2595297190 wait complete. 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 Processor -- start 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 -- start start 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41881037a0 0x7f4188113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41881145b0 con 0x7f4188102e00 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f418d8ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418810db40 con 0x7f41881037a0 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f4186ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f4186ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53352/0 (socket says 172.21.15.12:53352) 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.138+0000 7f4186ffd700 1 -- 172.21.15.12:0/1708619858 learned_addr learned my addr 172.21.15.12:0/1708619858 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f4186ffd700 1 -- 172.21.15.12:0/1708619858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41881037a0 msgr2=0x7f4188113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f41867fc700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41881037a0 0x7f4188113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f4186ffd700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41881037a0 0x7f4188113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f4186ffd700 1 -- 172.21.15.12:0/1708619858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178005040 con 0x7f4188102e00 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f4186ffd700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f417000b790 tx=0x7f4170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f417000f770 con 0x7f4188102e00 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f417000fda0 con 0x7f4188102e00 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f417000d310 con 0x7f4188102e00 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418810de20 con 0x7f4188102e00 2024-03-18T14:59:52.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.139+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418810e390 con 0x7f4188102e00 2024-03-18T14:59:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.140+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4188100110 con 0x7f4188102e00 2024-03-18T14:59:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.141+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f4170010420 con 0x7f4188102e00 2024-03-18T14:59:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.141+0000 7f417f7fe700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 0x7f4174070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.141+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f417008d6f0 con 0x7f4188102e00 2024-03-18T14:59:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.141+0000 7f41867fc700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 0x7f4174070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.142+0000 7f41867fc700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 0x7f4174070390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f417800b3b0 tx=0x7f4178009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.146+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4170056ce0 con 0x7f4188102e00 2024-03-18T14:59:52.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.364+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f418810efe0 con 0x7f417406dee0 2024-03-18T14:59:52.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.374+0000 7f417f7fe700 1 -- 172.21.15.12:0/1708619858 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f418810efe0 con 0x7f417406dee0 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (5m) 83s ago 16m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (3m) 83s ago 17m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (3m) 16s ago 15m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (5m) 83s ago 16m 83.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (11m) 83s ago 11m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (11m) 83s ago 11m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (11m) 16s ago 11m 17.4M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (11m) 16s ago 11m 14.2M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (7m) 83s ago 18m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (6m) 16s ago 15m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T14:59:52.375 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (4m) 83s ago 18m 51.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (3m) 16s ago 15m 39.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (6m) 83s ago 16m 15.4M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (6m) 16s ago 15m 15.9M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (2m) 83s ago 14m 103M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (2m) 83s ago 14m 107M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (88s) 83s ago 13m 14.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (52s) 16s ago 13m 109M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (19s) 16s ago 13m 14.8M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (12m) 16s ago 12m 239M 2581M 17.2.7-684-g6716c9f3 19857db3290c a4666e31de07 2024-03-18T14:59:52.376 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (5m) 83s ago 16m 66.3M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 msgr2=0x7f4174070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 0x7f4174070390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f417800b3b0 tx=0x7f4178009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 msgr2=0x7f4188113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f417000b790 tx=0x7f4170009250 comp rx=0 tx=0).stop 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 shutdown_connections 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f417406dee0 0x7f4174070390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4188102e00 0x7f4188113990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 --2- 172.21.15.12:0/1708619858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41881037a0 0x7f4188113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 >> 172.21.15.12:0/1708619858 conn(0x7f41880f89b0 msgr2=0x7f41880fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 shutdown_connections 2024-03-18T14:59:52.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.376+0000 7f418d8ae700 1 -- 172.21.15.12:0/1708619858 wait complete. 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 -- 172.21.15.12:0/3256830367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fb540 msgr2=0x7f3fac0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3256830367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fb540 0x7f3fac0fb990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f94004940 tx=0x7f3f94009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 -- 172.21.15.12:0/3256830367 shutdown_connections 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3256830367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac0fb540 0x7f3fac0fb990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3256830367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac102fd0 0x7f3fac1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 -- 172.21.15.12:0/3256830367 >> 172.21.15.12:0/3256830367 conn(0x7f3fac071530 msgr2=0x7f3fac071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 -- 172.21.15.12:0/3256830367 shutdown_connections 2024-03-18T14:59:52.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.479+0000 7f3fb2835700 1 -- 172.21.15.12:0/3256830367 wait complete. 2024-03-18T14:59:52.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 Processor -- start 2024-03-18T14:59:52.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 -- start start 2024-03-18T14:59:52.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fb540 0x7f3fac113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1144e0 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fb2835700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac10dae0 con 0x7f3fac0fb540 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53374/0 (socket says 172.21.15.12:53374) 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 -- 172.21.15.12:0/3130007838 learned_addr learned my addr 172.21.15.12:0/3130007838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fabfff700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fb540 0x7f3fac113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 -- 172.21.15.12:0/3130007838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fb540 msgr2=0x7f3fac113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fb540 0x7f3fac113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.480+0000 7f3fab7fe700 1 -- 172.21.15.12:0/3130007838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94005040 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fab7fe700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009180 tx=0x7f3f94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9400e040 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac10dd60 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9400bbc0 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9401c630 con 0x7f3fac102fd0 2024-03-18T14:59:52.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.481+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac10e320 con 0x7f3fac102fd0 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.482+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac1053b0 con 0x7f3fac102fd0 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.482+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3f9400bd30 con 0x7f3fac102fd0 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.483+0000 7f3fa8ff9700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 0x7f3f98070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.483+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f3f9408e970 con 0x7f3fac102fd0 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.483+0000 7f3fabfff700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 0x7f3f98070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.483+0000 7f3fabfff700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 0x7f3f98070390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007fd0 tx=0x7f3f9c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.487+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f9405bb30 con 0x7f3fac102fd0 2024-03-18T14:59:52.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:52 smithi012 ceph-mon[111904]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:52.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.783+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3fac10efa0 con 0x7f3fac102fd0 2024-03-18T14:59:52.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.784+0000 7f3fa8ff9700 1 -- 172.21.15.12:0/3130007838 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f3f9405b6c0 con 0x7f3fac102fd0 2024-03-18T14:59:52.784 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 1, 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 4 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T14:59:52.785 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 5, 2024-03-18T14:59:52.786 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 9 2024-03-18T14:59:52.786 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T14:59:52.786 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:59:52.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 msgr2=0x7f3f98070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 0x7f3f98070390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007fd0 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 msgr2=0x7f3fac113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009180 tx=0x7f3f94009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 shutdown_connections 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3f9806dee0 0x7f3f98070390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0fb540 0x7f3fac113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 --2- 172.21.15.12:0/3130007838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fac102fd0 0x7f3fac113e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 >> 172.21.15.12:0/3130007838 conn(0x7f3fac071530 msgr2=0x7f3fac0fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 shutdown_connections 2024-03-18T14:59:52.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.786+0000 7f3fb2835700 1 -- 172.21.15.12:0/3130007838 wait complete. 2024-03-18T14:59:52.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:52 smithi100 ceph-mon[84868]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:52.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 -- 172.21.15.12:0/332991280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 msgr2=0x7fc64c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 --2- 172.21.15.12:0/332991280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c1020f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009ce0 tx=0x7fc63c004d10 comp rx=0 tx=0).stop 2024-03-18T14:59:52.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 -- 172.21.15.12:0/332991280 shutdown_connections 2024-03-18T14:59:52.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 --2- 172.21.15.12:0/332991280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 --2- 172.21.15.12:0/332991280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c1020f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 -- 172.21.15.12:0/332991280 >> 172.21.15.12:0/332991280 conn(0x7fc64c0f7870 msgr2=0x7fc64c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 -- 172.21.15.12:0/332991280 shutdown_connections 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.889+0000 7fc651eb7700 1 -- 172.21.15.12:0/332991280 wait complete. 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 Processor -- start 2024-03-18T14:59:52.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 -- start start 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c104b20 con 0x7fc64c101d20 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc651eb7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c103600 con 0x7fc64c1026c0 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc64b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc64b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53386/0 (socket says 172.21.15.12:53386) 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc64b7fe700 1 -- 172.21.15.12:0/3625173152 learned_addr learned my addr 172.21.15.12:0/3625173152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.890+0000 7fc64affd700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc64b7fe700 1 -- 172.21.15.12:0/3625173152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 msgr2=0x7fc64c1030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc64b7fe700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c1030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc64b7fe700 1 -- 172.21.15.12:0/3625173152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c009940 con 0x7fc64c101d20 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc64affd700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:59:52.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc64b7fe700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc63c005010 tx=0x7fc63c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc63c00e040 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc63c008e20 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c103850 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc63c01c660 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.891+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c11a260 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.892+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c0fd470 con 0x7fc64c101d20 2024-03-18T14:59:52.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.892+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fc63c012a70 con 0x7fc64c101d20 2024-03-18T14:59:52.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.893+0000 7fc643fff700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 0x7fc638070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:52.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.893+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fc63c08f260 con 0x7fc64c101d20 2024-03-18T14:59:52.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.893+0000 7fc64affd700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 0x7fc638070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:52.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.893+0000 7fc64affd700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 0x7fc638070390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc634007fd0 tx=0x7fc634009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:52.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:52.897+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc63c0587d0 con 0x7fc64c101d20 2024-03-18T14:59:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.151+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc64c104090 con 0x7fc64c101d20 2024-03-18T14:59:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.152+0000 7fc643fff700 1 -- 172.21.15.12:0/3625173152 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fc63c05be90 con 0x7fc64c101d20 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:e15 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 15 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T14:48:53.920975+0000 2024-03-18T14:59:53.153 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14476} 2024-03-18T14:59:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:14476} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.12:6826/1453814985,v1:172.21.15.12:6827/1453814985] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:14488} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.12:6828/1193273652,v1:172.21.15.12:6829/1193273652] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 msgr2=0x7fc638070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 0x7fc638070390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc634007fd0 tx=0x7fc634009040 comp rx=0 tx=0).stop 2024-03-18T14:59:53.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 msgr2=0x7fc64c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc63c005010 tx=0x7fc63c004d10 comp rx=0 tx=0).stop 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 shutdown_connections 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fc63806dee0 0x7fc638070390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc64c101d20 0x7fc64c102b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 --2- 172.21.15.12:0/3625173152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc64c1026c0 0x7fc64c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 >> 172.21.15.12:0/3625173152 conn(0x7fc64c0f7870 msgr2=0x7fc64c066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 shutdown_connections 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.154+0000 7fc651eb7700 1 -- 172.21.15.12:0/3625173152 wait complete. 2024-03-18T14:59:53.157 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 15 2024-03-18T14:59:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 -- 172.21.15.12:0/2475040151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce0102e00 msgr2=0x7f2ce01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/2475040151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce0102e00 0x7f2ce01031d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0004940 tx=0x7f2cd0009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 -- 172.21.15.12:0/2475040151 shutdown_connections 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/2475040151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce01037a0 0x7f2ce00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/2475040151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce0102e00 0x7f2ce01031d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 -- 172.21.15.12:0/2475040151 >> 172.21.15.12:0/2475040151 conn(0x7f2ce00f89b0 msgr2=0x7f2ce00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 -- 172.21.15.12:0/2475040151 shutdown_connections 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.258+0000 7f2ce57b0700 1 -- 172.21.15.12:0/2475040151 wait complete. 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 Processor -- start 2024-03-18T14:59:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 -- start start 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 0x7f2ce0113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0114550 con 0x7f2ce01037a0 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2ce57b0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce010dac0 con 0x7f2ce0102e00 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2cde7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2cde7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53402/0 (socket says 172.21.15.12:53402) 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2cde7fc700 1 -- 172.21.15.12:0/3023482732 learned_addr learned my addr 172.21.15.12:0/3023482732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.259+0000 7f2cdeffd700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 0x7f2ce0113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cde7fc700 1 -- 172.21.15.12:0/3023482732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 msgr2=0x7f2ce0113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cde7fc700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 0x7f2ce0113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cde7fc700 1 -- 172.21.15.12:0/3023482732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0005040 con 0x7f2ce01037a0 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cdeffd700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 0x7f2ce0113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T14:59:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cde7fc700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b790 tx=0x7f2cd4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd400f6f0 con 0x7f2ce01037a0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cd400fd20 con 0x7f2ce01037a0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce010dda0 con 0x7f2ce01037a0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd40103a0 con 0x7f2ce01037a0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.260+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce010e360 con 0x7f2ce01037a0 2024-03-18T14:59:53.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.261+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00fe550 con 0x7f2ce01037a0 2024-03-18T14:59:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.262+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2cd4011400 con 0x7f2ce01037a0 2024-03-18T14:59:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.262+0000 7f2cc77fe700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 0x7f2cc8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.262+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f2cd408d480 con 0x7f2ce01037a0 2024-03-18T14:59:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.262+0000 7f2cdeffd700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 0x7f2cc8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.262+0000 7f2cdeffd700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 0x7f2cc8070390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0004f90 tx=0x7f2cd0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:53.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.267+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cd405a5a0 con 0x7f2ce01037a0 2024-03-18T14:59:53.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.496+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ce0113600 con 0x7f2cc806dee0 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.497+0000 7f2cc77fe700 1 -- 172.21.15.12:0/3023482732 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2ce0113600 con 0x7f2cc806dee0 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T14:59:53.498 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T14:59:53.499 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "11/21 daemons upgraded", 2024-03-18T14:59:53.499 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading osd daemons", 2024-03-18T14:59:53.499 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T14:59:53.499 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 msgr2=0x7f2cc8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 0x7f2cc8070390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0004f90 tx=0x7f2cd0009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 msgr2=0x7f2ce0113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b790 tx=0x7f2cd4009220 comp rx=0 tx=0).stop 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 shutdown_connections 2024-03-18T14:59:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2cc806dee0 0x7f2cc8070390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102e00 0x7f2ce0113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 --2- 172.21.15.12:0/3023482732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ce01037a0 0x7f2ce0113e70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.499+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 >> 172.21.15.12:0/3023482732 conn(0x7f2ce00f89b0 msgr2=0x7f2ce00f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.500+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 shutdown_connections 2024-03-18T14:59:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.500+0000 7f2ce57b0700 1 -- 172.21.15.12:0/3023482732 wait complete. 2024-03-18T14:59:53.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 -- 172.21.15.12:0/617708201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 msgr2=0x7f358c0fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 --2- 172.21.15.12:0/617708201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c0fb890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3580004940 tx=0x7f3580009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 -- 172.21.15.12:0/617708201 shutdown_connections 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 --2- 172.21.15.12:0/617708201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c0fb890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 --2- 172.21.15.12:0/617708201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 0x7f358c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 -- 172.21.15.12:0/617708201 >> 172.21.15.12:0/617708201 conn(0x7f358c072380 msgr2=0x7f358c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 -- 172.21.15.12:0/617708201 shutdown_connections 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 -- 172.21.15.12:0/617708201 wait complete. 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.607+0000 7f3591adc700 1 Processor -- start 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3591adc700 1 -- start start 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3591adc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3591adc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 0x7f358c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3591adc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1145b0 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3591adc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10db40 con 0x7f358c102ea0 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53430/0 (socket says 172.21.15.12:53430) 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 -- 172.21.15.12:0/3967769954 learned_addr learned my addr 172.21.15.12:0/3967769954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358affd700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 0x7f358c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 -- 172.21.15.12:0/3967769954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 msgr2=0x7f358c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 0x7f358c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 -- 172.21.15.12:0/3967769954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580005040 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f358b7fe700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b790 tx=0x7f357c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.608+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357c00f6f0 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.609+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357c00fd20 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.609+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c10de20 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.609+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357c0103a0 con 0x7f358c0fb420 2024-03-18T14:59:53.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.609+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c10e360 con 0x7f358c0fb420 2024-03-18T14:59:53.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.609+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c100150 con 0x7f358c0fb420 2024-03-18T14:59:53.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.610+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f357c011400 con 0x7f358c0fb420 2024-03-18T14:59:53.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.611+0000 7f3573fff700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 0x7f3574070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T14:59:53.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.611+0000 7f358affd700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 0x7f3574070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T14:59:53.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.611+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f357c08d470 con 0x7f358c0fb420 2024-03-18T14:59:53.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.611+0000 7f358affd700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 0x7f3574070390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3580004f90 tx=0x7f3580009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T14:59:53.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.615+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f357c056a60 con 0x7f358c0fb420 2024-03-18T14:59:53.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:53 smithi012 ceph-mon[111904]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:53.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:53 smithi012 ceph-mon[111904]: pgmap v151: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 952 B/s rd, 1 op/s 2024-03-18T14:59:53.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:53 smithi012 ceph-mon[111904]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:53.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:53 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3130007838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:53.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:53 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3625173152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:59:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:53 smithi100 ceph-mon[84868]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:53 smithi100 ceph-mon[84868]: pgmap v151: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 952 B/s rd, 1 op/s 2024-03-18T14:59:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:53 smithi100 ceph-mon[84868]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:53 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3130007838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T14:59:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:53 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3625173152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T14:59:53.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.907+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f358c0fe740 con 0x7f358c0fb420 2024-03-18T14:59:53.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.908+0000 7f3573fff700 1 -- 172.21.15.12:0/3967769954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f357c05a120 con 0x7f358c0fb420 2024-03-18T14:59:53.909 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T14:59:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 msgr2=0x7f3574070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 0x7f3574070390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3580004f90 tx=0x7f3580009e30 comp rx=0 tx=0).stop 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 msgr2=0x7f358c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b790 tx=0x7f357c009220 comp rx=0 tx=0).stop 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 shutdown_connections 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f357406dee0 0x7f3574070390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f358c0fb420 0x7f358c113990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 --2- 172.21.15.12:0/3967769954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f358c102ea0 0x7f358c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 >> 172.21.15.12:0/3967769954 conn(0x7f358c072380 msgr2=0x7f358c0fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 shutdown_connections 2024-03-18T14:59:53.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T14:59:53.910+0000 7f3591adc700 1 -- 172.21.15.12:0/3967769954 wait complete. 2024-03-18T14:59:54.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:54 smithi012 ceph-mon[111904]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:54.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:54 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3967769954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:59:54.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:54 smithi100 ceph-mon[84868]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T14:59:54.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:54 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3967769954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T14:59:55.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:55 smithi012 ceph-mon[111904]: pgmap v152: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:55.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:55 smithi100 ceph-mon[84868]: pgmap v152: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T14:59:57.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:56 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:57.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:56 smithi012 ceph-mon[111904]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:57.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:56 smithi012 ceph-mon[111904]: Upgrade: osd.5 is safe to restart 2024-03-18T14:59:57.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:56 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:57.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:56 smithi100 ceph-mon[84868]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-18T14:59:57.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:56 smithi100 ceph-mon[84868]: Upgrade: osd.5 is safe to restart 2024-03-18T14:59:58.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:57 smithi100 ceph-mon[84868]: pgmap v153: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:58.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:57 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:58.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:57 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:59:58.238 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:57 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:58.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:57 smithi012 ceph-mon[111904]: pgmap v153: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T14:59:58.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T14:59:58.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T14:59:58.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:57 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T14:59:58.967 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:58 smithi100 ceph-mon[84868]: Upgrade: Updating osd.5 2024-03-18T14:59:58.967 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:58 smithi100 ceph-mon[84868]: Deploying daemon osd.5 on smithi100 2024-03-18T14:59:59.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:58 smithi012 ceph-mon[111904]: Upgrade: Updating osd.5 2024-03-18T14:59:59.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:58 smithi012 ceph-mon[111904]: Deploying daemon osd.5 on smithi100 2024-03-18T15:00:00.182 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 14:59:59 smithi100 ceph-mon[84868]: pgmap v154: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:00:00.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 14:59:59 smithi012 ceph-mon[111904]: pgmap v154: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:00:00.820 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:00 smithi100 systemd[1]: Stopping Ceph osd.5 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:00:01.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:00 smithi012 ceph-mon[111904]: overall HEALTH_OK 2024-03-18T15:00:01.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:00 smithi012 ceph-mon[111904]: osd.5 marked itself down and dead 2024-03-18T15:00:01.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:00 smithi100 ceph-mon[84868]: overall HEALTH_OK 2024-03-18T15:00:01.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:00 smithi100 ceph-mon[84868]: osd.5 marked itself down and dead 2024-03-18T15:00:01.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T15:00:00.819+0000 7efe9f626700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:00:01.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T15:00:00.819+0000 7efe9f626700 -1 osd.5 73 *** Got signal Terminated *** 2024-03-18T15:00:01.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:00 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[56393]: 2024-03-18T15:00:00.819+0000 7efe9f626700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:00:01.770 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:01 smithi100 podman[104067]: 2024-03-18 15:00:01.493449047 +0000 UTC m=+0.786000594 container died a4666e31de07870ba60a5431845787513d1171e7d17eaa179a66507afce95e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=quincy-6716c9f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:00:02.044 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:01 smithi100 podman[104067]: 2024-03-18 15:00:01.869416231 +0000 UTC m=+1.161967764 container cleanup a4666e31de07870ba60a5431845787513d1171e7d17eaa179a66507afce95e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, RELEASE=quincy-6716c9f, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-03-18T15:00:02.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:01 smithi100 bash[104067]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: pgmap v155: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: osdmap e74: 6 total, 5 up, 6 in 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:02.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:02.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: pgmap v155: 65 pgs: 65 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:00:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-18T15:00:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-18T15:00:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: osdmap e74: 6 total, 5 up, 6 in 2024-03-18T15:00:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:02.488 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104084]: 2024-03-18 15:00:02.154449833 +0000 UTC m=+0.661062516 container remove a4666e31de07870ba60a5431845787513d1171e7d17eaa179a66507afce95e78 (image=quay.ceph.io/ceph-ci/ceph@sha256:a75c953aa7dcb8597f235a84417a8be589209dba39db0305cebbb10d212afaf9, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-6716c9f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:00:02.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104178]: 2024-03-18 15:00:02.488567942 +0000 UTC m=+0.107451095 container create 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:00:02.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104178]: 2024-03-18 15:00:02.399730706 +0000 UTC m=+0.018613860 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:00:02.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104178]: 2024-03-18 15:00:02.716645028 +0000 UTC m=+0.335528193 container init 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-18T15:00:02.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104178]: 2024-03-18 15:00:02.719911216 +0000 UTC m=+0.338794379 container start 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-18T15:00:03.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:02 smithi012 ceph-mon[111904]: osdmap e75: 6 total, 5 up, 6 in 2024-03-18T15:00:03.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:02 smithi100 ceph-mon[84868]: osdmap e75: 6 total, 5 up, 6 in 2024-03-18T15:00:03.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:02 smithi100 podman[104178]: 2024-03-18 15:00:02.799359711 +0000 UTC m=+0.418242866 container attach 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-18T15:00:03.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 podman[104178]: 2024-03-18 15:00:03.012719287 +0000 UTC m=+0.631602442 container died 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:00:03.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 podman[104205]: 2024-03-18 15:00:03.594686803 +0000 UTC m=+0.572278224 container remove 25b995135126c685695e2c0e4bad546c6a29f6bf79d0040ebc4fab1fe7049bff (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-18T15:00:03.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-18T15:00:03.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 systemd[1]: Stopped Ceph osd.5 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T15:00:03.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.5.service: Consumed 1min 3.810s CPU time. 2024-03-18T15:00:04.168 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:03 smithi100 ceph-mon[84868]: pgmap v158: 65 pgs: 13 peering, 8 stale+active+clean, 44 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-18T15:00:04.168 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:03 smithi100 ceph-mon[84868]: Health check failed: Reduced data availability: 5 pgs peering (PG_AVAILABILITY) 2024-03-18T15:00:04.168 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:03 smithi100 systemd[1]: Starting Ceph osd.5 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:00:04.168 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:03.935315083 +0000 UTC m=+0.017625119 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:00:04.168 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:04.054122228 +0000 UTC m=+0.136432246 container create 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T15:00:04.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:03 smithi012 ceph-mon[111904]: pgmap v158: 65 pgs: 13 peering, 8 stale+active+clean, 44 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-18T15:00:04.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:03 smithi012 ceph-mon[111904]: Health check failed: Reduced data availability: 5 pgs peering (PG_AVAILABILITY) 2024-03-18T15:00:04.544 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:04.248346731 +0000 UTC m=+0.330656755 container init 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True) 2024-03-18T15:00:04.545 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:04.251435099 +0000 UTC m=+0.333745119 container start 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:00:04.545 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:04.314937985 +0000 UTC m=+0.397248017 container attach 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T15:00:05.010 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-18T15:00:05.010 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-18T15:00:05.010 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-18T15:00:05.010 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-18T15:00:05.010 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 bash[104306]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate[104321]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-18T15:00:05.011 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:04 smithi100 podman[104306]: 2024-03-18 15:00:04.717636172 +0000 UTC m=+0.799946183 container died 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-18T15:00:05.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:05 smithi100 ceph-mon[84868]: Health check failed: Degraded data redundancy: 2/216 objects degraded (0.926%), 2 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:05.496 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:05 smithi012 ceph-mon[111904]: Health check failed: Degraded data redundancy: 2/216 objects degraded (0.926%), 2 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:05.616 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 podman[104440]: 2024-03-18 15:00:05.310109494 +0000 UTC m=+0.587346340 container remove 5f2aa282c1c90ddf6ccc2f7a6362094f3750f835acb022527da54109041c8524 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-18T15:00:05.617 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 podman[104501]: 2024-03-18 15:00:05.519304097 +0000 UTC m=+0.112758933 container create 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T15:00:05.617 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 podman[104501]: 2024-03-18 15:00:05.42502659 +0000 UTC m=+0.018481428 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:00:06.014 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 podman[104501]: 2024-03-18 15:00:05.696505225 +0000 UTC m=+0.289960076 container init 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-18T15:00:06.014 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 podman[104501]: 2024-03-18 15:00:05.6994457 +0000 UTC m=+0.292900545 container start 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:00:06.014 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 ceph-osd[104521]: -- 172.21.15.100:0/4045881473 <== mon.1 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c81a0f680 con 0x562c819d9000 2024-03-18T15:00:06.014 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 bash[104501]: 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 2024-03-18T15:00:06.014 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:05 smithi100 systemd[1]: Started Ceph osd.5 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T15:00:06.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:06 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:00:06.244+0000 7f808c43b7c0 -1 Falling back to public interface 2024-03-18T15:00:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:06 smithi100 ceph-mon[84868]: pgmap v159: 65 pgs: 2 active+undersized+degraded, 13 peering, 7 stale+active+clean, 1 active+undersized, 42 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 2/216 objects degraded (0.926%) 2024-03-18T15:00:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:06 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:06 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:06.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:06 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:06.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:06 smithi012 ceph-mon[111904]: pgmap v159: 65 pgs: 2 active+undersized+degraded, 13 peering, 7 stale+active+clean, 1 active+undersized, 42 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 2/216 objects degraded (0.926%) 2024-03-18T15:00:06.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:06 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:06.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:06 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:06.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:06 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:08.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:08 smithi100 ceph-mon[84868]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:08.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:08 smithi012 ceph-mon[111904]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:10.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:09 smithi100 ceph-mon[84868]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:10.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:09 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:10.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:09 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:10.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:09 smithi100 ceph-mon[84868]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-18T15:00:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:09 smithi012 ceph-mon[111904]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:09 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:09 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:10.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:09 smithi012 ceph-mon[111904]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-18T15:00:10.794 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:10 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:00:10.717+0000 7f808c43b7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-18T15:00:11.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:10 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:11.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:10 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:11.268 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:10 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:11.268 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:10 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:11.544 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:11 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:00:11.267+0000 7f808c43b7c0 -1 osd.5 73 log_to_monitors true 2024-03-18T15:00:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:11 smithi012 ceph-mon[111904]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 669 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:11 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 30/216 objects degraded (13.889%), 12 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:11 smithi012 ceph-mon[111904]: from='osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:00:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:11 smithi012 ceph-mon[111904]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:00:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:11 smithi100 ceph-mon[84868]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 669 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:11 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 30/216 objects degraded (13.889%), 12 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:11 smithi100 ceph-mon[84868]: from='osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:00:12.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:11 smithi100 ceph-mon[84868]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:00:13.145 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:00:12.847+0000 7f807988b700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:00:13.146 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-mon[84868]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T15:00:13.146 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-mon[84868]: osdmap e76: 6 total, 5 up, 6 in 2024-03-18T15:00:13.146 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-mon[84868]: from='osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T15:00:13.146 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-mon[84868]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T15:00:13.146 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:12 smithi100 ceph-mon[84868]: from='osd.5 ' entity='osd.5' 2024-03-18T15:00:13.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:12 smithi012 ceph-mon[111904]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T15:00:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:12 smithi012 ceph-mon[111904]: osdmap e76: 6 total, 5 up, 6 in 2024-03-18T15:00:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:12 smithi012 ceph-mon[111904]: from='osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T15:00:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:12 smithi012 ceph-mon[111904]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-18T15:00:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:12 smithi012 ceph-mon[111904]: from='osd.5 ' entity='osd.5' 2024-03-18T15:00:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: pgmap v164: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:14.266 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-18T15:00:14.267 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408] boot 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: osdmap e77: 6 total, 6 up, 6 in 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.268 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-18T15:00:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: pgmap v164: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-18T15:00:14.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: osd.5 [v2:172.21.15.100:6816/1888487408,v1:172.21.15.100:6817/1888487408] boot 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: osdmap e77: 6 total, 6 up, 6 in 2024-03-18T15:00:14.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:00:14.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:14.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:13 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all osd 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: Upgrade: Setting require_osd_release to 19 squid 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: osdmap e78: 6 total, 6 up, 6 in 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:15.199 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:15.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all osd 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: Upgrade: Setting require_osd_release to 19 squid 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: osdmap e78: 6 total, 6 up, 6 in 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:15.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:14 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: pgmap v167: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6828/3243464363,v1:172.21.15.12:6829/3243464363] up:boot 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:00:16.211 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:15 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: pgmap v167: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/216 objects degraded (13.889%) 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6828/3243464363,v1:172.21.15.12:6829/3243464363] up:boot 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:00:16.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:15 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:17.395 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: pgmap v168: 65 pgs: 5 active+undersized+degraded, 11 active+undersized, 49 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 10/216 objects degraded (4.630%) 2024-03-18T15:00:17.395 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: Upgrade: Updating mds.cephfs.smithi012.xtadxo 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: Deploying daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T15:00:17.396 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:17 smithi100 ceph-mon[84868]: Health check update: Degraded data redundancy: 10/216 objects degraded (4.630%), 5 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:17.415 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: pgmap v168: 65 pgs: 5 active+undersized+degraded, 11 active+undersized, 49 active+clean; 193 MiB data, 812 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 10/216 objects degraded (4.630%) 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: Upgrade: Updating mds.cephfs.smithi012.xtadxo 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.xtadxo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: Deploying daemon mds.cephfs.smithi012.xtadxo on smithi012 2024-03-18T15:00:17.416 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:17 smithi012 ceph-mon[111904]: Health check update: Degraded data redundancy: 10/216 objects degraded (4.630%), 5 pgs degraded (PG_DEGRADED) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[111900]: 2024-03-18T15:00:19.126+0000 7fe1435f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: pgmap v169: 65 pgs: 1 active+undersized+degraded, 1 active+undersized, 63 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 164 B/s rd, 0 op/s; 1/216 objects degraded (0.463%) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: osdmap e79: 6 total, 6 up, 6 in 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: Standby daemon mds.cephfs.smithi100.dvtbrf assigned to filesystem cephfs as rank 0 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:00:19.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:19 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:replay} 2 up:standby 2024-03-18T15:00:19.544 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:00:19.152+0000 7fb5c5f92700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-03-18T15:00:19.544 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:00:19.153+0000 7fb5c2f8c700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: pgmap v169: 65 pgs: 1 active+undersized+degraded, 1 active+undersized, 63 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 164 B/s rd, 0 op/s; 1/216 objects degraded (0.463%) 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: osdmap e79: 6 total, 6 up, 6 in 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: Standby daemon mds.cephfs.smithi100.dvtbrf assigned to filesystem cephfs as rank 0 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:00:19.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:19 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:replay} 2 up:standby 2024-03-18T15:00:20.411 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:20 smithi012 ceph-mon[111904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/216 objects degraded (0.463%), 1 pg degraded) 2024-03-18T15:00:20.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:20 smithi100 ceph-mon[84868]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/216 objects degraded (0.463%), 1 pg degraded) 2024-03-18T15:00:21.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:21 smithi012 ceph-mon[111904]: pgmap v171: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-03-18T15:00:21.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:21 smithi100 ceph-mon[84868]: pgmap v171: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-03-18T15:00:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:23 smithi012 ceph-mon[111904]: pgmap v172: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 4 op/s 2024-03-18T15:00:23.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:23 smithi100 ceph-mon[84868]: pgmap v172: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 4 op/s 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- 172.21.15.12:0/419223242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 msgr2=0x7f38a00fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/419223242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a00fbd30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f388c004940 tx=0x7f388c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- 172.21.15.12:0/419223242 shutdown_connections 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/419223242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a00fbd30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/419223242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- 172.21.15.12:0/419223242 >> 172.21.15.12:0/419223242 conn(0x7f38a00f7870 msgr2=0x7f38a00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- 172.21.15.12:0/419223242 shutdown_connections 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- 172.21.15.12:0/419223242 wait complete. 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 Processor -- start 2024-03-18T15:00:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.029+0000 7f38a6b4f700 1 -- start start 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a6b4f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a0113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a6b4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a6b4f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01144d0 con 0x7f38a01026c0 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a6b4f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a010dad0 con 0x7f38a0101d20 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f389ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a48eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a0113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f389ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56646/0 (socket says 172.21.15.12:56646) 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f389ffff700 1 -- 172.21.15.12:0/368596021 learned_addr learned my addr 172.21.15.12:0/368596021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a48eb700 1 -- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 msgr2=0x7f38a0113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a48eb700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f38a48eb700 1 -- 172.21.15.12:0/368596021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c005040 con 0x7f38a0101d20 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.030+0000 7f389ffff700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f38a48eb700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a0113940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3894009fd0 tx=0x7f389400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3894004e90 con 0x7f38a0101d20 2024-03-18T15:00:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f389400f040 con 0x7f38a0101d20 2024-03-18T15:00:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389400ca50 con 0x7f38a0101d20 2024-03-18T15:00:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a010dd50 con 0x7f38a0101d20 2024-03-18T15:00:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.031+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a010e290 con 0x7f38a0101d20 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.032+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a00fd470 con 0x7f38a0101d20 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.032+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3894007cc0 con 0x7f38a0101d20 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.033+0000 7f389d7fa700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 0x7f3890070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.033+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f389408d0b0 con 0x7f38a0101d20 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.033+0000 7f389ffff700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 0x7f3890070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.033+0000 7f389ffff700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 0x7f3890070390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38a010ec90 tx=0x7f388c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.038+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f389405a180 con 0x7f38a0101d20 2024-03-18T15:00:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.263+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38a0108e70 con 0x7f389006dee0 2024-03-18T15:00:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.264+0000 7f389d7fa700 1 -- 172.21.15.12:0/368596021 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f38a0108e70 con 0x7f389006dee0 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 msgr2=0x7f3890070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 0x7f3890070390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38a010ec90 tx=0x7f388c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 msgr2=0x7f38a0113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a0113940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3894009fd0 tx=0x7f389400b040 comp rx=0 tx=0).stop 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 shutdown_connections 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389006dee0 0x7f3890070390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a0101d20 0x7f38a0113940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 --2- 172.21.15.12:0/368596021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38a01026c0 0x7f38a0113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 >> 172.21.15.12:0/368596021 conn(0x7f38a00f7870 msgr2=0x7f38a00f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 shutdown_connections 2024-03-18T15:00:24.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.266+0000 7f38a6b4f700 1 -- 172.21.15.12:0/368596021 wait complete. 2024-03-18T15:00:24.277 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T15:00:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 -- 172.21.15.12:0/3744165354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 msgr2=0x7f52d0106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 --2- 172.21.15.12:0/3744165354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0106ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52c0004940 tx=0x7f52c0009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 -- 172.21.15.12:0/3744165354 shutdown_connections 2024-03-18T15:00:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 --2- 172.21.15.12:0/3744165354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0106ed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 --2- 172.21.15.12:0/3744165354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 0x7f52d00fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.370+0000 7f52d74f2700 1 -- 172.21.15.12:0/3744165354 >> 172.21.15.12:0/3744165354 conn(0x7f52d00f78c0 msgr2=0x7f52d00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 -- 172.21.15.12:0/3744165354 shutdown_connections 2024-03-18T15:00:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 -- 172.21.15.12:0/3744165354 wait complete. 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 Processor -- start 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 -- start start 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 0x7f52d01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d0114530 con 0x7f52d00fe800 2024-03-18T15:00:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d74f2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d010db30 con 0x7f52d00fde60 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d4a8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d528e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 0x7f52d01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d4a8d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56654/0 (socket says 172.21.15.12:56654) 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.371+0000 7f52d4a8d700 1 -- 172.21.15.12:0/1322437753 learned_addr learned my addr 172.21.15.12:0/1322437753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d4a8d700 1 -- 172.21.15.12:0/1322437753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 msgr2=0x7f52d01139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d4a8d700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 0x7f52d01139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d4a8d700 1 -- 172.21.15.12:0/1322437753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0005040 con 0x7f52d00fe800 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d4a8d700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b3b0 tx=0x7f52c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c000e040 con 0x7f52d00fe800 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52c0013420 con 0x7f52d00fe800 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d010ddb0 con 0x7f52d00fe800 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c001c5f0 con 0x7f52d00fe800 2024-03-18T15:00:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.372+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d010e2f0 con 0x7f52d00fe800 2024-03-18T15:00:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.373+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d0066ad0 con 0x7f52d00fe800 2024-03-18T15:00:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.374+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f52c0013590 con 0x7f52d00fe800 2024-03-18T15:00:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.375+0000 7f52c5ffb700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 0x7f52bc070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.375+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f52c008e7a0 con 0x7f52d00fe800 2024-03-18T15:00:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.375+0000 7f52d528e700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 0x7f52bc070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.375+0000 7f52d528e700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 0x7f52bc070390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004f40 tx=0x7f52cc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.380+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52c005b900 con 0x7f52d00fe800 2024-03-18T15:00:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.608+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d0109ee0 con 0x7f52bc06dee0 2024-03-18T15:00:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.610+0000 7f52c5ffb700 1 -- 172.21.15.12:0/1322437753 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f52d0109ee0 con 0x7f52bc06dee0 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 msgr2=0x7f52bc070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 0x7f52bc070390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004f40 tx=0x7f52cc00b040 comp rx=0 tx=0).stop 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 msgr2=0x7f52d0113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b3b0 tx=0x7f52c0009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 shutdown_connections 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f52bc06dee0 0x7f52bc070390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52d00fde60 0x7f52d01139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 --2- 172.21.15.12:0/1322437753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52d00fe800 0x7f52d0113ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 >> 172.21.15.12:0/1322437753 conn(0x7f52d00f78c0 msgr2=0x7f52d01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 shutdown_connections 2024-03-18T15:00:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.612+0000 7f52d74f2700 1 -- 172.21.15.12:0/1322437753 wait complete. 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 -- 172.21.15.12:0/1363295333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7901037a0 msgr2=0x7fe7900fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 --2- 172.21.15.12:0/1363295333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7901037a0 0x7fe7900fce10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe78c004940 tx=0x7fe78c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 -- 172.21.15.12:0/1363295333 shutdown_connections 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 --2- 172.21.15.12:0/1363295333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7901037a0 0x7fe7900fce10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 --2- 172.21.15.12:0/1363295333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe790102e00 0x7fe7901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 -- 172.21.15.12:0/1363295333 >> 172.21.15.12:0/1363295333 conn(0x7fe7900f89b0 msgr2=0x7fe7900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 -- 172.21.15.12:0/1363295333 shutdown_connections 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.714+0000 7fe798164700 1 -- 172.21.15.12:0/1363295333 wait complete. 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 Processor -- start 2024-03-18T15:00:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 -- start start 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7901037a0 0x7fe790070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe790072210 con 0x7fe790102e00 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe798164700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe790070d40 con 0x7fe7901037a0 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe795f00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe7956ff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7901037a0 0x7fe790070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe795f00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56674/0 (socket says 172.21.15.12:56674) 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.715+0000 7fe795f00700 1 -- 172.21.15.12:0/3923966897 learned_addr learned my addr 172.21.15.12:0/3923966897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe795f00700 1 -- 172.21.15.12:0/3923966897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7901037a0 msgr2=0x7fe790070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe795f00700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7901037a0 0x7fe790070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe795f00700 1 -- 172.21.15.12:0/3923966897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c005040 con 0x7fe790102e00 2024-03-18T15:00:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe795f00700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c8c0 tx=0x7fe7840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78400a760 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe78400ad90 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7840113a0 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe790071020 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.716+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79011a130 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.717+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7901052d0 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.718+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe784012400 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.718+0000 7fe7827fc700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 0x7fe77c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.718+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe78408d650 con 0x7fe790102e00 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.718+0000 7fe7956ff700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 0x7fe77c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.719+0000 7fe7956ff700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 0x7fe77c070390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7900719f0 tx=0x7fe78c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.723+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe784056c80 con 0x7fe790102e00 2024-03-18T15:00:24.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.943+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7900717a0 con 0x7fe77c06dee0 2024-03-18T15:00:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.953+0000 7fe7827fc700 1 -- 172.21.15.12:0/3923966897 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe7900717a0 con 0x7fe77c06dee0 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (6m) 115s ago 17m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (3m) 115s ago 17m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (3m) 16s ago 16m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (5m) 115s ago 16m 83.9M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (11m) 115s ago 11m 109M - 17.2.7-684-g6716c9f3 19857db3290c 1664967a9ffe 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (11m) 115s ago 11m 151M - 17.2.7-684-g6716c9f3 19857db3290c 995070b6275b 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (11m) 16s ago 11m 17.5M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T15:00:24.954 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (11m) 16s ago 11m 14.3M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (7m) 115s ago 18m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (7m) 16s ago 16m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (4m) 115s ago 18m 51.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (4m) 16s ago 16m 41.3M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (6m) 115s ago 17m 15.4M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (6m) 16s ago 16m 16.5M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (3m) 115s ago 15m 103M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T15:00:24.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (2m) 115s ago 14m 107M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T15:00:24.956 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (2m) 115s ago 14m 14.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T15:00:24.956 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (85s) 16s ago 14m 111M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T15:00:24.956 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (52s) 16s ago 13m 102M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T15:00:24.956 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (19s) 16s ago 13m 15.0M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9978289a0827 2024-03-18T15:00:24.956 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (6m) 115s ago 16m 66.3M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 msgr2=0x7fe77c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 0x7fe77c070390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7900719f0 tx=0x7fe78c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 msgr2=0x7fe7900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c8c0 tx=0x7fe7840044c0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 shutdown_connections 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe77c06dee0 0x7fe77c070390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe790102e00 0x7fe7900702c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 --2- 172.21.15.12:0/3923966897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7901037a0 0x7fe790070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 >> 172.21.15.12:0/3923966897 conn(0x7fe7900f89b0 msgr2=0x7fe7901021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 shutdown_connections 2024-03-18T15:00:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:24.955+0000 7fe798164700 1 -- 172.21.15.12:0/3923966897 wait complete. 2024-03-18T15:00:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 -- 172.21.15.12:0/131420982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c102e00 msgr2=0x7f874c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 --2- 172.21.15.12:0/131420982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c102e00 0x7f874c1031d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8748004940 tx=0x7f8748009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 -- 172.21.15.12:0/131420982 shutdown_connections 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 --2- 172.21.15.12:0/131420982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c1037a0 0x7f874c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 --2- 172.21.15.12:0/131420982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c102e00 0x7f874c1031d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 -- 172.21.15.12:0/131420982 >> 172.21.15.12:0/131420982 conn(0x7f874c0f89f0 msgr2=0x7f874c0fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 -- 172.21.15.12:0/131420982 shutdown_connections 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.057+0000 7f87546bb700 1 -- 172.21.15.12:0/131420982 wait complete. 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 Processor -- start 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 -- start start 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c102e00 0x7f874c113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c114410 con 0x7f874c1037a0 2024-03-18T15:00:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f87546bb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c10da50 con 0x7f874c102e00 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56698/0 (socket says 172.21.15.12:56698) 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 -- 172.21.15.12:0/1485398004 learned_addr learned my addr 172.21.15.12:0/1485398004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8752457700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c102e00 0x7f874c113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 -- 172.21.15.12:0/1485398004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c102e00 msgr2=0x7f874c113880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c102e00 0x7f874c113880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.058+0000 7f8751c56700 1 -- 172.21.15.12:0/1485398004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8748005040 con 0x7f874c1037a0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f8751c56700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f873c002a50 tx=0x7f873c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c007cc0 con 0x7f874c1037a0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f873c00f040 con 0x7f874c1037a0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c10dd30 con 0x7f874c1037a0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c00db30 con 0x7f874c1037a0 2024-03-18T15:00:25.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.059+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c10e2f0 con 0x7f874c1037a0 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.060+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c100110 con 0x7f874c1037a0 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.060+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f873c00dc90 con 0x7f874c1037a0 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.061+0000 7f8742ffd700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 0x7f8738070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.061+0000 7f8752457700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 0x7f8738070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.061+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f873c08d210 con 0x7f874c1037a0 2024-03-18T15:00:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.061+0000 7f8752457700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 0x7f8738070390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8748004f90 tx=0x7f8748009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.066+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f873c05a2d0 con 0x7f874c1037a0 2024-03-18T15:00:25.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.363+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f874c109f00 con 0x7f874c1037a0 2024-03-18T15:00:25.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:25 smithi012 ceph-mon[111904]: pgmap v173: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:00:25.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:25 smithi012 ceph-mon[111904]: from='client.44271 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:25 smithi012 ceph-mon[111904]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.364 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:25 smithi012 ceph-mon[111904]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.363+0000 7f8742ffd700 1 -- 172.21.15.12:0/1485398004 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f873c059e60 con 0x7f874c1037a0 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T15:00:25.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 3 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 3, 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 10 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T15:00:25.366 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 msgr2=0x7f8738070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 0x7f8738070390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8748004f90 tx=0x7f8748009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 msgr2=0x7f874c113dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f873c002a50 tx=0x7f873c0046b0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 shutdown_connections 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f873806dee0 0x7f8738070390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.366+0000 7f87546bb700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c102e00 0x7f874c113880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.367+0000 7f87546bb700 1 --2- 172.21.15.12:0/1485398004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f874c1037a0 0x7f874c113dc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.367+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 >> 172.21.15.12:0/1485398004 conn(0x7f874c0f89f0 msgr2=0x7f874c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.367+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 shutdown_connections 2024-03-18T15:00:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.367+0000 7f87546bb700 1 -- 172.21.15.12:0/1485398004 wait complete. 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- 172.21.15.12:0/493914313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc102720 msgr2=0x7f7bfc0fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 --2- 172.21.15.12:0/493914313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc102720 0x7f7bfc0fbd90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0004940 tx=0x7f7bf0009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- 172.21.15.12:0/493914313 shutdown_connections 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 --2- 172.21.15.12:0/493914313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc102720 0x7f7bfc0fbd90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 --2- 172.21.15.12:0/493914313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc101d80 0x7f7bfc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- 172.21.15.12:0/493914313 >> 172.21.15.12:0/493914313 conn(0x7f7bfc0f78f0 msgr2=0x7f7bfc0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- 172.21.15.12:0/493914313 shutdown_connections 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- 172.21.15.12:0/493914313 wait complete. 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 Processor -- start 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- start start 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc102720 0x7f7bfc10efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc10f4f0 con 0x7f7bfc101d80 2024-03-18T15:00:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.470+0000 7f7c0456b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc10f630 con 0x7f7bfc102720 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56724/0 (socket says 172.21.15.12:56724) 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 -- 172.21.15.12:0/3710136826 learned_addr learned my addr 172.21.15.12:0/3710136826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c01b06700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc102720 0x7f7bfc10efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 -- 172.21.15.12:0/3710136826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc102720 msgr2=0x7f7bfc10efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc102720 0x7f7bfc10efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 -- 172.21.15.12:0/3710136826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0005040 con 0x7f7bfc101d80 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c02307700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001db0 tx=0x7f7bec009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec00ed50 con 0x7f7bfc101d80 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bec007df0 con 0x7f7bfc101d80 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc117ae0 con 0x7f7bfc101d80 2024-03-18T15:00:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec014640 con 0x7f7bfc101d80 2024-03-18T15:00:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.471+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc117f00 con 0x7f7bfc101d80 2024-03-18T15:00:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.472+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc104250 con 0x7f7bfc101d80 2024-03-18T15:00:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.473+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7bec007920 con 0x7f7bfc101d80 2024-03-18T15:00:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.473+0000 7f7bfaffd700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 0x7f7be4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.473+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7bec08d420 con 0x7f7bfc101d80 2024-03-18T15:00:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.473+0000 7f7c01b06700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 0x7f7be4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.474+0000 7f7c01b06700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 0x7f7be4070390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc066a60 tx=0x7f7bf0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.478+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7bec056a60 con 0x7f7bfc101d80 2024-03-18T15:00:25.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:25 smithi100 ceph-mon[84868]: pgmap v173: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:00:25.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:25 smithi100 ceph-mon[84868]: from='client.44271 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:25 smithi100 ceph-mon[84868]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:25 smithi100 ceph-mon[84868]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.730+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7bfc108e60 con 0x7f7bfc101d80 2024-03-18T15:00:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.730+0000 7f7bfaffd700 1 -- 172.21.15.12:0/3710136826 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1704 (secure 0 0 0) 0x7f7bec05a120 con 0x7f7bfc101d80 2024-03-18T15:00:25.731 INFO:teuthology.orchestra.run.smithi012.stdout:e19 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T15:00:25.732 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T15:00:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 19 2024-03-18T15:00:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-18T15:00:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T15:00:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T15:00:19.133089+0000 2024-03-18T15:00:25.733 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T15:00:25.734 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 79 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14484} 2024-03-18T15:00:25.735 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T15:00:25.736 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{0:14484} state up:replay seq 3 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T15:00:25.737 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.12:6828/3243464363,v1:172.21.15.12:6829/3243464363] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 msgr2=0x7f7be4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 0x7f7be4070390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc066a60 tx=0x7f7bf0009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 msgr2=0x7f7bfc10ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001db0 tx=0x7f7bec009ee0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 shutdown_connections 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7be406dee0 0x7f7be4070390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bfc101d80 0x7f7bfc10ea70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 --2- 172.21.15.12:0/3710136826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bfc102720 0x7f7bfc10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 >> 172.21.15.12:0/3710136826 conn(0x7f7bfc0f78f0 msgr2=0x7f7bfc1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 shutdown_connections 2024-03-18T15:00:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.733+0000 7f7c0456b700 1 -- 172.21.15.12:0/3710136826 wait complete. 2024-03-18T15:00:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 -- 172.21.15.12:0/4127816734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fe830 msgr2=0x7f866c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 --2- 172.21.15.12:0/4127816734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fe830 0x7f866c106f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8660004940 tx=0x7f8660009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 -- 172.21.15.12:0/4127816734 shutdown_connections 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 --2- 172.21.15.12:0/4127816734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fe830 0x7f866c106f00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 --2- 172.21.15.12:0/4127816734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fde90 0x7f866c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 -- 172.21.15.12:0/4127816734 >> 172.21.15.12:0/4127816734 conn(0x7f866c072bc0 msgr2=0x7f866c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 -- 172.21.15.12:0/4127816734 shutdown_connections 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 -- 172.21.15.12:0/4127816734 wait complete. 2024-03-18T15:00:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.838+0000 7f867261c700 1 Processor -- start 2024-03-18T15:00:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f867261c700 1 -- start start 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f867261c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f867261c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fe830 0x7f866c114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f867261c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1146b0 con 0x7f866c0fde90 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f867261c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c10daa0 con 0x7f866c0fe830 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56746/0 (socket says 172.21.15.12:56746) 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 -- 172.21.15.12:0/4223848045 learned_addr learned my addr 172.21.15.12:0/4223848045 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866b7fe700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fe830 0x7f866c114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 -- 172.21.15.12:0/4223848045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fe830 msgr2=0x7f866c114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fe830 0x7f866c114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 -- 172.21.15.12:0/4223848045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660005040 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f866bfff700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f865c002a80 tx=0x7f865c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.839+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865c007cc0 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.840+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c10dd80 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.840+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f865c00f040 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.840+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865c00dac0 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.840+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c10e2c0 con 0x7f866c0fde90 2024-03-18T15:00:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.841+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c065e30 con 0x7f866c0fde90 2024-03-18T15:00:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.841+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f865c00dc20 con 0x7f866c0fde90 2024-03-18T15:00:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.842+0000 7f8668ff9700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 0x7f8654070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.842+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f865c08d3e0 con 0x7f866c0fde90 2024-03-18T15:00:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.842+0000 7f866b7fe700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 0x7f8654070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.842+0000 7f866b7fe700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 0x7f8654070390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f866c10ec60 tx=0x7f8660009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:25.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:25.846+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f865c05a380 con 0x7f866c0fde90 2024-03-18T15:00:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.073+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f866c109f10 con 0x7f865406dee0 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.075+0000 7f8668ff9700 1 -- 172.21.15.12:0/4223848045 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f866c109f10 con 0x7f865406dee0 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T15:00:26.075 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "osd", 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "12/21 daemons upgraded", 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mds daemons", 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T15:00:26.076 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:00:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 msgr2=0x7f8654070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 0x7f8654070390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f866c10ec60 tx=0x7f8660009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 msgr2=0x7f866c113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f865c002a80 tx=0x7f865c0046b0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 shutdown_connections 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f865406dee0 0x7f8654070390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f866c0fde90 0x7f866c113b20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 --2- 172.21.15.12:0/4223848045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f866c0fe830 0x7f866c114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 >> 172.21.15.12:0/4223848045 conn(0x7f866c072bc0 msgr2=0x7f866c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 shutdown_connections 2024-03-18T15:00:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.077+0000 7f867261c700 1 -- 172.21.15.12:0/4223848045 wait complete. 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 -- 172.21.15.12:0/2403344405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800102710 msgr2=0x7fd8000fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 --2- 172.21.15.12:0/2403344405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800102710 0x7fd8000fbd80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc004940 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 -- 172.21.15.12:0/2403344405 shutdown_connections 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 --2- 172.21.15.12:0/2403344405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800102710 0x7fd8000fbd80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 --2- 172.21.15.12:0/2403344405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800101d70 0x7fd800102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 -- 172.21.15.12:0/2403344405 >> 172.21.15.12:0/2403344405 conn(0x7fd800072bc0 msgr2=0x7fd800072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:26.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 -- 172.21.15.12:0/2403344405 shutdown_connections 2024-03-18T15:00:26.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.179+0000 7fd80744d700 1 -- 172.21.15.12:0/2403344405 wait complete. 2024-03-18T15:00:26.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 Processor -- start 2024-03-18T15:00:26.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 -- start start 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800102710 0x7fd800111ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd800112510 con 0x7fd800101d70 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd80744d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80010b900 con 0x7fd800102710 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd8051e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd8051e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56764/0 (socket says 172.21.15.12:56764) 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd8051e9700 1 -- 172.21.15.12:0/1948156181 learned_addr learned my addr 172.21.15.12:0/1948156181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.180+0000 7fd8049e8700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800102710 0x7fd800111ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd8051e9700 1 -- 172.21.15.12:0/1948156181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800102710 msgr2=0x7fd800111ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd8051e9700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800102710 0x7fd800111ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd8051e9700 1 -- 172.21.15.12:0/1948156181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc005040 con 0x7fd800101d70 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd8051e9700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0004f50 tx=0x7fd7f0004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f000fc10 con 0x7fd800101d70 2024-03-18T15:00:26.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f0011040 con 0x7fd800101d70 2024-03-18T15:00:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80010bbe0 con 0x7fd800101d70 2024-03-18T15:00:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f000b310 con 0x7fd800101d70 2024-03-18T15:00:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.181+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80010c170 con 0x7fd800101d70 2024-03-18T15:00:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.182+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd800104240 con 0x7fd800101d70 2024-03-18T15:00:26.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.182+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd7f00153f0 con 0x7fd800101d70 2024-03-18T15:00:26.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.183+0000 7fd7f5ffb700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 0x7fd7ec0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:26.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.183+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd7f008fc10 con 0x7fd800101d70 2024-03-18T15:00:26.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.183+0000 7fd8049e8700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 0x7fd7ec0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:26.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.183+0000 7fd8049e8700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 0x7fd7ec0703a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd80010cbe0 tx=0x7fd7fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:26.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.188+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7f005ccd0 con 0x7fd800101d70 2024-03-18T15:00:26.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:26 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1485398004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:26.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:26 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3710136826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:00:26.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:26 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.487+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd80010cfd0 con 0x7fd800101d70 2024-03-18T15:00:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.487+0000 7fd7f5ffb700 1 -- 172.21.15.12:0/1948156181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7fd7f005c860 con 0x7fd800101d70 2024-03-18T15:00:26.488 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_WARN 1 filesystem is degraded 2024-03-18T15:00:26.488 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-18T15:00:26.488 INFO:teuthology.orchestra.run.smithi012.stdout: fs cephfs is degraded 2024-03-18T15:00:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 msgr2=0x7fd7ec0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 0x7fd7ec0703a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd80010cbe0 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 msgr2=0x7fd800111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0004f50 tx=0x7fd7f0004b80 comp rx=0 tx=0).stop 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 shutdown_connections 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd7ec06def0 0x7fd7ec0703a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd800101d70 0x7fd800111980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 --2- 172.21.15.12:0/1948156181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800102710 0x7fd800111ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 >> 172.21.15.12:0/1948156181 conn(0x7fd800072bc0 msgr2=0x7fd8000fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 shutdown_connections 2024-03-18T15:00:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:26.490+0000 7fd80744d700 1 -- 172.21.15.12:0/1948156181 wait complete. 2024-03-18T15:00:26.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:26 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1485398004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:26.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:26 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3710136826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:00:26.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:26 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:27 smithi012 ceph-mon[111904]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:27 smithi012 ceph-mon[111904]: pgmap v174: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-18T15:00:27.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:27 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1948156181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:00:27.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:27 smithi100 ceph-mon[84868]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:27.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:27 smithi100 ceph-mon[84868]: pgmap v174: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-18T15:00:27.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:27 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1948156181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: pgmap v175: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0.000999973 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0.000999973 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:reconnect 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:reconnect} 2 up:standby 2024-03-18T15:00:29.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:29.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:29.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:29 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: pgmap v175: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0.000999973 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0.000999973 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:reconnect 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:reconnect} 2 up:standby 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:29.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:29 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:30.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:30 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:rejoin 2024-03-18T15:00:30.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:30 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:boot 2024-03-18T15:00:30.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:30 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:rejoin} 3 up:standby 2024-03-18T15:00:30.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:30 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T15:00:30.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:30 smithi012 ceph-mon[111904]: daemon mds.cephfs.smithi100.dvtbrf is now active in filesystem cephfs as rank 0 2024-03-18T15:00:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:30 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:rejoin 2024-03-18T15:00:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:30 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:boot 2024-03-18T15:00:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:30 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.dvtbrf=up:rejoin} 3 up:standby 2024-03-18T15:00:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:30 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T15:00:30.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:30 smithi100 ceph-mon[84868]: daemon mds.cephfs.smithi100.dvtbrf is now active in filesystem cephfs as rank 0 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: pgmap v176: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 93 B/s wr, 5 op/s 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:active 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 3 up:standby 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:31.457 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: pgmap v176: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 93 B/s wr, 5 op/s 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] up:active 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 3 up:standby 2024-03-18T15:00:31.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:31.546 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:33.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:33 smithi012 ceph-mon[111904]: pgmap v177: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-18T15:00:33.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:33 smithi100 ceph-mon[84868]: pgmap v177: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-18T15:00:35.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:35 smithi012 ceph-mon[111904]: pgmap v178: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-18T15:00:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:35.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:35 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:35.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:35 smithi100 ceph-mon[84868]: pgmap v178: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-18T15:00:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:35.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:35 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:36.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:36 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:36.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:36 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:36 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:36 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:37.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:37 smithi012 ceph-mon[111904]: pgmap v179: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-18T15:00:38.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:37 smithi100 ceph-mon[84868]: pgmap v179: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: pgmap v180: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:39.883 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: pgmap v180: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:40.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: Upgrade: Updating mds.cephfs.smithi012.oheuui 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:42.224 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:41 smithi012 ceph-mon[111904]: Deploying daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: Upgrade: Updating mds.cephfs.smithi012.oheuui 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi012.oheuui", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:41 smithi100 ceph-mon[84868]: Deploying daemon mds.cephfs.smithi012.oheuui on smithi012 2024-03-18T15:00:44.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:43 smithi012 ceph-mon[111904]: pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-18T15:00:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:43 smithi100 ceph-mon[84868]: pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-18T15:00:45.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:44 smithi012 ceph-mon[111904]: osdmap e80: 6 total, 6 up, 6 in 2024-03-18T15:00:45.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:44 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 2 up:standby 2024-03-18T15:00:45.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:44 smithi100 ceph-mon[84868]: osdmap e80: 6 total, 6 up, 6 in 2024-03-18T15:00:45.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:44 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 2 up:standby 2024-03-18T15:00:46.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:45 smithi012 ceph-mon[111904]: pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-03-18T15:00:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:46.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:45 smithi100 ceph-mon[84868]: pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-03-18T15:00:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:47.221 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:00:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:47 smithi012 ceph-mon[111904]: pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:48.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:47 smithi100 ceph-mon[84868]: pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: pgmap v186: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6828/846568565,v1:172.21.15.12:6829/846568565] up:boot 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 3 up:standby 2024-03-18T15:00:49.644 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:49.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: pgmap v186: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6828/846568565,v1:172.21.15.12:6829/846568565] up:boot 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi100.dvtbrf=up:active} 3 up:standby 2024-03-18T15:00:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:00:51.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:51 smithi100 ceph-mon[84868]: pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:51.858 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:51 smithi012 ceph-mon[111904]: pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:53 smithi100 ceph-mon[84868]: pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:53.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:53 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:53.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:53 smithi012 ceph-mon[111904]: pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:53.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:53.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:53 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:55 smithi012 ceph-mon[111904]: pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:55 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:55.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:55 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:55.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:55 smithi100 ceph-mon[84868]: pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:55.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:55 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:55.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:55 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.600+0000 7f13967ed700 1 -- 172.21.15.12:0/1380010010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 msgr2=0x7f13901031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.600+0000 7f13967ed700 1 --2- 172.21.15.12:0/1380010010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13901031e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1380004940 tx=0x7f1380009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.600+0000 7f13967ed700 1 -- 172.21.15.12:0/1380010010 shutdown_connections 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.600+0000 7f13967ed700 1 --2- 172.21.15.12:0/1380010010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 0x7f13900fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.600+0000 7f13967ed700 1 --2- 172.21.15.12:0/1380010010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13901031e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- 172.21.15.12:0/1380010010 >> 172.21.15.12:0/1380010010 conn(0x7f13900f89a0 msgr2=0x7f13900fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- 172.21.15.12:0/1380010010 shutdown_connections 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- 172.21.15.12:0/1380010010 wait complete. 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 Processor -- start 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- start start 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 0x7f1390070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390070d40 con 0x7f1390102e10 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.601+0000 7f13967ed700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390070e80 con 0x7f13901037b0 2024-03-18T15:00:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43256/0 (socket says 172.21.15.12:43256) 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 -- 172.21.15.12:0/2086505987 learned_addr learned my addr 172.21.15.12:0/2086505987 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138f7fe700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 0x7f1390070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 -- 172.21.15.12:0/2086505987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 msgr2=0x7f1390070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 0x7f1390070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 -- 172.21.15.12:0/2086505987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380005040 con 0x7f1390102e10 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138ffff700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f138000b3b0 tx=0x7f1380009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138000e040 con 0x7f1390102e10 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1380013420 con 0x7f1390102e10 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13900710d0 con 0x7f1390102e10 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138001c630 con 0x7f1390102e10 2024-03-18T15:00:56.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.602+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139011a2f0 con 0x7f1390102e10 2024-03-18T15:00:56.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.603+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1390100120 con 0x7f1390102e10 2024-03-18T15:00:56.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.604+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f1380013590 con 0x7f1390102e10 2024-03-18T15:00:56.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.604+0000 7f138cff9700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 0x7f1378070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.605+0000 7f138f7fe700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 0x7f1378070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.605+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f138008e860 con 0x7f1390102e10 2024-03-18T15:00:56.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.605+0000 7f138f7fe700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 0x7f1378070390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1390071bb0 tx=0x7f138400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:56.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.609+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1380057df0 con 0x7f1390102e10 2024-03-18T15:00:56.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.836+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f139004bed0 con 0x7f137806dee0 2024-03-18T15:00:56.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.837+0000 7f138cff9700 1 -- 172.21.15.12:0/2086505987 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f139004bed0 con 0x7f137806dee0 2024-03-18T15:00:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 msgr2=0x7f1378070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 0x7f1378070390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1390071bb0 tx=0x7f138400b040 comp rx=0 tx=0).stop 2024-03-18T15:00:56.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 msgr2=0x7f13900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f138000b3b0 tx=0x7f1380009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 shutdown_connections 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f137806dee0 0x7f1378070390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1390102e10 0x7f13900702c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 --2- 172.21.15.12:0/2086505987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13901037b0 0x7f1390070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 >> 172.21.15.12:0/2086505987 conn(0x7f13900f89a0 msgr2=0x7f13900facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 shutdown_connections 2024-03-18T15:00:56.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.840+0000 7f13967ed700 1 -- 172.21.15.12:0/2086505987 wait complete. 2024-03-18T15:00:56.852 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 -- 172.21.15.12:0/1470710373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e94102e00 msgr2=0x7f7e941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1470710373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e94102e00 0x7f7e941031d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ce0 tx=0x7f7e84004d10 comp rx=0 tx=0).stop 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 -- 172.21.15.12:0/1470710373 shutdown_connections 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1470710373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e941037a0 0x7f7e940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1470710373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e94102e00 0x7f7e941031d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 -- 172.21.15.12:0/1470710373 >> 172.21.15.12:0/1470710373 conn(0x7f7e940f89b0 msgr2=0x7f7e940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 -- 172.21.15.12:0/1470710373 shutdown_connections 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.947+0000 7f7e9c148700 1 -- 172.21.15.12:0/1470710373 wait complete. 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 Processor -- start 2024-03-18T15:00:56.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 -- start start 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e94102e00 0x7f7e940702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940721c0 con 0x7f7e941037a0 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e9c148700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94070d40 con 0x7f7e94102e00 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e996e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e996e3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43262/0 (socket says 172.21.15.12:43262) 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.948+0000 7f7e996e3700 1 -- 172.21.15.12:0/1671131067 learned_addr learned my addr 172.21.15.12:0/1671131067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e99ee4700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e94102e00 0x7f7e940702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e996e3700 1 -- 172.21.15.12:0/1671131067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e94102e00 msgr2=0x7f7e940702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e996e3700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e94102e00 0x7f7e940702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e996e3700 1 -- 172.21.15.12:0/1671131067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84009940 con 0x7f7e941037a0 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e996e3700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e88002a50 tx=0x7f7e88004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:56.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e88007cc0 con 0x7f7e941037a0 2024-03-18T15:00:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8800f040 con 0x7f7e941037a0 2024-03-18T15:00:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e94071020 con 0x7f7e941037a0 2024-03-18T15:00:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9411a2e0 con 0x7f7e941037a0 2024-03-18T15:00:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.949+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8800d9c0 con 0x7f7e941037a0 2024-03-18T15:00:56.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.950+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e800052b0 con 0x7f7e941037a0 2024-03-18T15:00:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.951+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7e8800dc00 con 0x7f7e941037a0 2024-03-18T15:00:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.951+0000 7f7e927fc700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 0x7f7e7c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.951+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7e8808d420 con 0x7f7e941037a0 2024-03-18T15:00:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.951+0000 7f7e99ee4700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 0x7f7e7c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:56.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.952+0000 7f7e99ee4700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 0x7f7e7c070390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400b740 tx=0x7f7e84004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:56.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:56.956+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e88056930 con 0x7f7e941037a0 2024-03-18T15:00:57.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.187+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e80000bc0 con 0x7f7e7c06dee0 2024-03-18T15:00:57.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.189+0000 7f7e927fc700 1 -- 172.21.15.12:0/1671131067 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7e80000bc0 con 0x7f7e7c06dee0 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 msgr2=0x7f7e7c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 0x7f7e7c070390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400b740 tx=0x7f7e84004d10 comp rx=0 tx=0).stop 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 msgr2=0x7f7e94070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e88002a50 tx=0x7f7e88004600 comp rx=0 tx=0).stop 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 shutdown_connections 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7e7c06dee0 0x7f7e7c070390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e94102e00 0x7f7e940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 --2- 172.21.15.12:0/1671131067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e941037a0 0x7f7e94070800 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 >> 172.21.15.12:0/1671131067 conn(0x7f7e940f89b0 msgr2=0x7f7e941073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 shutdown_connections 2024-03-18T15:00:57.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.192+0000 7f7e9c148700 1 -- 172.21.15.12:0/1671131067 wait complete. 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 -- 172.21.15.12:0/1215004562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418102690 msgr2=0x7ff4180fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/1215004562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418102690 0x7ff4180fbd00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff400004940 tx=0x7ff400009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 -- 172.21.15.12:0/1215004562 shutdown_connections 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/1215004562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418102690 0x7ff4180fbd00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/1215004562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418101cf0 0x7ff4181020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 -- 172.21.15.12:0/1215004562 >> 172.21.15.12:0/1215004562 conn(0x7ff4180f78a0 msgr2=0x7ff4180f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 -- 172.21.15.12:0/1215004562 shutdown_connections 2024-03-18T15:00:57.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.299+0000 7ff41dcdc700 1 -- 172.21.15.12:0/1215004562 wait complete. 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 Processor -- start 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 -- start start 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 0x7ff41810ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41810d390 con 0x7ff418101cf0 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff41dcdc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41810d500 con 0x7ff418102690 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff4177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff4177fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43286/0 (socket says 172.21.15.12:43286) 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff4177fe700 1 -- 172.21.15.12:0/2730803040 learned_addr learned my addr 172.21.15.12:0/2730803040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.300+0000 7ff40effd700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 0x7ff41810ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff4177fe700 1 -- 172.21.15.12:0/2730803040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 msgr2=0x7ff41810ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff4177fe700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 0x7ff41810ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff4177fe700 1 -- 172.21.15.12:0/2730803040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400005040 con 0x7ff418101cf0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff40effd700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 0x7ff41810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff4177fe700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c830 tx=0x7ff408004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40800a760 con 0x7ff418101cf0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff40800ad90 con 0x7ff418101cf0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff418119f80 con 0x7ff418101cf0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41811a350 con 0x7ff418101cf0 2024-03-18T15:00:57.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.301+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4080113a0 con 0x7ff418101cf0 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.302+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f80052b0 con 0x7ff418101cf0 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.303+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ff408011600 con 0x7ff418101cf0 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.303+0000 7ff414ff9700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 0x7ff4040703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.303+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff40808d5a0 con 0x7ff418101cf0 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.303+0000 7ff40effd700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 0x7ff4040703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.304+0000 7ff40effd700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 0x7ff4040703a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff418066890 tx=0x7ff400009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:57.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.308+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff40805a700 con 0x7ff418101cf0 2024-03-18T15:00:57.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:57 smithi012 ceph-mon[111904]: pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:57.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:57 smithi012 ceph-mon[111904]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:57.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.527+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3f8000bc0 con 0x7ff40406def0 2024-03-18T15:00:57.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.537+0000 7ff414ff9700 1 -- 172.21.15.12:0/2730803040 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff3f8000bc0 con 0x7ff40406def0 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (6m) 4s ago 17m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (4m) 4s ago 18m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (4m) 48s ago 16m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (6m) 4s ago 17m 85.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (9s) 4s ago 12m 12.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 89022f9c2ba2 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (28s) 4s ago 12m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 269c33e068f3 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (12m) 48s ago 12m 17.5M - 17.2.7-684-g6716c9f3 19857db3290c 944db8112caa 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (12m) 48s ago 12m 14.3M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (8m) 4s ago 19m 509M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (7m) 48s ago 16m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T15:00:57.538 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (5m) 4s ago 19m 57.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (4m) 48s ago 16m 41.3M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (7m) 4s ago 18m 15.7M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (7m) 48s ago 16m 16.5M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (3m) 4s ago 15m 110M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (3m) 4s ago 15m 120M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (2m) 4s ago 15m 104M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (118s) 48s ago 14m 111M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (84s) 48s ago 14m 102M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (52s) 48s ago 13m 15.0M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9978289a0827 2024-03-18T15:00:57.539 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (7m) 4s ago 17m 66.4M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 msgr2=0x7ff4040703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 0x7ff4040703a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff418066890 tx=0x7ff400009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 msgr2=0x7ff41810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c830 tx=0x7ff408004470 comp rx=0 tx=0).stop 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 shutdown_connections 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ff40406def0 0x7ff4040703a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff418101cf0 0x7ff41810c910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 --2- 172.21.15.12:0/2730803040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff418102690 0x7ff41810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 >> 172.21.15.12:0/2730803040 conn(0x7ff4180f78a0 msgr2=0x7ff4180f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 shutdown_connections 2024-03-18T15:00:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.539+0000 7ff41dcdc700 1 -- 172.21.15.12:0/2730803040 wait complete. 2024-03-18T15:00:57.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:57 smithi100 ceph-mon[84868]: pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:00:57.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:57 smithi100 ceph-mon[84868]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 -- 172.21.15.12:0/499373124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 msgr2=0x7fcf1c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/499373124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c1031d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004940 tx=0x7fcf0c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 -- 172.21.15.12:0/499373124 shutdown_connections 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/499373124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 0x7fcf1c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/499373124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c1031d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.642+0000 7fcf20cc2700 1 -- 172.21.15.12:0/499373124 >> 172.21.15.12:0/499373124 conn(0x7fcf1c0f89b0 msgr2=0x7fcf1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 -- 172.21.15.12:0/499373124 shutdown_connections 2024-03-18T15:00:57.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 -- 172.21.15.12:0/499373124 wait complete. 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 Processor -- start 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 -- start start 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 0x7fcf1c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c114520 con 0x7fcf1c102e00 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf20cc2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c10db70 con 0x7fcf1c1037a0 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf1a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf1a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43304/0 (socket says 172.21.15.12:43304) 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf1a59c700 1 -- 172.21.15.12:0/3601514275 learned_addr learned my addr 172.21.15.12:0/3601514275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.643+0000 7fcf19d9b700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 0x7fcf1c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf1a59c700 1 -- 172.21.15.12:0/3601514275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 msgr2=0x7fcf1c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf1a59c700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 0x7fcf1c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf1a59c700 1 -- 172.21.15.12:0/3601514275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c005040 con 0x7fcf1c102e00 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf1a59c700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b3b0 tx=0x7fcf0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c00e040 con 0x7fcf1c102e00 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0c013420 con 0x7fcf1c102e00 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c10ddf0 con 0x7fcf1c102e00 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c01c630 con 0x7fcf1c102e00 2024-03-18T15:00:57.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.644+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c10e3b0 con 0x7fcf1c102e00 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.645+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c100110 con 0x7fcf1c102e00 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.645+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcf0c013590 con 0x7fcf1c102e00 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.646+0000 7fcf12ffd700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 0x7fcf08070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.646+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcf0c08e7f0 con 0x7fcf1c102e00 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.646+0000 7fcf19d9b700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 0x7fcf08070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:57.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.646+0000 7fcf19d9b700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 0x7fcf08070390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c10ece0 tx=0x7fcf0400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:57.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.650+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcf0c05b950 con 0x7fcf1c102e00 2024-03-18T15:00:57.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.954+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcf1c10efe0 con 0x7fcf1c102e00 2024-03-18T15:00:57.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.955+0000 7fcf12ffd700 1 -- 172.21.15.12:0/3601514275 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fcf0c05b4e0 con 0x7fcf1c102e00 2024-03-18T15:00:57.955 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:00:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T15:00:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:00:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 17.2.7-684-g6716c9f3 (6716c9f3976cf9fc2fd546e9f1182c2d4421b451) quincy (stable)": 2, 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 12 2024-03-18T15:00:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T15:00:57.957 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:00:57.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 msgr2=0x7fcf08070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 0x7fcf08070390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c10ece0 tx=0x7fcf0400b040 comp rx=0 tx=0).stop 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 msgr2=0x7fcf1c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b3b0 tx=0x7fcf0c009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 shutdown_connections 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcf0806dee0 0x7fcf08070390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1c102e00 0x7fcf1c113990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 --2- 172.21.15.12:0/3601514275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf1c1037a0 0x7fcf1c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 >> 172.21.15.12:0/3601514275 conn(0x7fcf1c0f89b0 msgr2=0x7fcf1c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 shutdown_connections 2024-03-18T15:00:57.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:57.957+0000 7fcf20cc2700 1 -- 172.21.15.12:0/3601514275 wait complete. 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/744556776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 msgr2=0x7f7c441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/744556776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c441031d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c34004940 tx=0x7f7c34009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/744556776 shutdown_connections 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/744556776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 0x7f7c440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/744556776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c441031d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/744556776 >> 172.21.15.12:0/744556776 conn(0x7f7c440f89b0 msgr2=0x7f7c440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:58.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/744556776 shutdown_connections 2024-03-18T15:00:58.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.060+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/744556776 wait complete. 2024-03-18T15:00:58.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 Processor -- start 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 -- start start 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 0x7f7c441160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44116780 con 0x7f7c44102e00 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c4a3fa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4410fcd0 con 0x7f7c441037a0 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43324/0 (socket says 172.21.15.12:43324) 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 -- 172.21.15.12:0/3863550709 learned_addr learned my addr 172.21.15.12:0/3863550709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c437fe700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 0x7f7c441160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 -- 172.21.15.12:0/3863550709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 msgr2=0x7f7c441160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 0x7f7c441160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.061+0000 7f7c43fff700 1 -- 172.21.15.12:0/3863550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c34005040 con 0x7f7c44102e00 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c43fff700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400b3b0 tx=0x7f7c34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3400e040 con 0x7f7c44102e00 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3400bbc0 con 0x7f7c44102e00 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3401c3f0 con 0x7f7c44102e00 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4410fef0 con 0x7f7c44102e00 2024-03-18T15:00:58.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.062+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c441104b0 con 0x7f7c44102e00 2024-03-18T15:00:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.063+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c440fe550 con 0x7f7c44102e00 2024-03-18T15:00:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.064+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7c3400bd30 con 0x7f7c44102e00 2024-03-18T15:00:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.064+0000 7f7c40ff9700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 0x7f7c2c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.064+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7c3408e8b0 con 0x7f7c44102e00 2024-03-18T15:00:58.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.064+0000 7f7c437fe700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 0x7f7c2c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.064+0000 7f7c437fe700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 0x7f7c2c070390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.069+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c3405b970 con 0x7f7c44102e00 2024-03-18T15:00:58.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.328+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c441111b0 con 0x7f7c44102e00 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.328+0000 7f7c40ff9700 1 -- 172.21.15.12:0/3863550709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1891 (secure 0 0 0) 0x7f7c3405b500 con 0x7f7c44102e00 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:e24 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 22 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-18T15:00:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T15:00:30.219376+0000 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 79 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T15:00:58.330 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=14484} 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 14484 members: 14484 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{0:14484} state up:active seq 179 join_fscid=1 addr [v2:172.21.15.100:6824/857186455,v1:172.21.15.100:6825/857186455] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:58.331 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:24277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.12:6828/846568565,v1:172.21.15.12:6829/846568565] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:00:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 msgr2=0x7f7c2c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 0x7f7c2c070390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38008040 comp rx=0 tx=0).stop 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 msgr2=0x7f7c44115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400b3b0 tx=0x7f7c34009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 shutdown_connections 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7c2c06dee0 0x7f7c2c070390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c44102e00 0x7f7c44115b60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 --2- 172.21.15.12:0/3863550709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c441037a0 0x7f7c441160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 >> 172.21.15.12:0/3863550709 conn(0x7f7c440f89b0 msgr2=0x7f7c44071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 shutdown_connections 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.330+0000 7f7c4a3fa700 1 -- 172.21.15.12:0/3863550709 wait complete. 2024-03-18T15:00:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- 172.21.15.12:0/2662882700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8101d70 msgr2=0x7fd8f8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 --2- 172.21.15.12:0/2662882700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8101d70 0x7fd8f8102140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009ce0 tx=0x7fd8f0004d10 comp rx=0 tx=0).stop 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- 172.21.15.12:0/2662882700 shutdown_connections 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 --2- 172.21.15.12:0/2662882700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8102710 0x7fd8f80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 --2- 172.21.15.12:0/2662882700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8101d70 0x7fd8f8102140 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- 172.21.15.12:0/2662882700 >> 172.21.15.12:0/2662882700 conn(0x7fd8f8072bc0 msgr2=0x7fd8f8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- 172.21.15.12:0/2662882700 shutdown_connections 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- 172.21.15.12:0/2662882700 wait complete. 2024-03-18T15:00:58.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 Processor -- start 2024-03-18T15:00:58.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.434+0000 7fd8ff090700 1 -- start start 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8ff090700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8101d70 0x7fd8f8115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8ff090700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8ff090700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81169c0 con 0x7fd8f8102710 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8ff090700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f810fe40 con 0x7fd8f8101d70 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39302/0 (socket says 172.21.15.12:39302) 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 -- 172.21.15.12:0/406779502 learned_addr learned my addr 172.21.15.12:0/406779502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8fce2c700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8101d70 0x7fd8f8115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 -- 172.21.15.12:0/406779502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8101d70 msgr2=0x7fd8f8115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8101d70 0x7fd8f8115e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 -- 172.21.15.12:0/406779502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0009940 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f7fff700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8002a50 tx=0x7fd8e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e8007cc0 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8110120 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8e800f040 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.435+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e800d9c0 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.436+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f81106e0 con 0x7fd8f8102710 2024-03-18T15:00:58.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.436+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f80fd4c0 con 0x7fd8f8102710 2024-03-18T15:00:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.437+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd8e8009730 con 0x7fd8f8102710 2024-03-18T15:00:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.437+0000 7fd8f57fa700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 0x7fd8e4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.437+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd8e808d430 con 0x7fd8f8102710 2024-03-18T15:00:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.437+0000 7fd8fce2c700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 0x7fd8e4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.438+0000 7fd8fce2c700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 0x7fd8e4070390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000b740 tx=0x7fd8f0004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.442+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8e8056940 con 0x7fd8f8102710 2024-03-18T15:00:58.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.669+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8f8108e70 con 0x7fd8e406dee0 2024-03-18T15:00:58.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.670+0000 7fd8f57fa700 1 -- 172.21.15.12:0/406779502 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd8f8108e70 con 0x7fd8e406dee0 2024-03-18T15:00:58.671 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:00:58.671 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T15:00:58.671 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T15:00:58.671 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "osd", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "14/21 daemons upgraded", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mds daemons", 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T15:00:58.672 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:00:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 msgr2=0x7fd8e4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 0x7fd8e4070390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000b740 tx=0x7fd8f0004d10 comp rx=0 tx=0).stop 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 msgr2=0x7fd8f8116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8002a50 tx=0x7fd8e8004600 comp rx=0 tx=0).stop 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 shutdown_connections 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd8e406dee0 0x7fd8e4070390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f8101d70 0x7fd8f8115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 --2- 172.21.15.12:0/406779502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8f8102710 0x7fd8f8116370 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 >> 172.21.15.12:0/406779502 conn(0x7fd8f8072bc0 msgr2=0x7fd8f8065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 shutdown_connections 2024-03-18T15:00:58.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.673+0000 7fd8ff090700 1 -- 172.21.15.12:0/406779502 wait complete. 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 -- 172.21.15.12:0/2807058434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2941026c0 msgr2=0x7fd2940fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 --2- 172.21.15.12:0/2807058434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2941026c0 0x7fd2940fbd30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd290004940 tx=0x7fd290009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 -- 172.21.15.12:0/2807058434 shutdown_connections 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 --2- 172.21.15.12:0/2807058434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2941026c0 0x7fd2940fbd30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 --2- 172.21.15.12:0/2807058434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd294101d20 0x7fd2941020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 -- 172.21.15.12:0/2807058434 >> 172.21.15.12:0/2807058434 conn(0x7fd2940f7870 msgr2=0x7fd2940f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 -- 172.21.15.12:0/2807058434 shutdown_connections 2024-03-18T15:00:58.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.776+0000 7fd29b644700 1 -- 172.21.15.12:0/2807058434 wait complete. 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 Processor -- start 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 -- start start 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2941026c0 0x7fd294108bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29410a690 con 0x7fd294101d20 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd29b644700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294109130 con 0x7fd2941026c0 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39322/0 (socket says 172.21.15.12:39322) 2024-03-18T15:00:58.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 -- 172.21.15.12:0/233593782 learned_addr learned my addr 172.21.15.12:0/233593782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd298bdf700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2941026c0 0x7fd294108bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 -- 172.21.15.12:0/233593782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2941026c0 msgr2=0x7fd294108bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2941026c0 0x7fd294108bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.777+0000 7fd2993e0700 1 -- 172.21.15.12:0/233593782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290005040 con 0x7fd294101d20 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd2993e0700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b790 tx=0x7fd284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28400f770 con 0x7fd294101d20 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd28400fda0 con 0x7fd294101d20 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294109410 con 0x7fd294101d20 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28400d310 con 0x7fd294101d20 2024-03-18T15:00:58.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.778+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29411a210 con 0x7fd294101d20 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.779+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2940fd470 con 0x7fd294101d20 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.779+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd284010420 con 0x7fd294101d20 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.780+0000 7fd289ffb700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 0x7fd280070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.780+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd28408d720 con 0x7fd294101d20 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.780+0000 7fd298bdf700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 0x7fd280070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:00:58.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.780+0000 7fd298bdf700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 0x7fd280070390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd29000b3b0 tx=0x7fd290009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:00:58.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:58.784+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd284056cb0 con 0x7fd294101d20 2024-03-18T15:00:58.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3601514275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:58.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:58.993 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:58 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3863550709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:00:59.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.082+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd29404c710 con 0x7fd294101d20 2024-03-18T15:00:59.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.082+0000 7fd289ffb700 1 -- 172.21.15.12:0/233593782 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd28405a370 con 0x7fd294101d20 2024-03-18T15:00:59.083 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 msgr2=0x7fd280070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 0x7fd280070390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd29000b3b0 tx=0x7fd290009e30 comp rx=0 tx=0).stop 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 msgr2=0x7fd2941086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b790 tx=0x7fd284009250 comp rx=0 tx=0).stop 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 shutdown_connections 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd28006dee0 0x7fd280070390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd294101d20 0x7fd2941086b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 --2- 172.21.15.12:0/233593782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2941026c0 0x7fd294108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 >> 172.21.15.12:0/233593782 conn(0x7fd2940f7870 msgr2=0x7fd2940f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 shutdown_connections 2024-03-18T15:00:59.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:00:59.084+0000 7fd29b644700 1 -- 172.21.15.12:0/233593782 wait complete. 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:00:59.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3601514275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:00:59.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:58 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3863550709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:01:00.206 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: pgmap v191: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:01:00.206 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:00.206 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/233593782' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:01:00.207 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:00.207 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:01:00.207 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:00:59 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:00.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: pgmap v191: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:01:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/233593782' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:01:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.dvtbrf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:01:00.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:00:59 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:01.214 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:00 smithi100 ceph-mon[84868]: Upgrade: Updating mds.cephfs.smithi100.dvtbrf 2024-03-18T15:01:01.214 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:00 smithi100 ceph-mon[84868]: Deploying daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T15:01:01.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:00 smithi012 ceph-mon[111904]: Upgrade: Updating mds.cephfs.smithi100.dvtbrf 2024-03-18T15:01:01.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:00 smithi012 ceph-mon[111904]: Deploying daemon mds.cephfs.smithi100.dvtbrf on smithi100 2024-03-18T15:01:01.896 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:01.665+0000 7fb5c3f8e700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-03-18T15:01:01.896 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:01.666+0000 7fb5c5791700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-18T15:01:01.896 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:01.666+0000 7fb5c0787700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 5 2024-03-18T15:01:01.896 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:01.669+0000 7fb5c1f8a700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 14 2024-03-18T15:01:01.944 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[111900]: 2024-03-18T15:01:01.645+0000 7fe1435f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: pgmap v192: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: osdmap e81: 6 total, 6 up, 6 in 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: Standby daemon mds.cephfs.smithi100.rutxmj assigned to filesystem cephfs as rank 0 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:01:02.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:01 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:replay} 2 up:standby 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: pgmap v192: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: osdmap e81: 6 total, 6 up, 6 in 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: Standby daemon mds.cephfs.smithi100.rutxmj assigned to filesystem cephfs as rank 0 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:01:02.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:01 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:replay} 2 up:standby 2024-03-18T15:01:03.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:02 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:03.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:02 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:04.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:03 smithi012 ceph-mon[111904]: pgmap v194: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-18T15:01:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:03 smithi100 ceph-mon[84868]: pgmap v194: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-18T15:01:06.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:05 smithi012 ceph-mon[111904]: pgmap v195: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-18T15:01:06.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:05 smithi100 ceph-mon[84868]: pgmap v195: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-03-18T15:01:08.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:07 smithi012 ceph-mon[111904]: pgmap v196: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-03-18T15:01:08.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:07 smithi100 ceph-mon[84868]: pgmap v196: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-03-18T15:01:10.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:09 smithi012 ceph-mon[111904]: pgmap v197: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:01:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:09 smithi100 ceph-mon[84868]: pgmap v197: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:01:11.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:10 smithi100 ceph-mon[84868]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0.000999973 2024-03-18T15:01:11.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:10 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:reconnect 2024-03-18T15:01:11.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:10 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:reconnect} 2 up:standby 2024-03-18T15:01:11.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:10 smithi100 ceph-mon[84868]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0.00499986 2024-03-18T15:01:11.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:10 smithi012 ceph-mon[111904]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0.000999973 2024-03-18T15:01:11.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:10 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:reconnect 2024-03-18T15:01:11.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:10 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:reconnect} 2 up:standby 2024-03-18T15:01:11.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:10 smithi012 ceph-mon[111904]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0.00499986 2024-03-18T15:01:12.108 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:11 smithi100 ceph-mon[84868]: pgmap v198: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-18T15:01:12.108 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:11 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:rejoin 2024-03-18T15:01:12.108 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:11 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:rejoin} 2 up:standby 2024-03-18T15:01:12.108 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:11 smithi100 ceph-mon[84868]: daemon mds.cephfs.smithi100.rutxmj is now active in filesystem cephfs as rank 0 2024-03-18T15:01:12.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:11 smithi012 ceph-mon[111904]: pgmap v198: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-18T15:01:12.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:11 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:rejoin 2024-03-18T15:01:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:11 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi100.rutxmj=up:rejoin} 2 up:standby 2024-03-18T15:01:12.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:11 smithi012 ceph-mon[111904]: daemon mds.cephfs.smithi100.rutxmj is now active in filesystem cephfs as rank 0 2024-03-18T15:01:13.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:active 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi100.rutxmj=up:active} 2 up:standby 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:13.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:12 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:13.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6826/2597653923,v1:172.21.15.100:6827/2597653923] up:active 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi100.rutxmj=up:active} 2 up:standby 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:13.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:12 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:14.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:13 smithi012 ceph-mon[111904]: pgmap v199: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 97 B/s wr, 5 op/s 2024-03-18T15:01:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:13 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6824/3815597664,v1:172.21.15.100:6825/3815597664] up:boot 2024-03-18T15:01:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:13 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi100.rutxmj=up:active} 3 up:standby 2024-03-18T15:01:14.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:13 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:14.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:13 smithi100 ceph-mon[84868]: pgmap v199: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 97 B/s wr, 5 op/s 2024-03-18T15:01:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:14 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6824/3815597664,v1:172.21.15.100:6825/3815597664] up:boot 2024-03-18T15:01:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:14 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi100.rutxmj=up:active} 3 up:standby 2024-03-18T15:01:14.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:14 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:16.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:15 smithi012 ceph-mon[111904]: pgmap v200: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 5 op/s 2024-03-18T15:01:16.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:15 smithi100 ceph-mon[84868]: pgmap v200: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 85 B/s wr, 5 op/s 2024-03-18T15:01:17.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:17.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:18.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:17 smithi100 ceph-mon[84868]: pgmap v201: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-18T15:01:18.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:17 smithi012 ceph-mon[111904]: pgmap v201: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-18T15:01:20.293 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:19 smithi012 ceph-mon[111904]: pgmap v202: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-18T15:01:20.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:20.294 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:19 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:20.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:19 smithi100 ceph-mon[84868]: pgmap v202: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-18T15:01:20.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:20.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:19 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:21.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:21 smithi012 ceph-mon[111904]: pgmap v203: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-18T15:01:21.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:21.492 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:21 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:21 smithi100 ceph-mon[84868]: pgmap v203: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-18T15:01:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:21.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:21 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:23.446 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:23 smithi100 ceph-mon[84868]: pgmap v204: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:23.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:23 smithi012 ceph-mon[111904]: pgmap v204: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:24.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:24 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:24.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:24 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:25.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:25 smithi100 ceph-mon[84868]: pgmap v205: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-18T15:01:25.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:25.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:01:25.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:25 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:25.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:25 smithi012 ceph-mon[111904]: pgmap v205: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 14 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-18T15:01:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi100.rutxmj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:01:25.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:25 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:26.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:26 smithi012 ceph-mon[111904]: Upgrade: Updating mds.cephfs.smithi100.rutxmj 2024-03-18T15:01:26.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:26 smithi012 ceph-mon[111904]: Deploying daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T15:01:26.778 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:26 smithi100 ceph-mon[84868]: Upgrade: Updating mds.cephfs.smithi100.rutxmj 2024-03-18T15:01:26.778 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:26 smithi100 ceph-mon[84868]: Deploying daemon mds.cephfs.smithi100.rutxmj on smithi100 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[111900]: 2024-03-18T15:01:27.307+0000 7fe1435f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: pgmap v206: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: osdmap e82: 6 total, 6 up, 6 in 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: Standby daemon mds.cephfs.smithi012.xtadxo assigned to filesystem cephfs as rank 0 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:01:27.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:01:27.743 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:27 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:replay} 2 up:standby 2024-03-18T15:01:27.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:27.331+0000 7fb5c1789700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2024-03-18T15:01:27.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:27.331+0000 7fb5c3f8e700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-18T15:01:27.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:27.333+0000 7fb5c478f700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 12 2024-03-18T15:01:27.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:27.335+0000 7fb5c5f92700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 19 2024-03-18T15:01:27.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:27.335+0000 7fb5c2f8c700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 22 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: pgmap v206: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: osdmap e82: 6 total, 6 up, 6 in 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: Standby daemon mds.cephfs.smithi012.xtadxo assigned to filesystem cephfs as rank 0 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-18T15:01:27.796 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:27 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:replay} 2 up:standby 2024-03-18T15:01:29.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.191+0000 7f7a84209700 1 -- 172.21.15.12:0/39858326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fded0 msgr2=0x7f7a7c0fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.191+0000 7f7a84209700 1 --2- 172.21.15.12:0/39858326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c0fe2a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a78004940 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 -- 172.21.15.12:0/39858326 shutdown_connections 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 --2- 172.21.15.12:0/39858326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 --2- 172.21.15.12:0/39858326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c0fe2a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 -- 172.21.15.12:0/39858326 >> 172.21.15.12:0/39858326 conn(0x7f7a7c072bc0 msgr2=0x7f7a7c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 -- 172.21.15.12:0/39858326 shutdown_connections 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 -- 172.21.15.12:0/39858326 wait complete. 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 Processor -- start 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 -- start start 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.192+0000 7f7a84209700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a84209700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a84209700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c1146f0 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a84209700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c10daa0 con 0x7f7a7c0fded0 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33028/0 (socket says 172.21.15.12:33028) 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 -- 172.21.15.12:0/3181192218 learned_addr learned my addr 172.21.15.12:0/3181192218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a81fa5700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 -- 172.21.15.12:0/3181192218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fded0 msgr2=0x7f7a7c113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 -- 172.21.15.12:0/3181192218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78005040 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a817a4700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00b790 tx=0x7f7a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.193+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c00f770 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.194+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a6c00fda0 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.194+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c10dd80 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.194+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a6c00d310 con 0x7f7a7c0fe870 2024-03-18T15:01:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.194+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c10e2c0 con 0x7f7a7c0fe870 2024-03-18T15:01:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.195+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c065df0 con 0x7f7a7c0fe870 2024-03-18T15:01:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.195+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7a6c010420 con 0x7f7a7c0fe870 2024-03-18T15:01:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.196+0000 7f7a727fc700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 0x7f7a680703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.196+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a6c08d7f0 con 0x7f7a7c0fe870 2024-03-18T15:01:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.196+0000 7f7a81fa5700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 0x7f7a680703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.197+0000 7f7a81fa5700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 0x7f7a680703a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009fd0 tx=0x7f7a78009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.205+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a6c05a810 con 0x7f7a7c0fe870 2024-03-18T15:01:29.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.434+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a7c109f10 con 0x7f7a6806def0 2024-03-18T15:01:29.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.436+0000 7f7a727fc700 1 -- 172.21.15.12:0/3181192218 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7a7c109f10 con 0x7f7a6806def0 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 msgr2=0x7f7a680703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 0x7f7a680703a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009fd0 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 msgr2=0x7f7a7c114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00b790 tx=0x7f7a6c009250 comp rx=0 tx=0).stop 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 shutdown_connections 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7a6806def0 0x7f7a680703a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c0fded0 0x7f7a7c113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 --2- 172.21.15.12:0/3181192218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a7c0fe870 0x7f7a7c114010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 >> 172.21.15.12:0/3181192218 conn(0x7f7a7c072bc0 msgr2=0x7f7a7c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 shutdown_connections 2024-03-18T15:01:29.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.438+0000 7f7a84209700 1 -- 172.21.15.12:0/3181192218 wait complete. 2024-03-18T15:01:29.449 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.543+0000 7ffae0295700 1 -- 172.21.15.12:0/4190223259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 msgr2=0x7ffad81092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.543+0000 7ffae0295700 1 --2- 172.21.15.12:0/4190223259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad81092f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffac8004940 tx=0x7ffac8009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- 172.21.15.12:0/4190223259 shutdown_connections 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 --2- 172.21.15.12:0/4190223259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad81092f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 --2- 172.21.15.12:0/4190223259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 0x7ffad80fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- 172.21.15.12:0/4190223259 >> 172.21.15.12:0/4190223259 conn(0x7ffad8072bc0 msgr2=0x7ffad8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- 172.21.15.12:0/4190223259 shutdown_connections 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- 172.21.15.12:0/4190223259 wait complete. 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 Processor -- start 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- start start 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 0x7ffad8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8116880 con 0x7ffad80fc590 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.544+0000 7ffae0295700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad810fc50 con 0x7ffad80fbc80 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33042/0 (socket says 172.21.15.12:33042) 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 -- 172.21.15.12:0/2950983174 learned_addr learned my addr 172.21.15.12:0/2950983174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffade031700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 0x7ffad8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 -- 172.21.15.12:0/2950983174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 msgr2=0x7ffad8115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 0x7ffad8115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 -- 172.21.15.12:0/2950983174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8005040 con 0x7ffad80fc590 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffadd830700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009b30 tx=0x7ffac8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac8021070 con 0x7ffad80fc590 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffac8013c00 con 0x7ffad80fc590 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad810fed0 con 0x7ffad80fc590 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac800e070 con 0x7ffad80fc590 2024-03-18T15:01:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.545+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81103b0 con 0x7ffad80fc590 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.547+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8065e30 con 0x7ffad80fc590 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.548+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ffac800b790 con 0x7ffad80fc590 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.548+0000 7ffaceffd700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 0x7ffac4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.548+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffac808ebd0 con 0x7ffad80fc590 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.548+0000 7ffade031700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 0x7ffac4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.549+0000 7ffade031700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 0x7ffac4070390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009b80 tx=0x7ffad400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.553+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffac805bbf0 con 0x7ffad80fc590 2024-03-18T15:01:29.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:29 smithi012 ceph-mon[111904]: pgmap v208: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s wr, 2 op/s 2024-03-18T15:01:29.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.779+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffad810c1f0 con 0x7ffac406dee0 2024-03-18T15:01:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.781+0000 7ffaceffd700 1 -- 172.21.15.12:0/2950983174 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffad810c1f0 con 0x7ffac406dee0 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 msgr2=0x7ffac4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 0x7ffac4070390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009b80 tx=0x7ffad400b040 comp rx=0 tx=0).stop 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 msgr2=0x7ffad8116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009b30 tx=0x7ffac8009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 shutdown_connections 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffac406dee0 0x7ffac4070390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80fbc80 0x7ffad8115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 --2- 172.21.15.12:0/2950983174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad80fc590 0x7ffad8116230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 >> 172.21.15.12:0/2950983174 conn(0x7ffad8072bc0 msgr2=0x7ffad8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 shutdown_connections 2024-03-18T15:01:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.783+0000 7ffae0295700 1 -- 172.21.15.12:0/2950983174 wait complete. 2024-03-18T15:01:29.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:29 smithi100 ceph-mon[84868]: pgmap v208: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s wr, 2 op/s 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 -- 172.21.15.12:0/1366606680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 msgr2=0x7f2d500fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 --2- 172.21.15.12:0/1366606680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d500fe1e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d40004940 tx=0x7f2d40009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 -- 172.21.15.12:0/1366606680 shutdown_connections 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 --2- 172.21.15.12:0/1366606680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 0x7f2d50106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 --2- 172.21.15.12:0/1366606680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d500fe1e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 -- 172.21.15.12:0/1366606680 >> 172.21.15.12:0/1366606680 conn(0x7f2d500f7880 msgr2=0x7f2d500f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 -- 172.21.15.12:0/1366606680 shutdown_connections 2024-03-18T15:01:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.888+0000 7f2d55037700 1 -- 172.21.15.12:0/1366606680 wait complete. 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 Processor -- start 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 -- start start 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 0x7f2d50114330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50114980 con 0x7f2d500fde10 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d55037700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5010de50 con 0x7f2d500fe7b0 2024-03-18T15:01:29.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d4ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d4ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33064/0 (socket says 172.21.15.12:33064) 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d4ed9d700 1 -- 172.21.15.12:0/2623479287 learned_addr learned my addr 172.21.15.12:0/2623479287 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.889+0000 7f2d4e59c700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 0x7f2d50114330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d4ed9d700 1 -- 172.21.15.12:0/2623479287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 msgr2=0x7f2d50114330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d4ed9d700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 0x7f2d50114330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d4ed9d700 1 -- 172.21.15.12:0/2623479287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40005040 con 0x7f2d500fde10 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d4ed9d700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009380 tx=0x7f2d40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4000e040 con 0x7f2d500fde10 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d40009af0 con 0x7f2d500fde10 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5010e0d0 con 0x7f2d500fde10 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5010e610 con 0x7f2d500fde10 2024-03-18T15:01:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.890+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4001c3f0 con 0x7f2d500fde10 2024-03-18T15:01:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.891+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c0052b0 con 0x7f2d500fde10 2024-03-18T15:01:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.892+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2d40013420 con 0x7f2d500fde10 2024-03-18T15:01:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.892+0000 7f2d377fe700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 0x7f2d38070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.892+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2d4008e9b0 con 0x7f2d500fde10 2024-03-18T15:01:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.892+0000 7f2d4e59c700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 0x7f2d38070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.893+0000 7f2d4e59c700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 0x7f2d38070390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d501027b0 tx=0x7f2d44009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:29.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:29.901+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d40057ea0 con 0x7f2d500fde10 2024-03-18T15:01:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.120+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d3c000bc0 con 0x7f2d3806dee0 2024-03-18T15:01:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.131+0000 7f2d377fe700 1 -- 172.21.15.12:0/2623479287 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2d3c000bc0 con 0x7f2d3806dee0 2024-03-18T15:01:30.132 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:01:30.132 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (7m) 36s ago 18m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T15:01:30.132 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (4m) 36s ago 18m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (4m) 10s ago 17m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (7m) 36s ago 18m 85.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (42s) 36s ago 12m 12.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 89022f9c2ba2 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (61s) 36s ago 12m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 269c33e068f3 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (18s) 10s ago 12m 11.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 18a5a74d9437 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (12m) 10s ago 12m 80.5M - 17.2.7-684-g6716c9f3 19857db3290c a675e4f507ef 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (8m) 36s ago 19m 509M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (8m) 10s ago 17m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (5m) 36s ago 20m 57.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (5m) 10s ago 17m 44.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (7m) 36s ago 18m 15.7M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (7m) 10s ago 17m 15.4M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T15:01:30.133 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (4m) 36s ago 16m 110M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (3m) 36s ago 16m 120M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (3m) 36s ago 15m 104M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (2m) 10s ago 15m 124M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (117s) 10s ago 14m 106M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (84s) 10s ago 14m 108M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9978289a0827 2024-03-18T15:01:30.134 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (7m) 36s ago 17m 66.4M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T15:01:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 msgr2=0x7f2d38070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 0x7f2d38070390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d501027b0 tx=0x7f2d44009040 comp rx=0 tx=0).stop 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 msgr2=0x7f2d50113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009380 tx=0x7f2d40009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 shutdown_connections 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2d3806dee0 0x7f2d38070390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d500fde10 0x7f2d50113df0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 --2- 172.21.15.12:0/2623479287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500fe7b0 0x7f2d50114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 >> 172.21.15.12:0/2623479287 conn(0x7f2d500f7880 msgr2=0x7f2d500f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 shutdown_connections 2024-03-18T15:01:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.135+0000 7f2d55037700 1 -- 172.21.15.12:0/2623479287 wait complete. 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 -- 172.21.15.12:0/3479041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c4102e00 msgr2=0x7f68c41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 --2- 172.21.15.12:0/3479041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c4102e00 0x7f68c41031d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f68b4004940 tx=0x7f68b4009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 -- 172.21.15.12:0/3479041566 shutdown_connections 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 --2- 172.21.15.12:0/3479041566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c41037a0 0x7f68c40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 --2- 172.21.15.12:0/3479041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c4102e00 0x7f68c41031d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 -- 172.21.15.12:0/3479041566 >> 172.21.15.12:0/3479041566 conn(0x7f68c40f89b0 msgr2=0x7f68c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 -- 172.21.15.12:0/3479041566 shutdown_connections 2024-03-18T15:01:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.259+0000 7f68ca65d700 1 -- 172.21.15.12:0/3479041566 wait complete. 2024-03-18T15:01:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 Processor -- start 2024-03-18T15:01:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 -- start start 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c4102e00 0x7f68c4113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41147c0 con 0x7f68c41037a0 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68ca65d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c410db40 con 0x7f68c4102e00 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68c37fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33092/0 (socket says 172.21.15.12:33092) 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.260+0000 7f68c37fe700 1 -- 172.21.15.12:0/2449263066 learned_addr learned my addr 172.21.15.12:0/2449263066 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c3fff700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c4102e00 0x7f68c4113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c37fe700 1 -- 172.21.15.12:0/2449263066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c4102e00 msgr2=0x7f68c4113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c37fe700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c4102e00 0x7f68c4113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c37fe700 1 -- 172.21.15.12:0/2449263066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4005040 con 0x7f68c41037a0 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c37fe700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b790 tx=0x7f68b8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b800f6f0 con 0x7f68c41037a0 2024-03-18T15:01:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b800fd20 con 0x7f68c41037a0 2024-03-18T15:01:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b80103a0 con 0x7f68c41037a0 2024-03-18T15:01:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c410ddf0 con 0x7f68c41037a0 2024-03-18T15:01:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.261+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c410e2e0 con 0x7f68c41037a0 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.262+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c40fe550 con 0x7f68c41037a0 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.263+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f68b8011400 con 0x7f68c41037a0 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.263+0000 7f68c0ff9700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 0x7f68ac070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.263+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f68b808d550 con 0x7f68c41037a0 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.264+0000 7f68c3fff700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 0x7f68ac070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.264+0000 7f68c3fff700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 0x7f68ac070390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f68b4004f90 tx=0x7f68b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:30.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.268+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68b8056a40 con 0x7f68c41037a0 2024-03-18T15:01:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.568+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68c4109f70 con 0x7f68c41037a0 2024-03-18T15:01:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.569+0000 7f68c0ff9700 1 -- 172.21.15.12:0/2449263066 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f68b805a100 con 0x7f68c41037a0 2024-03-18T15:01:30.569 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:01:30.570 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T15:01:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 13 2024-03-18T15:01:30.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T15:01:30.571 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 msgr2=0x7f68ac070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 0x7f68ac070390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f68b4004f90 tx=0x7f68b4009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 msgr2=0x7f68c41140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b790 tx=0x7f68b8009220 comp rx=0 tx=0).stop 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 shutdown_connections 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f68ac06dee0 0x7f68ac070390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c4102e00 0x7f68c4113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 --2- 172.21.15.12:0/2449263066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68c41037a0 0x7f68c41140e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 >> 172.21.15.12:0/2449263066 conn(0x7f68c40f89b0 msgr2=0x7f68c40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 shutdown_connections 2024-03-18T15:01:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.572+0000 7f68ca65d700 1 -- 172.21.15.12:0/2449263066 wait complete. 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 -- 172.21.15.12:0/433341531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 msgr2=0x7fb398102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/433341531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398102140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb394004940 tx=0x7fb394009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 -- 172.21.15.12:0/433341531 shutdown_connections 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/433341531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 0x7fb3980fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/433341531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398102140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 -- 172.21.15.12:0/433341531 >> 172.21.15.12:0/433341531 conn(0x7fb398072bc0 msgr2=0x7fb398072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 -- 172.21.15.12:0/433341531 shutdown_connections 2024-03-18T15:01:30.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.680+0000 7fb3a05bb700 1 -- 172.21.15.12:0/433341531 wait complete. 2024-03-18T15:01:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 Processor -- start 2024-03-18T15:01:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 -- start start 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 0x7fb398112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981135d0 con 0x7fb398101d70 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb3a05bb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39810c9c0 con 0x7fb398102710 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb39e357700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb39e357700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33120/0 (socket says 172.21.15.12:33120) 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.681+0000 7fb39e357700 1 -- 172.21.15.12:0/1327385525 learned_addr learned my addr 172.21.15.12:0/1327385525 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb39db56700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 0x7fb398112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb39e357700 1 -- 172.21.15.12:0/1327385525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 msgr2=0x7fb398112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb39e357700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 0x7fb398112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb39e357700 1 -- 172.21.15.12:0/1327385525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394005040 con 0x7fb398101d70 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb39e357700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b3b0 tx=0x7fb394009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39400e040 con 0x7fb398101d70 2024-03-18T15:01:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb39400bbc0 con 0x7fb398101d70 2024-03-18T15:01:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39401c3f0 con 0x7fb398101d70 2024-03-18T15:01:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39810cc40 con 0x7fb398101d70 2024-03-18T15:01:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.682+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39810d1b0 con 0x7fb398101d70 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.683+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3980ff080 con 0x7fb398101d70 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.685+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb39400bd30 con 0x7fb398101d70 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.685+0000 7fb38effd700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 0x7fb384070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.685+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb39408e7e0 con 0x7fb398101d70 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.685+0000 7fb39db56700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 0x7fb384070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.686+0000 7fb39db56700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 0x7fb384070390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb39810db80 tx=0x7fb38800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.689+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb39405b800 con 0x7fb398101d70 2024-03-18T15:01:30.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:30 smithi100 ceph-mon[84868]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:30.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:30 smithi100 ceph-mon[84868]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:30.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.990+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb39810e090 con 0x7fb398101d70 2024-03-18T15:01:30.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:30 smithi012 ceph-mon[111904]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:30.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:30 smithi012 ceph-mon[111904]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.990+0000 7fb38effd700 1 -- 172.21.15.12:0/1327385525 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1749 (secure 0 0 0) 0x7fb39405b390 con 0x7fb398101d70 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:e32 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 32 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-18T15:01:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T15:01:27.315552+0000 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 82 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T15:01:30.993 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=34406} 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 0 members: 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:34406} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:01:30.994 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T15:01:30.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:01:30.995 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.12:6828/846568565,v1:172.21.15.12:6829/846568565] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:01:30.995 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6824/3815597664,v1:172.21.15.100:6825/3815597664] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:01:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 msgr2=0x7fb384070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 0x7fb384070390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb39810db80 tx=0x7fb38800b040 comp rx=0 tx=0).stop 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 msgr2=0x7fb398112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b3b0 tx=0x7fb394009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 shutdown_connections 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb38406dee0 0x7fb384070390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb398101d70 0x7fb398112a40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 --2- 172.21.15.12:0/1327385525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb398102710 0x7fb398112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 >> 172.21.15.12:0/1327385525 conn(0x7fb398072bc0 msgr2=0x7fb3980fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 shutdown_connections 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:30.993+0000 7fb3a05bb700 1 -- 172.21.15.12:0/1327385525 wait complete. 2024-03-18T15:01:30.996 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 -- 172.21.15.12:0/3684866847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 msgr2=0x7fe3a81092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3684866847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a81092f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe398004940 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 -- 172.21.15.12:0/3684866847 shutdown_connections 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3684866847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a81092f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3684866847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 0x7fe3a80fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 -- 172.21.15.12:0/3684866847 >> 172.21.15.12:0/3684866847 conn(0x7fe3a8072bc0 msgr2=0x7fe3a8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 -- 172.21.15.12:0/3684866847 shutdown_connections 2024-03-18T15:01:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.102+0000 7fe3b0112700 1 -- 172.21.15.12:0/3684866847 wait complete. 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 Processor -- start 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 -- start start 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 0x7fe3a8115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8116860 con 0x7fe3a80fc590 2024-03-18T15:01:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3b0112700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a810fc30 con 0x7fe3a80fbc80 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33142/0 (socket says 172.21.15.12:33142) 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 -- 172.21.15.12:0/3949547507 learned_addr learned my addr 172.21.15.12:0/3949547507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3adeae700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 0x7fe3a8115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 -- 172.21.15.12:0/3949547507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 msgr2=0x7fe3a8115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 0x7fe3a8115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.103+0000 7fe3ad6ad700 1 -- 172.21.15.12:0/3949547507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398005040 con 0x7fe3a80fc590 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe3ad6ad700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe398009b30 tx=0x7fe398009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe398021070 con 0x7fe3a80fc590 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a810feb0 con 0x7fe3a80fc590 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe398013c00 con 0x7fe3a80fc590 2024-03-18T15:01:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe39800e070 con 0x7fe3a80fc590 2024-03-18T15:01:31.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.104+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a8110340 con 0x7fe3a80fc590 2024-03-18T15:01:31.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.105+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a8065e30 con 0x7fe3a80fc590 2024-03-18T15:01:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.106+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe39801cad0 con 0x7fe3a80fc590 2024-03-18T15:01:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.106+0000 7fe39e7fc700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 0x7fe394070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.107+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe39808ebe0 con 0x7fe3a80fc590 2024-03-18T15:01:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.107+0000 7fe3adeae700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 0x7fe394070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.107+0000 7fe3adeae700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 0x7fe394070390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009b80 tx=0x7fe3a400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:31.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.112+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe39805bc00 con 0x7fe3a80fc590 2024-03-18T15:01:31.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.356+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3a8065e30 con 0x7fe39406dee0 2024-03-18T15:01:31.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.358+0000 7fe39e7fc700 1 -- 172.21.15.12:0/3949547507 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe3a8065e30 con 0x7fe39406dee0 2024-03-18T15:01:31.358 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:01:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-18T15:01:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": true, 2024-03-18T15:01:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [ 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "osd", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "mon", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "crash" 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": "15/21 daemons upgraded", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "Currently upgrading mds daemons", 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T15:01:31.359 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:01:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 msgr2=0x7fe394070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 0x7fe394070390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009b80 tx=0x7fe3a400b040 comp rx=0 tx=0).stop 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 msgr2=0x7fe3a8116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe398009b30 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 shutdown_connections 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe39406dee0 0x7fe394070390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a80fbc80 0x7fe3a8115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 --2- 172.21.15.12:0/3949547507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3a80fc590 0x7fe3a8116210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 >> 172.21.15.12:0/3949547507 conn(0x7fe3a8072bc0 msgr2=0x7fe3a8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 shutdown_connections 2024-03-18T15:01:31.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.360+0000 7fe3b0112700 1 -- 172.21.15.12:0/3949547507 wait complete. 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 -- 172.21.15.12:0/1621944973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070300 msgr2=0x7fd6080706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/1621944973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070300 0x7fd6080706d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8004970 tx=0x7fd5f8009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 -- 172.21.15.12:0/1621944973 shutdown_connections 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/1621944973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070c10 0x7fd608109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/1621944973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070300 0x7fd6080706d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.465+0000 7fd60ccbb700 1 -- 172.21.15.12:0/1621944973 >> 172.21.15.12:0/1621944973 conn(0x7fd6080f8980 msgr2=0x7fd6080fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 -- 172.21.15.12:0/1621944973 shutdown_connections 2024-03-18T15:01:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 -- 172.21.15.12:0/1621944973 wait complete. 2024-03-18T15:01:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 Processor -- start 2024-03-18T15:01:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 -- start start 2024-03-18T15:01:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070300 0x7fd608115e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd608116a80 con 0x7fd608070c10 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd60ccbb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60810fdc0 con 0x7fd608070300 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd605d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd605d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33158/0 (socket says 172.21.15.12:33158) 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.466+0000 7fd605d9b700 1 -- 172.21.15.12:0/3942627970 learned_addr learned my addr 172.21.15.12:0/3942627970 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd60659c700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070300 0x7fd608115e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd605d9b700 1 -- 172.21.15.12:0/3942627970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070300 msgr2=0x7fd608115e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd605d9b700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070300 0x7fd608115e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd605d9b700 1 -- 172.21.15.12:0/3942627970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8005040 con 0x7fd608070c10 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd605d9b700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b790 tx=0x7fd5f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:31.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f000f6f0 con 0x7fd608070c10 2024-03-18T15:01:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f000fd20 con 0x7fd608070c10 2024-03-18T15:01:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd608110040 con 0x7fd608070c10 2024-03-18T15:01:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f00103a0 con 0x7fd608070c10 2024-03-18T15:01:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.467+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd608110520 con 0x7fd608070c10 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.468+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd608071a40 con 0x7fd608070c10 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.469+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd5f0011400 con 0x7fd608070c10 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.470+0000 7fd5feffd700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 0x7fd5f4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.470+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd5f008d440 con 0x7fd608070c10 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.470+0000 7fd60659c700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 0x7fd5f4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:01:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.470+0000 7fd60659c700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 0x7fd5f4070390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8004f50 tx=0x7fd5f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:01:31.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.474+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5f005a520 con 0x7fd608070c10 2024-03-18T15:01:31.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:31.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: pgmap v209: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-03-18T15:01:31.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2449263066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:31.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1327385525' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:01:31.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:31.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:31.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.797+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd60805f5f0 con 0x7fd608070c10 2024-03-18T15:01:31.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.797+0000 7fd5feffd700 1 -- 172.21.15.12:0/3942627970 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7fd5f005a0b0 con 0x7fd608070c10 2024-03-18T15:01:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_WARN 1 filesystem is degraded 2024-03-18T15:01:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-18T15:01:31.798 INFO:teuthology.orchestra.run.smithi012.stdout: fs cephfs is degraded 2024-03-18T15:01:31.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.799+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 msgr2=0x7fd5f4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.799+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 0x7fd5f4070390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8004f50 tx=0x7fd5f8009e30 comp rx=0 tx=0).stop 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 msgr2=0x7fd6081163a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b790 tx=0x7fd5f0009250 comp rx=0 tx=0).stop 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 shutdown_connections 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd5f406dee0 0x7fd5f4070390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd608070300 0x7fd608115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 --2- 172.21.15.12:0/3942627970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd608070c10 0x7fd6081163a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 >> 172.21.15.12:0/3942627970 conn(0x7fd6080f8980 msgr2=0x7fd6080f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 shutdown_connections 2024-03-18T15:01:31.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:01:31.800+0000 7fd60ccbb700 1 -- 172.21.15.12:0/3942627970 wait complete. 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: pgmap v209: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2449263066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1327385525' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:32.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:32.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:32 smithi012 ceph-mon[111904]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:32.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:32 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3942627970' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:01:33.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:32 smithi100 ceph-mon[84868]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:01:33.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:32 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3942627970' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:01:33.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:33 smithi012 ceph-mon[111904]: pgmap v210: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:01:34.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:33 smithi100 ceph-mon[84868]: pgmap v210: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-18T15:01:35.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:35 smithi012 ceph-mon[111904]: pgmap v211: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:01:36.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:35 smithi100 ceph-mon[84868]: pgmap v211: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:01:36.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:36 smithi012 ceph-mon[111904]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0 2024-03-18T15:01:36.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:36 smithi012 ceph-mon[111904]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0 2024-03-18T15:01:36.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:36 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:reconnect 2024-03-18T15:01:36.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:36 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:reconnect} 2 up:standby 2024-03-18T15:01:37.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:36 smithi100 ceph-mon[84868]: reconnect by client.24303 v1:172.21.15.12:0/3414348782 after 0 2024-03-18T15:01:37.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:36 smithi100 ceph-mon[84868]: reconnect by client.14544 v1:172.21.15.100:0/2705169264 after 0 2024-03-18T15:01:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:36 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:reconnect 2024-03-18T15:01:37.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:36 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:reconnect} 2 up:standby 2024-03-18T15:01:37.791 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:37 smithi100 ceph-mon[84868]: pgmap v212: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-03-18T15:01:37.791 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:37 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:rejoin 2024-03-18T15:01:37.792 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:37 smithi100 ceph-mon[84868]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:rejoin} 2 up:standby 2024-03-18T15:01:37.792 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:37 smithi100 ceph-mon[84868]: daemon mds.cephfs.smithi012.xtadxo is now active in filesystem cephfs as rank 0 2024-03-18T15:01:37.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:37 smithi012 ceph-mon[111904]: pgmap v212: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-03-18T15:01:37.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:37 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:rejoin 2024-03-18T15:01:37.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:37 smithi012 ceph-mon[111904]: fsmap cephfs:1/1 {0=cephfs.smithi012.xtadxo=up:rejoin} 2 up:standby 2024-03-18T15:01:37.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:37 smithi012 ceph-mon[111904]: daemon mds.cephfs.smithi012.xtadxo is now active in filesystem cephfs as rank 0 2024-03-18T15:01:38.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:38 smithi100 ceph-mon[84868]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:01:38.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:38 smithi100 ceph-mon[84868]: Cluster is now healthy 2024-03-18T15:01:38.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:38 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:active 2024-03-18T15:01:38.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:38 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T15:01:38.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:38 smithi012 ceph-mon[111904]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:01:38.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:38 smithi012 ceph-mon[111904]: Cluster is now healthy 2024-03-18T15:01:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:38 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] up:active 2024-03-18T15:01:38.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:38 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 2 up:standby 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: pgmap v213: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: mds.? [v2:172.21.15.100:6826/3606275918,v1:172.21.15.100:6827/3606275918] up:boot 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:39.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:39 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T15:01:39.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: pgmap v213: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-18T15:01:39.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: mds.? [v2:172.21.15.100:6826/3606275918,v1:172.21.15.100:6827/3606275918] up:boot 2024-03-18T15:01:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:39.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:39 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T15:01:42.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:41 smithi012 ceph-mon[111904]: pgmap v214: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-03-18T15:01:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:42.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:41 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:42.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:41 smithi100 ceph-mon[84868]: pgmap v214: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-03-18T15:01:42.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:42.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:41 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:43.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:42 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:43.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:42 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:43.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:42 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:43.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:42 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:44.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:43 smithi012 ceph-mon[111904]: pgmap v215: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-03-18T15:01:44.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:43 smithi100 ceph-mon[84868]: pgmap v215: 65 pgs: 65 active+clean; 193 MiB data, 816 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-03-18T15:01:46.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: pgmap v216: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.oheuui"}]': finished 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.xtadxo"}]': finished 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.dvtbrf"}]': finished 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.rutxmj"}]': finished 2024-03-18T15:01:46.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:45 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: pgmap v216: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.oheuui"}]: dispatch 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.oheuui"}]': finished 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.xtadxo"}]: dispatch 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi012.xtadxo"}]': finished 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.dvtbrf"}]: dispatch 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.dvtbrf"}]': finished 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.rutxmj"}]: dispatch 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi100.rutxmj"}]': finished 2024-03-18T15:01:46.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:45 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-18T15:01:47.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all mds 2024-03-18T15:01:47.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: Upgrade: Setting filesystem cephfs Joinable 2024-03-18T15:01:47.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-18T15:01:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:47.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-18T15:01:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all mds 2024-03-18T15:01:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: Upgrade: Setting filesystem cephfs Joinable 2024-03-18T15:01:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:01:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-18T15:01:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-18T15:01:47.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:47.542+0000 7fb5c478f700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2024-03-18T15:01:47.794 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:47.543+0000 7fb5c1789700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-18T15:01:47.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:47.543+0000 7fb5c6f94700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-03-18T15:01:47.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:47.544+0000 7fb5c7f96700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 10 2024-03-18T15:01:47.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:01:47.545+0000 7fb5c5f92700 -1 bdev(0x5602b88f2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 14 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: pgmap v217: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.243 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-18T15:01:48.244 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.245 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:48.246 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: pgmap v217: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 3 up:standby 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: fsmap cephfs:1 {0=cephfs.smithi012.xtadxo=up:active} 1 up:standby-replay 2 up:standby 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-18T15:01:48.296 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-18T15:01:48.297 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-18T15:01:48.298 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:01:48.299 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:49.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all rgw 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all rbd-mirror 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all ceph-exporter 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all iscsi 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all nfs 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Setting container_image for all nvmeof 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Finalizing container_image settings 2024-03-18T15:01:49.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:48 smithi012 ceph-mon[111904]: Upgrade: Complete! 2024-03-18T15:01:49.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all rgw 2024-03-18T15:01:49.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all rbd-mirror 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all ceph-exporter 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all iscsi 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all nfs 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Setting container_image for all nvmeof 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Finalizing container_image settings 2024-03-18T15:01:49.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:48 smithi100 ceph-mon[84868]: Upgrade: Complete! 2024-03-18T15:01:50.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:49 smithi012 ceph-mon[111904]: pgmap v218: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 347 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:50.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:49 smithi100 ceph-mon[84868]: pgmap v218: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 347 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:52.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:51 smithi012 ceph-mon[111904]: pgmap v219: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:52.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:51 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:52.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:51 smithi100 ceph-mon[84868]: pgmap v219: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-18T15:01:52.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:51 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:01:54.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:53 smithi012 ceph-mon[111904]: pgmap v220: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 9.1 MiB/s rd, 83 B/s wr, 4 op/s 2024-03-18T15:01:54.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:53 smithi100 ceph-mon[84868]: pgmap v220: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 9.1 MiB/s rd, 83 B/s wr, 4 op/s 2024-03-18T15:01:56.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:55 smithi012 ceph-mon[111904]: pgmap v221: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-03-18T15:01:56.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:55 smithi100 ceph-mon[84868]: pgmap v221: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-03-18T15:01:58.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:57 smithi012 ceph-mon[111904]: pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:01:58.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:57 smithi100 ceph-mon[84868]: pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:02:00.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:01:59 smithi012 ceph-mon[111904]: pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:02:00.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:01:59 smithi100 ceph-mon[84868]: pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-18T15:02:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 -- 172.21.15.12:0/627939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 msgr2=0x7f38ac1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 --2- 172.21.15.12:0/627939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac1053e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f389c009ce0 tx=0x7f389c004d10 comp rx=0 tx=0).stop 2024-03-18T15:02:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 -- 172.21.15.12:0/627939552 shutdown_connections 2024-03-18T15:02:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 --2- 172.21.15.12:0/627939552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 0x7f38ac0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:01.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 --2- 172.21.15.12:0/627939552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac1053e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:01.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 -- 172.21.15.12:0/627939552 >> 172.21.15.12:0/627939552 conn(0x7f38ac0faba0 msgr2=0x7f38ac0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:01.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 -- 172.21.15.12:0/627939552 shutdown_connections 2024-03-18T15:02:01.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.905+0000 7f38b384c700 1 -- 172.21.15.12:0/627939552 wait complete. 2024-03-18T15:02:01.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 Processor -- start 2024-03-18T15:02:01.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 -- start start 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 0x7f38ac116460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac116ab0 con 0x7f38ac105010 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b384c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10fe80 con 0x7f38ac1059b0 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b15e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b15e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35794/0 (socket says 172.21.15.12:35794) 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b15e8700 1 -- 172.21.15.12:0/2044214595 learned_addr learned my addr 172.21.15.12:0/2044214595 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.906+0000 7f38b0de7700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 0x7f38ac116460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b15e8700 1 -- 172.21.15.12:0/2044214595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 msgr2=0x7f38ac116460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b15e8700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 0x7f38ac116460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b15e8700 1 -- 172.21.15.12:0/2044214595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009940 con 0x7f38ac105010 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b15e8700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f389c003400 tx=0x7f389c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:01.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c00e040 con 0x7f38ac105010 2024-03-18T15:02:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f389c004610 con 0x7f38ac105010 2024-03-18T15:02:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac110100 con 0x7f38ac105010 2024-03-18T15:02:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c01c640 con 0x7f38ac105010 2024-03-18T15:02:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.907+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac1106c0 con 0x7f38ac105010 2024-03-18T15:02:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.908+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac100760 con 0x7f38ac105010 2024-03-18T15:02:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.909+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f389c012a70 con 0x7f38ac105010 2024-03-18T15:02:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.909+0000 7f38a9ffb700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 0x7f3894070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.909+0000 7f38b0de7700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 0x7f3894070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.909+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f389c08f0f0 con 0x7f38ac105010 2024-03-18T15:02:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.910+0000 7f38b0de7700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 0x7f3894070390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38ac1117c0 tx=0x7f38a000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:01.914+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f389c05c110 con 0x7f38ac105010 2024-03-18T15:02:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.141+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ac04bed0 con 0x7f389406dee0 2024-03-18T15:02:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.142+0000 7f38a9ffb700 1 -- 172.21.15.12:0/2044214595 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f38ac04bed0 con 0x7f389406dee0 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 msgr2=0x7f3894070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 0x7f3894070390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38ac1117c0 tx=0x7f38a000b040 comp rx=0 tx=0).stop 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 msgr2=0x7f38ac115f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f389c003400 tx=0x7f389c004d10 comp rx=0 tx=0).stop 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 shutdown_connections 2024-03-18T15:02:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f389406dee0 0x7f3894070390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38ac105010 0x7f38ac115f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 --2- 172.21.15.12:0/2044214595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1059b0 0x7f38ac116460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 >> 172.21.15.12:0/2044214595 conn(0x7f38ac0faba0 msgr2=0x7f38ac0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 shutdown_connections 2024-03-18T15:02:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:02.144+0000 7f38b384c700 1 -- 172.21.15.12:0/2044214595 wait complete. 2024-03-18T15:02:02.156 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:01 smithi012 ceph-mon[111904]: pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-18T15:02:02.156 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:02.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:01 smithi100 ceph-mon[84868]: pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-18T15:02:02.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:02.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-18T15:02:03.193 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:04.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:03 smithi012 ceph-mon[111904]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:04.255 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:03 smithi012 ceph-mon[111904]: pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-18T15:02:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:03 smithi100 ceph-mon[84868]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:04.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:03 smithi100 ceph-mon[84868]: pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 -- 172.21.15.12:0/1313891336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fe7c0 msgr2=0x7f3ed0106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/1313891336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0106e00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0004940 tx=0x7f3ec0009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 -- 172.21.15.12:0/1313891336 shutdown_connections 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/1313891336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0106e00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/1313891336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fde20 0x7f3ed00fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 -- 172.21.15.12:0/1313891336 >> 172.21.15.12:0/1313891336 conn(0x7f3ed00f7870 msgr2=0x7f3ed00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 -- 172.21.15.12:0/1313891336 shutdown_connections 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.800+0000 7f3ed86cf700 1 -- 172.21.15.12:0/1313891336 wait complete. 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 Processor -- start 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 -- start start 2024-03-18T15:02:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0114510 con 0x7f3ed00fde20 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed86cf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed010db10 con 0x7f3ed00fe7c0 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35806/0 (socket says 172.21.15.12:35806) 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 -- 172.21.15.12:0/3459882929 learned_addr learned my addr 172.21.15.12:0/3459882929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 -- 172.21.15.12:0/3459882929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fe7c0 msgr2=0x7f3ed0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed5c6a700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.801+0000 7f3ed646b700 1 -- 172.21.15.12:0/3459882929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0005040 con 0x7f3ed00fde20 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ed646b700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc002a50 tx=0x7f3ecc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc007cc0 con 0x7f3ed00fde20 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed010ddf0 con 0x7f3ed00fde20 2024-03-18T15:02:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ecc00f040 con 0x7f3ed00fde20 2024-03-18T15:02:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc009640 con 0x7f3ed00fde20 2024-03-18T15:02:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.802+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed010e330 con 0x7f3ed00fde20 2024-03-18T15:02:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.803+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0066b70 con 0x7f3ed00fde20 2024-03-18T15:02:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.803+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f3ecc0097a0 con 0x7f3ed00fde20 2024-03-18T15:02:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.804+0000 7f3ec6ffd700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 0x7f3ebc070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.804+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3ecc08d540 con 0x7f3ed00fde20 2024-03-18T15:02:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.804+0000 7f3ed5c6a700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 0x7f3ebc070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.804+0000 7f3ed5c6a700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 0x7f3ebc070390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b3b0 tx=0x7f3ec0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:04.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:04.809+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ecc05a4e0 con 0x7f3ed00fde20 2024-03-18T15:02:05.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.047+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ed0109eb0 con 0x7f3ebc06dee0 2024-03-18T15:02:05.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.061+0000 7f3ec6ffd700 1 -- 172.21.15.12:0/3459882929 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3ed0109eb0 con 0x7f3ebc06dee0 2024-03-18T15:02:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:02:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (7m) 71s ago 18m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T15:02:05.061 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (5m) 71s ago 19m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (5m) 24s ago 18m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (7m) 71s ago 18m 85.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (76s) 71s ago 13m 12.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 89022f9c2ba2 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (96s) 71s ago 13m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 269c33e068f3 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (53s) 24s ago 13m 12.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 18a5a74d9437 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (26s) 24s ago 13m 13.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a9412221f4bb 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (9m) 71s ago 20m 509M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (8m) 24s ago 17m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (6m) 71s ago 20m 57.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (6m) 24s ago 17m 44.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (8m) 71s ago 19m 15.7M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T15:02:05.062 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (8m) 24s ago 17m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (4m) 71s ago 17m 110M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (4m) 71s ago 16m 120M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (3m) 71s ago 16m 104M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (3m) 24s ago 15m 126M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (2m) 24s ago 15m 106M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (119s) 24s ago 14m 109M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9978289a0827 2024-03-18T15:02:05.063 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (8m) 71s ago 18m 66.4M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 msgr2=0x7f3ebc070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 0x7f3ebc070390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b3b0 tx=0x7f3ec0009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 msgr2=0x7f3ed0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc002a50 tx=0x7f3ecc0046b0 comp rx=0 tx=0).stop 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 shutdown_connections 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f3ebc06dee0 0x7f3ebc070390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ed00fde20 0x7f3ed0113980 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 --2- 172.21.15.12:0/3459882929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00fe7c0 0x7f3ed0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 >> 172.21.15.12:0/3459882929 conn(0x7f3ed00f7870 msgr2=0x7f3ed00f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:05.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 shutdown_connections 2024-03-18T15:02:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:05.063+0000 7f3ed86cf700 1 -- 172.21.15.12:0/3459882929 wait complete. 2024-03-18T15:02:05.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade status' 2024-03-18T15:02:06.057 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:06.181 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:05 smithi012 ceph-mon[111904]: pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 2 op/s 2024-03-18T15:02:06.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:05 smithi100 ceph-mon[84868]: pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s rd, 2 op/s 2024-03-18T15:02:07.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:07 smithi012 ceph-mon[111904]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:07.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:07 smithi100 ceph-mon[84868]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.842+0000 7f33ab1a1700 1 -- 172.21.15.12:0/2360041807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 msgr2=0x7f33a40fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.842+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/2360041807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a40fbfe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3398004940 tx=0x7f3398009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.842+0000 7f33ab1a1700 1 -- 172.21.15.12:0/2360041807 shutdown_connections 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/2360041807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a40fbfe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/2360041807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 0x7f33a4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- 172.21.15.12:0/2360041807 >> 172.21.15.12:0/2360041807 conn(0x7f33a4071b70 msgr2=0x7f33a4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- 172.21.15.12:0/2360041807 shutdown_connections 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- 172.21.15.12:0/2360041807 wait complete. 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 Processor -- start 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- start start 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 0x7f33a410ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410d210 con 0x7f33a40fbb70 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33ab1a1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410d380 con 0x7f33a4101ba0 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.843+0000 7f33aa19f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35822/0 (socket says 172.21.15.12:35822) 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 -- 172.21.15.12:0/1466186247 learned_addr learned my addr 172.21.15.12:0/1466186247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33a999e700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 0x7f33a410ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 -- 172.21.15.12:0/1466186247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 msgr2=0x7f33a410ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 0x7f33a410ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 -- 172.21.15.12:0/1466186247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398005040 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33aa19f700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c8c0 tx=0x7f339c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339c00a760 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a4119e00 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f339c00ad90 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339c0113a0 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.844+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a411a1d0 con 0x7f33a40fbb70 2024-03-18T15:02:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.845+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a40fd200 con 0x7f33a40fbb70 2024-03-18T15:02:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.847+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f339c012400 con 0x7f33a40fbb70 2024-03-18T15:02:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.847+0000 7f33a2ffd700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 0x7f338c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.847+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f339c08d780 con 0x7f33a40fbb70 2024-03-18T15:02:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.847+0000 7f33a999e700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 0x7f338c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.848+0000 7f33a999e700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 0x7f338c0703a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33a4066000 tx=0x7f3398009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:07.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:07.852+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f339c05a7a0 con 0x7f33a40fbb70 2024-03-18T15:02:08.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.080+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a404bed0 con 0x7f338c06def0 2024-03-18T15:02:08.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.081+0000 7f33a2ffd700 1 -- 172.21.15.12:0/1466186247 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f33a404bed0 con 0x7f338c06def0 2024-03-18T15:02:08.081 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:02:08.081 INFO:teuthology.orchestra.run.smithi012.stdout: "target_image": null, 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "in_progress": false, 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "which": "", 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "services_complete": [], 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "progress": null, 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "message": "", 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout: "is_paused": false 2024-03-18T15:02:08.082 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 msgr2=0x7f338c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 0x7f338c0703a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33a4066000 tx=0x7f3398009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 msgr2=0x7f33a410c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c8c0 tx=0x7f339c0044c0 comp rx=0 tx=0).stop 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 shutdown_connections 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f338c06def0 0x7f338c0703a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a40fbb70 0x7f33a410c790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 --2- 172.21.15.12:0/1466186247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a4101ba0 0x7f33a410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 >> 172.21.15.12:0/1466186247 conn(0x7f33a4071b70 msgr2=0x7f33a40fa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 shutdown_connections 2024-03-18T15:02:08.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:08.083+0000 7f33ab1a1700 1 -- 172.21.15.12:0/1466186247 wait complete. 2024-03-18T15:02:08.120 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:08 smithi012 ceph-mon[111904]: pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.8 MiB/s rd, 2 op/s 2024-03-18T15:02:08.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:08 smithi100 ceph-mon[84868]: pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.8 MiB/s rd, 2 op/s 2024-03-18T15:02:08.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph health detail' 2024-03-18T15:02:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:10.260 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:10 smithi012 ceph-mon[111904]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:10.261 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:10 smithi012 ceph-mon[111904]: pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 1 op/s 2024-03-18T15:02:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:10 smithi100 ceph-mon[84868]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:02:10.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:10 smithi100 ceph-mon[84868]: pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 1 op/s 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- 172.21.15.12:0/2033205850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08980702e0 msgr2=0x7f08980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 --2- 172.21.15.12:0/2033205850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08980702e0 0x7f08980706b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0888009ce0 tx=0x7f0888004d10 comp rx=0 tx=0).stop 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- 172.21.15.12:0/2033205850 shutdown_connections 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 --2- 172.21.15.12:0/2033205850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0898070bf0 0x7f0898109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 --2- 172.21.15.12:0/2033205850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08980702e0 0x7f08980706b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- 172.21.15.12:0/2033205850 >> 172.21.15.12:0/2033205850 conn(0x7f08980f89b0 msgr2=0x7f08980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- 172.21.15.12:0/2033205850 shutdown_connections 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- 172.21.15.12:0/2033205850 wait complete. 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 Processor -- start 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- start start 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 0x7f0898115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898116ae0 con 0x7f0898070bf0 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.796+0000 7f089fce1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089810feb0 con 0x7f08980702e0 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089da7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 0x7f0898115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089da7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 0x7f0898115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.12:50312/0 (socket says 172.21.15.12:50312) 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37884/0 (socket says 172.21.15.12:37884) 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089da7d700 1 -- 172.21.15.12:0/3912764979 learned_addr learned my addr 172.21.15.12:0/3912764979 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 -- 172.21.15.12:0/3912764979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 msgr2=0x7f0898115f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 0x7f0898115f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 -- 172.21.15.12:0/3912764979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0888009940 con 0x7f0898070bf0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089d27c700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c7e0 tx=0x7f088c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088c0106f0 con 0x7f0898070bf0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0898110190 con 0x7f0898070bf0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f088c010d20 con 0x7f0898070bf0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.797+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088c0113a0 con 0x7f0898070bf0 2024-03-18T15:02:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.798+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08981106d0 con 0x7f0898070bf0 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.798+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0898071a20 con 0x7f0898070bf0 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.799+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f088c012400 con 0x7f0898070bf0 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.799+0000 7f08967fc700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 0x7f0880070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.800+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f088c08d620 con 0x7f0898070bf0 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.800+0000 7f089da7d700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 0x7f0880070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:10.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.800+0000 7f089da7d700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 0x7f0880070390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f088800b740 tx=0x7f0888004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:10.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:10.804+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f088c05a5c0 con 0x7f0898070bf0 2024-03-18T15:02:11.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.129+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f089805f5f0 con 0x7f0898070bf0 2024-03-18T15:02:11.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.130+0000 7f08967fc700 1 -- 172.21.15.12:0/3912764979 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f088c05a150 con 0x7f0898070bf0 2024-03-18T15:02:11.131 INFO:teuthology.orchestra.run.smithi012.stdout:HEALTH_OK 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 msgr2=0x7f0880070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 0x7f0880070390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f088800b740 tx=0x7f0888004d10 comp rx=0 tx=0).stop 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 msgr2=0x7f0898116490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f088c00c7e0 tx=0x7f088c0043c0 comp rx=0 tx=0).stop 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 shutdown_connections 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f088006dee0 0x7f0880070390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08980702e0 0x7f0898115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 --2- 172.21.15.12:0/3912764979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898070bf0 0x7f0898116490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 >> 172.21.15.12:0/3912764979 conn(0x7f08980f89b0 msgr2=0x7f08980f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:11.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 shutdown_connections 2024-03-18T15:02:11.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:11.132+0000 7f089fce1700 1 -- 172.21.15.12:0/3912764979 wait complete. 2024-03-18T15:02:11.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-18T15:02:12.029 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:12.283 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:12 smithi012 ceph-mon[111904]: pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-03-18T15:02:12.284 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:12 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3912764979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:02:12.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:12 smithi100 ceph-mon[84868]: pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-03-18T15:02:12.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:12 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3912764979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-18T15:02:13.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/3512967844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f4102e10 msgr2=0x7fe3f41031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/3512967844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f4102e10 0x7fe3f41031e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4004940 tx=0x7fe3e4009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/3512967844 shutdown_connections 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/3512967844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f41037b0 0x7fe3f40fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/3512967844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f4102e10 0x7fe3f41031e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/3512967844 >> 172.21.15.12:0/3512967844 conn(0x7fe3f40f89a0 msgr2=0x7fe3f40fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/3512967844 shutdown_connections 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/3512967844 wait complete. 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 Processor -- start 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.668+0000 7fe3fc7d8700 1 -- start start 2024-03-18T15:02:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3fc7d8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f4102e10 0x7fe3f4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3fc7d8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3fc7d8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41147b0 con 0x7fe3f41037b0 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3fc7d8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f410db10 con 0x7fe3f4102e10 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37900/0 (socket says 172.21.15.12:37900) 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 -- 172.21.15.12:0/2346725666 learned_addr learned my addr 172.21.15.12:0/2346725666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3fa574700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f4102e10 0x7fe3f4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 -- 172.21.15.12:0/2346725666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f4102e10 msgr2=0x7fe3f4113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f4102e10 0x7fe3f4113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 -- 172.21.15.12:0/2346725666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4005040 con 0x7fe3f41037b0 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.669+0000 7fe3f9d73700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b790 tx=0x7fe3f0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:13.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.670+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f000f6f0 con 0x7fe3f41037b0 2024-03-18T15:02:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.670+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f410ddf0 con 0x7fe3f41037b0 2024-03-18T15:02:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.670+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3f000fd20 con 0x7fe3f41037b0 2024-03-18T15:02:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.670+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f00103a0 con 0x7fe3f41037b0 2024-03-18T15:02:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.670+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f410e3b0 con 0x7fe3f41037b0 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.671+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f40fe560 con 0x7fe3f41037b0 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.671+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe3f0011400 con 0x7fe3f41037b0 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.672+0000 7fe3eaffd700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 0x7fe3e0070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.672+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe3f008d430 con 0x7fe3f41037b0 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.672+0000 7fe3fa574700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 0x7fe3e0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:02:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.672+0000 7fe3fa574700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 0x7fe3e0070390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4004f90 tx=0x7fe3e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:02:13.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.679+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3f005a510 con 0x7fe3f41037b0 2024-03-18T15:02:13.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.977+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe3f4109f80 con 0x7fe3f41037b0 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.978+0000 7fe3eaffd700 1 -- 172.21.15.12:0/2346725666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe3f005a0a0 con 0x7fe3f41037b0 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:02:13.979 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T15:02:13.980 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:02:13.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 msgr2=0x7fe3e0070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:13.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 0x7fe3e0070390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4004f90 tx=0x7fe3e4009e30 comp rx=0 tx=0).stop 2024-03-18T15:02:13.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 msgr2=0x7fe3f41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:02:13.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000b790 tx=0x7fe3f0009220 comp rx=0 tx=0).stop 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 shutdown_connections 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe3e006dee0 0x7fe3e0070390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.980+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3f4102e10 0x7fe3f4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.981+0000 7fe3fc7d8700 1 --2- 172.21.15.12:0/2346725666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f41037b0 0x7fe3f41140d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.981+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 >> 172.21.15.12:0/2346725666 conn(0x7fe3f40f89a0 msgr2=0x7fe3f40f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.981+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 shutdown_connections 2024-03-18T15:02:13.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:02:13.981+0000 7fe3fc7d8700 1 -- 172.21.15.12:0/2346725666 wait complete. 2024-03-18T15:02:14.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:14 smithi012 ceph-mon[111904]: pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T15:02:14.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:14 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2346725666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:02:14.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:14 smithi100 ceph-mon[84868]: pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-18T15:02:14.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:14 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2346725666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:02:14.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-18T15:02:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:16.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:16 smithi100 ceph-mon[84868]: pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:16.335 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:16 smithi012 ceph-mon[111904]: pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:16.596 INFO:teuthology.orchestra.run.smithi012.stdout:wait for servicemap items w/ changing names to refresh 2024-03-18T15:02:17.292 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:17 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:17.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:17 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:17.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'sleep 60' 2024-03-18T15:02:17.618 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:02:18.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:18 smithi100 ceph-mon[84868]: pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:18.441 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:18 smithi012 ceph-mon[111904]: pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:20.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:20 smithi100 ceph-mon[84868]: pgmap v233: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:20.393 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:20 smithi012 ceph-mon[111904]: pgmap v233: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:22.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:22 smithi100 ceph-mon[84868]: pgmap v234: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:22.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:22 smithi012 ceph-mon[111904]: pgmap v234: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:24.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:24 smithi100 ceph-mon[84868]: pgmap v235: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:24.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:24 smithi012 ceph-mon[111904]: pgmap v235: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:26.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:26 smithi100 ceph-mon[84868]: pgmap v236: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:26.375 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:26 smithi012 ceph-mon[111904]: pgmap v236: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:28.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:28 smithi100 ceph-mon[84868]: pgmap v237: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:28.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:28 smithi012 ceph-mon[111904]: pgmap v237: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:30.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:30 smithi100 ceph-mon[84868]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:30.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:30 smithi012 ceph-mon[111904]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:32 smithi100 ceph-mon[84868]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:32 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:32.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:32 smithi012 ceph-mon[111904]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:32.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:32 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:33.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:33 smithi100 ceph-mon[84868]: pgmap v240: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:33.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:33 smithi012 ceph-mon[111904]: pgmap v240: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:35.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:35 smithi012 ceph-mon[111904]: pgmap v241: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:35.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:35 smithi100 ceph-mon[84868]: pgmap v241: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:37.742 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:37 smithi012 ceph-mon[111904]: pgmap v242: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:37.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:37 smithi100 ceph-mon[84868]: pgmap v242: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:39.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:39 smithi012 ceph-mon[111904]: pgmap v243: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:39.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:39 smithi100 ceph-mon[84868]: pgmap v243: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:41.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:41 smithi012 ceph-mon[111904]: pgmap v244: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:41.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:41 smithi100 ceph-mon[84868]: pgmap v244: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:43.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:43 smithi100 ceph-mon[84868]: pgmap v245: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:43.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:43 smithi012 ceph-mon[111904]: pgmap v245: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:45.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:45 smithi100 ceph-mon[84868]: pgmap v246: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:45.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:45 smithi012 ceph-mon[111904]: pgmap v246: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:46.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:46 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:46.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:46 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:02:47.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:47 smithi100 ceph-mon[84868]: pgmap v247: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:47.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:47 smithi012 ceph-mon[111904]: pgmap v247: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:02:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:48.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:48 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:48.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:02:48.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:48.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:48 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:49 smithi100 ceph-mon[84868]: pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:02:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:02:49.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:49 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:49.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:49 smithi012 ceph-mon[111904]: pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:49.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:02:49.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:02:49.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:49 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' 2024-03-18T15:02:51.793 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:51 smithi012 ceph-mon[111904]: pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:51.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:51 smithi100 ceph-mon[84868]: pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:02:53.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:53 smithi100 ceph-mon[84868]: pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:53.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:53 smithi012 ceph-mon[111904]: pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:55.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:55 smithi100 ceph-mon[84868]: pgmap v251: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:55.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:55 smithi012 ceph-mon[111904]: pgmap v251: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:57.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:57 smithi100 ceph-mon[84868]: pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:57.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:57 smithi012 ceph-mon[111904]: pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:02:59.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:02:59 smithi100 ceph-mon[84868]: pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:02:59.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:02:59 smithi012 ceph-mon[111904]: pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:01.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:01 smithi100 ceph-mon[84868]: pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:01.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:01 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:01.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:01 smithi012 ceph-mon[111904]: pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:01.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:01 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:03.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:03 smithi100 ceph-mon[84868]: pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:03.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:03 smithi012 ceph-mon[111904]: pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:05.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:05 smithi100 ceph-mon[84868]: pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:05.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:05 smithi012 ceph-mon[111904]: pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:07.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:07 smithi100 ceph-mon[84868]: pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:07.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:07 smithi012 ceph-mon[111904]: pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:09.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:09 smithi100 ceph-mon[84868]: pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:09.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:09 smithi012 ceph-mon[111904]: pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:11.795 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:11 smithi100 ceph-mon[84868]: pgmap v259: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:11.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:11 smithi012 ceph-mon[111904]: pgmap v259: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:13.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:13 smithi100 ceph-mon[84868]: pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:13.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:13 smithi012 ceph-mon[111904]: pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:15.794 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:15 smithi100 ceph-mon[84868]: pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:15.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:15 smithi012 ceph-mon[111904]: pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:16.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:16 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:17.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:16 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:17.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:17 smithi012 ceph-mon[111904]: pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:18.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:17 smithi100 ceph-mon[84868]: pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:19.991 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:19 smithi012 ceph-mon[111904]: pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:20.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-18T15:03:20.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:19 smithi100 ceph-mon[84868]: pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:03:21.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:21 smithi012 ceph-mon[111904]: pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:22.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:21 smithi100 ceph-mon[84868]: pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 -- 172.21.15.12:0/2012234879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c1037a0 msgr2=0x7fb29c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/2012234879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c1037a0 0x7fb29c0fce10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb290004940 tx=0x7fb290009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 -- 172.21.15.12:0/2012234879 shutdown_connections 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/2012234879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c1037a0 0x7fb29c0fce10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/2012234879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c102e00 0x7fb29c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 -- 172.21.15.12:0/2012234879 >> 172.21.15.12:0/2012234879 conn(0x7fb29c0f89b0 msgr2=0x7fb29c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:22.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 -- 172.21.15.12:0/2012234879 shutdown_connections 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.053+0000 7fb2a0c11700 1 -- 172.21.15.12:0/2012234879 wait complete. 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 Processor -- start 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 -- start start 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 0x7fb29c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c070d40 con 0x7fb29c102e00 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb2a0c11700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c070e80 con 0x7fb29c1037a0 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57758/0 (socket says 172.21.15.12:57758) 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 -- 172.21.15.12:0/1074609816 learned_addr learned my addr 172.21.15.12:0/1074609816 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb299d9b700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 0x7fb29c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 -- 172.21.15.12:0/1074609816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 msgr2=0x7fb29c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 0x7fb29c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb29a59c700 1 -- 172.21.15.12:0/1074609816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290005040 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.054+0000 7fb299d9b700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 0x7fb29c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb29a59c700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c8c0 tx=0x7fb28c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28c00a980 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb28c00ce60 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c071130 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28c0115e0 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.055+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c11a340 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.056+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c1052d0 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.057+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb28c012400 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.057+0000 7fb28affd700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 0x7fb284070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.057+0000 7fb299d9b700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 0x7fb284070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.057+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb28c08d740 con 0x7fb29c102e00 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.058+0000 7fb299d9b700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 0x7fb284070390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fd0 tx=0x7fb290009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.062+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb28c056bb0 con 0x7fb29c102e00 2024-03-18T15:03:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.276+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 --> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb29c071130 con 0x7fb28406dee0 2024-03-18T15:03:22.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.286+0000 7fb28affd700 1 -- 172.21.15.12:0/1074609816 <== mgr.34102 v2:172.21.15.12:6800/4292057823 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb29c071130 con 0x7fb28406dee0 2024-03-18T15:03:22.286 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.smithi012 smithi012 *:9093,9094 running (9m) 2m ago 20m 25.7M - 0.25.0 c8568f914cd2 64decf03a87c 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi012 smithi012 running (6m) 2m ago 20m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 495c6ea3fe0b 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:crash.smithi100 smithi100 running (6m) 101s ago 19m 6665k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 85186d2b9252 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:grafana.smithi012 smithi012 *:3000 running (8m) 2m ago 19m 85.1M - 9.4.12 a0ea5e3c46d9 085808dacf35 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.oheuui smithi012 running (2m) 2m ago 14m 12.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 89022f9c2ba2 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi012.xtadxo smithi012 running (2m) 2m ago 14m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 269c33e068f3 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.dvtbrf smithi100 running (2m) 101s ago 14m 12.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 18a5a74d9437 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mds.cephfs.smithi100.rutxmj smithi100 running (104s) 101s ago 14m 13.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a9412221f4bb 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi012.tmscgt smithi012 *:8443,9283,8765 running (10m) 2m ago 21m 509M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 27bb283a98cf 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.smithi100.rwnrcn smithi100 *:8443,9283,8765 running (10m) 101s ago 19m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 96177683b8af 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi012 smithi012 running (7m) 2m ago 21m 57.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b6df403a483 2024-03-18T15:03:22.287 INFO:teuthology.orchestra.run.smithi012.stdout:mon.smithi100 smithi100 running (7m) 101s ago 19m 44.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9af15206c43d 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi012 smithi012 *:9100 running (9m) 2m ago 20m 15.7M - 1.5.0 0da6a335fe13 64379a0eafed 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter.smithi100 smithi100 *:9100 running (9m) 101s ago 19m 16.0M - 1.5.0 0da6a335fe13 7d9b5dc81f55 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (6m) 2m ago 18m 110M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f0724e03f529 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (5m) 2m ago 17m 120M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b a33f797f55dc 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (4m) 2m ago 17m 104M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b f97e19b7fb9e 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi100 running (4m) 101s ago 17m 126M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 22484c48a3ea 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi100 running (3m) 101s ago 16m 106M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 66948a22ce8e 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi100 running (3m) 101s ago 16m 109M 2581M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9978289a0827 2024-03-18T15:03:22.288 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.smithi012 smithi012 *:9095 running (9m) 2m ago 19m 66.4M - 2.43.0 a07b618ecd1d 82be7eac11d9 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 msgr2=0x7fb284070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 0x7fb284070390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fd0 tx=0x7fb290009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 msgr2=0x7fb29c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c8c0 tx=0x7fb28c0044c0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 shutdown_connections 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb28406dee0 0x7fb284070390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb29c102e00 0x7fb29c0702c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 --2- 172.21.15.12:0/1074609816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb29c1037a0 0x7fb29c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:22.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 >> 172.21.15.12:0/1074609816 conn(0x7fb29c0f89b0 msgr2=0x7fb29c102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:22.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 shutdown_connections 2024-03-18T15:03:22.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:22.288+0000 7fb2a0c11700 1 -- 172.21.15.12:0/1074609816 wait complete. 2024-03-18T15:03:23.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-18T15:03:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:03:23.820 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:23 smithi012 ceph-mon[111904]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:03:23.820 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:23 smithi012 ceph-mon[111904]: pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:24.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:23 smithi100 ceph-mon[84868]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:03:24.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:23 smithi100 ceph-mon[84868]: pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- 172.21.15.12:0/1671978505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fe830 msgr2=0x7f7b40106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 --2- 172.21.15.12:0/1671978505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fe830 0x7f7b40106f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b34004940 tx=0x7f7b34009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- 172.21.15.12:0/1671978505 shutdown_connections 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 --2- 172.21.15.12:0/1671978505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fe830 0x7f7b40106f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 --2- 172.21.15.12:0/1671978505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fde90 0x7f7b400fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- 172.21.15.12:0/1671978505 >> 172.21.15.12:0/1671978505 conn(0x7f7b40072bc0 msgr2=0x7f7b40072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- 172.21.15.12:0/1671978505 shutdown_connections 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- 172.21.15.12:0/1671978505 wait complete. 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 Processor -- start 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.060+0000 7f7b47273700 1 -- start start 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b47273700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b47273700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fe830 0x7f7b401140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b47273700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b401147b0 con 0x7f7b400fde90 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b47273700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4010db10 con 0x7f7b400fe830 2024-03-18T15:03:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57776/0 (socket says 172.21.15.12:57776) 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 -- 172.21.15.12:0/1373563070 learned_addr learned my addr 172.21.15.12:0/1373563070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4480e700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fe830 0x7f7b401140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 -- 172.21.15.12:0/1373563070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fe830 msgr2=0x7f7b401140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fe830 0x7f7b401140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 -- 172.21.15.12:0/1373563070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34005040 con 0x7f7b400fde90 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.061+0000 7f7b4500f700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b790 tx=0x7f7b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.062+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3c00f770 con 0x7f7b400fde90 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.062+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4010ddf0 con 0x7f7b400fde90 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.062+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b3c00fda0 con 0x7f7b400fde90 2024-03-18T15:03:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.062+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3c00d310 con 0x7f7b400fde90 2024-03-18T15:03:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.062+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4010e300 con 0x7f7b400fde90 2024-03-18T15:03:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.063+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b40065e30 con 0x7f7b400fde90 2024-03-18T15:03:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.063+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f7b3c010420 con 0x7f7b400fde90 2024-03-18T15:03:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.064+0000 7f7b31ffb700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 0x7f7b2c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.064+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7b3c08d6d0 con 0x7f7b400fde90 2024-03-18T15:03:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.064+0000 7f7b4480e700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 0x7f7b2c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.064+0000 7f7b4480e700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 0x7f7b2c0703a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b4010edf0 tx=0x7f7b34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:25.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.069+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b3c05a7b0 con 0x7f7b400fde90 2024-03-18T15:03:25.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.408+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b4004bed0 con 0x7f7b400fde90 2024-03-18T15:03:25.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.409+0000 7f7b31ffb700 1 -- 172.21.15.12:0/1373563070 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7b3c05a340 con 0x7f7b400fde90 2024-03-18T15:03:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-03-18T15:03:25.409 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": { 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": { 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": { 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "overall": { 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-18T15:03:25.410 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-03-18T15:03:25.411 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 msgr2=0x7f7b2c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 0x7f7b2c0703a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b4010edf0 tx=0x7f7b34009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 msgr2=0x7f7b40113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b790 tx=0x7f7b3c009250 comp rx=0 tx=0).stop 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 shutdown_connections 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f7b2c06def0 0x7f7b2c0703a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b400fde90 0x7f7b40113b90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 --2- 172.21.15.12:0/1373563070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b400fe830 0x7f7b401140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 >> 172.21.15.12:0/1373563070 conn(0x7f7b40072bc0 msgr2=0x7f7b40103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 shutdown_connections 2024-03-18T15:03:25.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:25.411+0000 7f7b47273700 1 -- 172.21.15.12:0/1373563070 wait complete. 2024-03-18T15:03:25.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:25 smithi012 ceph-mon[111904]: pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:25.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:25 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1373563070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:25 smithi100 ceph-mon[84868]: pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:26.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:25 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1373563070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:26.099 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-18T15:03:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 -- 172.21.15.12:0/1852830624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c1037a0 msgr2=0x7f425c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 --2- 172.21.15.12:0/1852830624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c1037a0 0x7f425c0fce10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4250004940 tx=0x7f4250009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 -- 172.21.15.12:0/1852830624 shutdown_connections 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 --2- 172.21.15.12:0/1852830624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c1037a0 0x7f425c0fce10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 --2- 172.21.15.12:0/1852830624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c102e00 0x7f425c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 -- 172.21.15.12:0/1852830624 >> 172.21.15.12:0/1852830624 conn(0x7f425c0f89b0 msgr2=0x7f425c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 -- 172.21.15.12:0/1852830624 shutdown_connections 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.018+0000 7f426245c700 1 -- 172.21.15.12:0/1852830624 wait complete. 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 Processor -- start 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 -- start start 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c1037a0 0x7f425c1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c114780 con 0x7f425c102e00 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f426245c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c10dae0 con 0x7f425c1037a0 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f425bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f425bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57804/0 (socket says 172.21.15.12:57804) 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f425bfff700 1 -- 172.21.15.12:0/138064393 learned_addr learned my addr 172.21.15.12:0/138064393 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.019+0000 7f425b7fe700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c1037a0 0x7f425c1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f425bfff700 1 -- 172.21.15.12:0/138064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c1037a0 msgr2=0x7f425c1140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f425bfff700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c1037a0 0x7f425c1140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f425bfff700 1 -- 172.21.15.12:0/138064393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250005040 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f425bfff700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b790 tx=0x7f424c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f424c00f6f0 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f424c00fd20 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.020+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c10ddc0 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.021+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f424c0103a0 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.021+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c10e350 con 0x7f425c102e00 2024-03-18T15:03:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.021+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c100110 con 0x7f425c102e00 2024-03-18T15:03:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.022+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f424c011400 con 0x7f425c102e00 2024-03-18T15:03:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.022+0000 7f4258ff9700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 0x7f4244070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.022+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f424c08d290 con 0x7f425c102e00 2024-03-18T15:03:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.022+0000 7f425b7fe700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 0x7f4244070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.023+0000 7f425b7fe700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 0x7f4244070190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f425c10edc0 tx=0x7f4250009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.027+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f424c05a370 con 0x7f425c102e00 2024-03-18T15:03:28.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:27 smithi012 ceph-mon[111904]: pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:28.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:27 smithi100 ceph-mon[84868]: pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.318+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f425c10f1b0 con 0x7f425c102e00 2024-03-18T15:03:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.318+0000 7f4258ff9700 1 -- 172.21.15.12:0/138064393 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f424c0116b0 con 0x7f425c102e00 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 msgr2=0x7f4244070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 0x7f4244070190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f425c10edc0 tx=0x7f4250009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 msgr2=0x7f425c113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b790 tx=0x7f424c009250 comp rx=0 tx=0).stop 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 shutdown_connections 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f424406dce0 0x7f4244070190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f425c102e00 0x7f425c113b60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 --2- 172.21.15.12:0/138064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f425c1037a0 0x7f425c1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:28.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 >> 172.21.15.12:0/138064393 conn(0x7f425c0f89b0 msgr2=0x7f425c0fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:28.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 shutdown_connections 2024-03-18T15:03:28.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:28.320+0000 7f426245c700 1 -- 172.21.15.12:0/138064393 wait complete. 2024-03-18T15:03:28.331 INFO:teuthology.orchestra.run.smithi012.stdout:true 2024-03-18T15:03:29.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-18T15:03:29.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:28 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/138064393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:29.267 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:03:29.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:28 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/138064393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:30.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:29 smithi012 ceph-mon[111904]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:30.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:29 smithi100 ceph-mon[84868]: pgmap v268: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:30.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2166870745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 msgr2=0x7fe7b8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2166870745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b8102140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8004940 tx=0x7fe7a8009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2166870745 shutdown_connections 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2166870745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 0x7fe7b80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2166870745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b8102140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.939+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2166870745 >> 172.21.15.12:0/2166870745 conn(0x7fe7b8072bc0 msgr2=0x7fe7b8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.940+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2166870745 shutdown_connections 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.940+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2166870745 wait complete. 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.941+0000 7fe7bdb66700 1 Processor -- start 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.941+0000 7fe7bdb66700 1 -- start start 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7bdb66700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7bdb66700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 0x7fe7b810f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7bdb66700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b810f700 con 0x7fe7b8101d70 2024-03-18T15:03:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7bdb66700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b810f840 con 0x7fe7b8102710 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36966/0 (socket says 172.21.15.12:36966) 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 -- 172.21.15.12:0/2403159059 learned_addr learned my addr 172.21.15.12:0/2403159059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b6ffd700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 0x7fe7b810f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 -- 172.21.15.12:0/2403159059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 msgr2=0x7fe7b810f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 0x7fe7b810f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 -- 172.21.15.12:0/2403159059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8005040 con 0x7fe7b8101d70 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7b77fe700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800b3b0 tx=0x7fe7a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a800e040 con 0x7fe7b8101d70 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7a800bbc0 con 0x7fe7b8101d70 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b8117b30 con 0x7fe7b8101d70 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.942+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a801c3f0 con 0x7fe7b8101d70 2024-03-18T15:03:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.943+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b81180c0 con 0x7fe7b8101d70 2024-03-18T15:03:30.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.943+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b80ff080 con 0x7fe7b8101d70 2024-03-18T15:03:30.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.945+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe7a800bd30 con 0x7fe7b8101d70 2024-03-18T15:03:30.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.945+0000 7fe79ffff700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 0x7fe7a0070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.945+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe7a808efa0 con 0x7fe7b8101d70 2024-03-18T15:03:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.946+0000 7fe7b6ffd700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 0x7fe7a0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.946+0000 7fe7b6ffd700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 0x7fe7a0070390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac007fd0 tx=0x7fe7ac009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:30.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:30.949+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7a8058410 con 0x7fe7b8101d70 2024-03-18T15:03:31.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.244+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe7b80fd540 con 0x7fe7b8101d70 2024-03-18T15:03:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.244+0000 7fe79ffff700 1 -- 172.21.15.12:0/2403159059 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe7a805bad0 con 0x7fe7b8101d70 2024-03-18T15:03:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.246+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 msgr2=0x7fe7a0070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.246+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 0x7fe7a0070390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac007fd0 tx=0x7fe7ac009040 comp rx=0 tx=0).stop 2024-03-18T15:03:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 msgr2=0x7fe7b810ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800b3b0 tx=0x7fe7a8009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 shutdown_connections 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe7a006dee0 0x7fe7a0070390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7b8101d70 0x7fe7b810ec80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 --2- 172.21.15.12:0/2403159059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b8102710 0x7fe7b810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 >> 172.21.15.12:0/2403159059 conn(0x7fe7b8072bc0 msgr2=0x7fe7b80fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 shutdown_connections 2024-03-18T15:03:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:31.247+0000 7fe7bdb66700 1 -- 172.21.15.12:0/2403159059 wait complete. 2024-03-18T15:03:31.257 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)" 2024-03-18T15:03:32.029 DEBUG:teuthology.parallel:result is None 2024-03-18T15:03:32.029 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T15:03:32.040 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T15:03:32.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74f0a062-e535-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T15:03:32.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:31 smithi012 ceph-mon[111904]: pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:31 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:32.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:31 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2403159059' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:32.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:31 smithi100 ceph-mon[84868]: pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:31 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:32.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:31 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2403159059' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-18T15:03:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/mon.smithi012/config 2024-03-18T15:03:33.992 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:33 smithi012 ceph-mon[111904]: pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:34.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 -- 172.21.15.12:0/656801301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c103780 msgr2=0x7f698c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:34.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 --2- 172.21.15.12:0/656801301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c103780 0x7f698c0fcdf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6980004940 tx=0x7f6980009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:34.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 -- 172.21.15.12:0/656801301 shutdown_connections 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 --2- 172.21.15.12:0/656801301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c103780 0x7f698c0fcdf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 --2- 172.21.15.12:0/656801301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c102de0 0x7f698c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 -- 172.21.15.12:0/656801301 >> 172.21.15.12:0/656801301 conn(0x7f698c0f8950 msgr2=0x7f698c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 -- 172.21.15.12:0/656801301 shutdown_connections 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.061+0000 7f69923f4700 1 -- 172.21.15.12:0/656801301 wait complete. 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 Processor -- start 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 -- start start 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c103780 0x7f698c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0722a0 con 0x7f698c102de0 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f69923f4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c070d40 con 0x7f698c103780 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.062+0000 7f698bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36984/0 (socket says 172.21.15.12:36984) 2024-03-18T15:03:34.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 -- 172.21.15.12:0/3993834482 learned_addr learned my addr 172.21.15.12:0/3993834482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698b7fe700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c103780 0x7f698c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 -- 172.21.15.12:0/3993834482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c103780 msgr2=0x7f698c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c103780 0x7f698c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 -- 172.21.15.12:0/3993834482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980005040 con 0x7f698c102de0 2024-03-18T15:03:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f698bfff700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b790 tx=0x7f697c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697c00f770 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f697c00fda0 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697c00d310 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c071020 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.063+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c11a190 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.064+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c1000f0 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.065+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f697c010420 con 0x7f698c102de0 2024-03-18T15:03:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.065+0000 7f6988ff9700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 0x7f6974070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.066+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f697c08d7f0 con 0x7f698c102de0 2024-03-18T15:03:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.066+0000 7f698b7fe700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 0x7f6974070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.066+0000 7f698b7fe700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 0x7f6974070390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f698000b020 tx=0x7f6980009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:34.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.070+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f697c056ce0 con 0x7f698c102de0 2024-03-18T15:03:34.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:33 smithi100 ceph-mon[84868]: pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.326+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f698c0717a0 con 0x7f698c102de0 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.326+0000 7f6988ff9700 1 -- 172.21.15.12:0/3993834482 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1965 (secure 0 0 0) 0x7f697c05a3a0 con 0x7f698c102de0 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout:e39 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:34.327 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 39 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-03-18T14:48:36.088832+0000 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-03-18T15:01:47.530638+0000 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 60 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-03-18T15:03:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 82 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=34406} 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-03-18T15:03:34.329 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 1 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 34406 members: 34406 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.xtadxo{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.12:6826/1804438217,v1:172.21.15.12:6827/1804438217] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi012.oheuui{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.12:6828/846568565,v1:172.21.15.12:6829/846568565] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.dvtbrf{-1:44321} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6824/3815597664,v1:172.21.15.100:6825/3815597664] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:03:34.330 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.cephfs.smithi100.rutxmj{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.100:6826/3606275918,v1:172.21.15.100:6827/3606275918] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 msgr2=0x7f6974070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 0x7f6974070390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f698000b020 tx=0x7f6980009e30 comp rx=0 tx=0).stop 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 msgr2=0x7f698c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b790 tx=0x7f697c009250 comp rx=0 tx=0).stop 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 shutdown_connections 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f697406dee0 0x7f6974070390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f698c102de0 0x7f698c0702c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 --2- 172.21.15.12:0/3993834482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698c103780 0x7f698c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 >> 172.21.15.12:0/3993834482 conn(0x7f698c0f8950 msgr2=0x7f698c107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 shutdown_connections 2024-03-18T15:03:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:34.329+0000 7f69923f4700 1 -- 172.21.15.12:0/3993834482 wait complete. 2024-03-18T15:03:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-03-18T15:03:34.997 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-18T15:03:35.008 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:03:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 -- 172.21.15.12:0/437938347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62940fa890 msgr2=0x7f62940fad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 --2- 172.21.15.12:0/437938347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62940fa890 0x7f62940fad00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62840048d0 tx=0x7f6284009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 -- 172.21.15.12:0/437938347 shutdown_connections 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 --2- 172.21.15.12:0/437938347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62940fa890 0x7f62940fad00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 --2- 172.21.15.12:0/437938347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6294102240 0x7f6294102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 -- 172.21.15.12:0/437938347 >> 172.21.15.12:0/437938347 conn(0x7f62940708f0 msgr2=0x7f6294070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 -- 172.21.15.12:0/437938347 shutdown_connections 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.110+0000 7f629b39d640 1 -- 172.21.15.12:0/437938347 wait complete. 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 Processor -- start 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 -- start start 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fa890 0x7f6294112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294113790 con 0x7f6294102240 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f629b39d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629410ce30 con 0x7f62940fa890 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36996/0 (socket says 172.21.15.12:36996) 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 -- 172.21.15.12:0/2069371239 learned_addr learned my addr 172.21.15.12:0/2069371239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6299139640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fa890 0x7f6294112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 -- 172.21.15.12:0/2069371239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fa890 msgr2=0x7f6294112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fa890 0x7f6294112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 -- 172.21.15.12:0/2069371239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284005040 con 0x7f6294102240 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.111+0000 7f6298938640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6284004930 tx=0x7f6284009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.112+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62840094c0 con 0x7f6294102240 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.112+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6284032e50 con 0x7f6294102240 2024-03-18T15:03:35.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.112+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62840433b0 con 0x7f6294102240 2024-03-18T15:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.112+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629410d0b0 con 0x7f6294102240 2024-03-18T15:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.112+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629410d570 con 0x7f6294102240 2024-03-18T15:03:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.113+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62940640f0 con 0x7f6294102240 2024-03-18T15:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.113+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f628402f070 con 0x7f6294102240 2024-03-18T15:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.114+0000 7f6289ffb640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 0x7f6280070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.114+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62840af550 con 0x7f6294102240 2024-03-18T15:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.114+0000 7f6299139640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 0x7f6280070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.114+0000 7f6299139640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 0x7f6280070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6290009470 tx=0x7f6290009ed0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.119+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62840789c0 con 0x7f6294102240 2024-03-18T15:03:35.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:34 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3993834482' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:03:35.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:34 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3993834482' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:03:35.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.374+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6294109380 con 0x7f6294102240 2024-03-18T15:03:35.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.375+0000 7f6289ffb640 1 -- 172.21.15.12:0/2069371239 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f628407c080 con 0x7f6294102240 2024-03-18T15:03:35.376 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:35.376 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T15:01:27.308391+0000","last_up_change":"2024-03-18T15:00:13.844612+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-18T14:48:34.682425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-18T14:48:35.325444+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6803","nonce":1970485564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6805","nonce":1970485564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6809","nonce":1970485564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6807","nonce":1970485564}]},"public_addr":"172.21.15.12:6803/1970485564","cluster_addr":"172.21.15.12:6805/1970485564","heartbeat_back_addr":"172.21.15.12:6809/1970485564","heartbeat_front_addr":"172.21.15.12:6807/1970485564","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6811","nonce":4045714107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6813","nonce":4045714107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6817","nonce":4045714107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6815","nonce":4045714107}]},"public_addr":"172.21.15.12:6811/4045714107","cluster_addr":"172.21.15.12:6813/4045714107","heartbeat_back_addr":"172.21.15.12:6817/4045714107","heartbeat_front_addr":"172.21.15.12:6815/4045714107","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6819","nonce":3733153396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6821","nonce":3733153396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6825","nonce":3733153396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6823","nonce":3733153396}]},"public_addr":"172.21.15.12:6819/3733153396","cluster_addr":"172.21.15.12:6821/3733153396","heartbeat_back_addr":"172.21.15.12:6825/3733153396","heartbeat_front_addr":"172.21.15.12:6823/3733153396","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6801","nonce":1440116834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6803","nonce":1440116834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6807","nonce":1440116834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6805","nonce":1440116834}]},"public_addr":"172.21.15.100:6801/1440116834","cluster_addr":"172.21.15.100:6803/1440116834","heartbeat_back_addr":"172.21.15.100:6807/1440116834","heartbeat_front_addr":"172.21.15.100:6805/1440116834","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6809","nonce":2146066735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6811","nonce":2146066735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6815","nonce":2146066735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6813","nonce":2146066735}]},"public_addr":"172.21.15.100:6809/2146066735","cluster_addr":"172.21.15.100:6811/2146066735","heartbeat_back_addr":"172.21.15.100:6815/2146066735","heartbeat_front_addr":"172.21.15.100:6813/2146066735","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6817","nonce":1888487408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6819","nonce":1888487408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6823","nonce":1888487408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6821","nonce":1888487408}]},"public_addr":"172.21.15.100:6817/1888487408","cluster_addr":"172.21.15.100:6819/1888487408","heartbeat_back_addr":"172.21.15.100:6823/1888487408","heartbeat_front_addr":"172.21.15.100:6821/1888487408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6827/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6826/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6825/857186455":"2024-03-19T15:01:01.645715+0000","172.21.15.100:0/640996176":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6826/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2958888698":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2624983338":"2024-03-19T14:50:57.290812+0000","172.21.15.100:0/2236779162":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6801/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6828/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/1012959355":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/814075968":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.100:6829/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/3811799837":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/377430941":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/3753470626":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1103543303":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/1749134857":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6827/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6801/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/444337834":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6824/857186455":"2024-03-19T15:01:01.645715+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:03:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 msgr2=0x7f6280070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 0x7f6280070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6290009470 tx=0x7f6290009ed0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 msgr2=0x7f6294113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6284004930 tx=0x7f6284009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 shutdown_connections 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f628006de50 0x7f6280070300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6294102240 0x7f6294113190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 --2- 172.21.15.12:0/2069371239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62940fa890 0x7f6294112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 >> 172.21.15.12:0/2069371239 conn(0x7f62940708f0 msgr2=0x7f62940f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 shutdown_connections 2024-03-18T15:03:35.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.378+0000 7f629b39d640 1 -- 172.21.15.12:0/2069371239 wait complete. 2024-03-18T15:03:35.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T14:46:03.095708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-18T14:48:34.682425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-18T14:48:35.325444+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:03:35.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:03:35.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 -- 172.21.15.12:0/1898603416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 msgr2=0x7f2e240fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 --2- 172.21.15.12:0/1898603416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e240fb560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e140048d0 tx=0x7f2e14009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 -- 172.21.15.12:0/1898603416 shutdown_connections 2024-03-18T15:03:35.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 --2- 172.21.15.12:0/1898603416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e240fb560 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 --2- 172.21.15.12:0/1898603416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e24101140 0x7f2e24101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 -- 172.21.15.12:0/1898603416 >> 172.21.15.12:0/1898603416 conn(0x7f2e240710b0 msgr2=0x7f2e240714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 -- 172.21.15.12:0/1898603416 shutdown_connections 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.491+0000 7f2e2b665640 1 -- 172.21.15.12:0/1898603416 wait complete. 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 Processor -- start 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 -- start start 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e24101140 0x7f2e2410c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240651e0 con 0x7f2e240fb110 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24065320 con 0x7f2e24101140 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37000/0 (socket says 172.21.15.12:37000) 2024-03-18T15:03:35.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 -- 172.21.15.12:0/2767668750 learned_addr learned my addr 172.21.15.12:0/2767668750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 -- 172.21.15.12:0/2767668750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e24101140 msgr2=0x7f2e2410c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e24101140 0x7f2e2410c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 -- 172.21.15.12:0/2767668750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14005040 con 0x7f2e240fb110 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e29401640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000cb10 tx=0x7f2e20007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e20004200 con 0x7f2e240fb110 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e2000acf0 con 0x7f2e240fb110 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.492+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2410c890 con 0x7f2e240fb110 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.493+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e20003e40 con 0x7f2e240fb110 2024-03-18T15:03:35.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.493+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e241194d0 con 0x7f2e240fb110 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.494+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2e2000a820 con 0x7f2e240fb110 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.494+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e240fc800 con 0x7f2e240fb110 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.494+0000 7f2e19ffb640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 0x7f2e100702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.495+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2e2008def0 con 0x7f2e240fb110 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.495+0000 7f2e28c00640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 0x7f2e100702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.495+0000 7f2e28c00640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 0x7f2e100702b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009990 tx=0x7f2e14009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.499+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e20057360 con 0x7f2e240fb110 2024-03-18T15:03:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.680+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e2404bcc0 con 0x7f2e240fb110 2024-03-18T15:03:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.681+0000 7f2e19ffb640 1 -- 172.21.15.12:0/2767668750 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e2005aa20 con 0x7f2e240fb110 2024-03-18T15:03:35.681 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-03-18T15:03:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.682+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 msgr2=0x7f2e100702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.682+0000 7f2e2b665640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 0x7f2e100702b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009990 tx=0x7f2e14009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.682+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 msgr2=0x7f2e2410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.682+0000 7f2e2b665640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000cb10 tx=0x7f2e20007ee0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 shutdown_connections 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e24101140 0x7f2e2410c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2e1006de00 0x7f2e100702b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 --2- 172.21.15.12:0/2767668750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e240fb110 0x7f2e2410bcc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 >> 172.21.15.12:0/2767668750 conn(0x7f2e240710b0 msgr2=0x7f2e240f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 shutdown_connections 2024-03-18T15:03:35.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.683+0000 7f2e2b665640 1 -- 172.21.15.12:0/2767668750 wait complete. 2024-03-18T15:03:35.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-18T15:03:35.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- 172.21.15.12:0/15469674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 msgr2=0x7fd4800647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 --2- 172.21.15.12:0/15469674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800647f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4740048d0 tx=0x7fd474009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- 172.21.15.12:0/15469674 shutdown_connections 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 --2- 172.21.15.12:0/15469674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800647f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 --2- 172.21.15.12:0/15469674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4801025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- 172.21.15.12:0/15469674 >> 172.21.15.12:0/15469674 conn(0x7fd4800710b0 msgr2=0x7fd4800714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- 172.21.15.12:0/15469674 shutdown_connections 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- 172.21.15.12:0/15469674 wait complete. 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 Processor -- start 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.798+0000 7fd484a6e640 1 -- start start 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4800fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800fd0c0 con 0x7fd4800643a0 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800fbc60 con 0x7fd480102210 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37008/0 (socket says 172.21.15.12:37008) 2024-03-18T15:03:35.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 -- 172.21.15.12:0/2591098440 learned_addr learned my addr 172.21.15.12:0/2591098440 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 -- 172.21.15.12:0/2591098440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 msgr2=0x7fd4800fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47dd9b640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4800fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4800fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 -- 172.21.15.12:0/2591098440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd474005040 con 0x7fd4800643a0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47dd9b640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4800fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd47e59c640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd47000db00 tx=0x7fd470004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd470017070 con 0x7fd4800643a0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4800fbf40 con 0x7fd4800643a0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd47000f040 con 0x7fd4800643a0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd470003ae0 con 0x7fd4800643a0 2024-03-18T15:03:35.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.799+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4801193e0 con 0x7fd4800643a0 2024-03-18T15:03:35.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.800+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800fd910 con 0x7fd4800643a0 2024-03-18T15:03:35.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.801+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd470003c40 con 0x7fd4800643a0 2024-03-18T15:03:35.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.801+0000 7fd46effd640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 0x7fd468070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:35.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.801+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd47008d930 con 0x7fd4800643a0 2024-03-18T15:03:35.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.801+0000 7fd47dd9b640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 0x7fd468070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:35.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.802+0000 7fd47dd9b640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 0x7fd468070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd474000c00 tx=0x7fd474009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:35.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:35.806+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd470056da0 con 0x7fd4800643a0 2024-03-18T15:03:36.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.019+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd48004bcc0 con 0x7fd4800643a0 2024-03-18T15:03:36.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.020+0000 7fd46effd640 1 -- 172.21.15.12:0/2591098440 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd47005a460 con 0x7fd4800643a0 2024-03-18T15:03:36.020 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:36.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 msgr2=0x7fd468070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 0x7fd468070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd474000c00 tx=0x7fd474009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 msgr2=0x7fd4800fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd47000db00 tx=0x7fd470004e60 comp rx=0 tx=0).stop 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 shutdown_connections 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd480102210 0x7fd4800fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd46806de50 0x7fd468070300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 --2- 172.21.15.12:0/2591098440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4800643a0 0x7fd4800fb1e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 >> 172.21.15.12:0/2591098440 conn(0x7fd4800710b0 msgr2=0x7fd4800f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 shutdown_connections 2024-03-18T15:03:36.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.022+0000 7fd484a6e640 1 -- 172.21.15.12:0/2591098440 wait complete. 2024-03-18T15:03:36.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/4644226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 msgr2=0x7fd4f4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/4644226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f4102610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009c50 tx=0x7fd4e4004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/4644226 shutdown_connections 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/4644226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40643a0 0x7fd4f40647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/4644226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f4102610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/4644226 >> 172.21.15.12:0/4644226 conn(0x7fd4f40710b0 msgr2=0x7fd4f40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/4644226 shutdown_connections 2024-03-18T15:03:36.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.136+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/4644226 wait complete. 2024-03-18T15:03:36.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 Processor -- start 2024-03-18T15:03:36.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 -- start start 2024-03-18T15:03:36.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40643a0 0x7fd4f40fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40fd0c0 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4fbb9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40fbc60 con 0x7fd4f40643a0 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37028/0 (socket says 172.21.15.12:37028) 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 -- 172.21.15.12:0/273742387 learned_addr learned my addr 172.21.15.12:0/273742387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 -- 172.21.15.12:0/273742387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40643a0 msgr2=0x7fd4f40fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40643a0 0x7fd4f40fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 -- 172.21.15.12:0/273742387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40098b0 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.137+0000 7fd4f9138640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8002a50 tx=0x7fd4e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.138+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e8007c30 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.138+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4e800f040 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.138+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40fbf40 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.138+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e800db50 con 0x7fd4f4102240 2024-03-18T15:03:36.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.138+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f4119470 con 0x7fd4f4102240 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.139+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f40fd940 con 0x7fd4f4102240 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.139+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd4e800dcf0 con 0x7fd4f4102240 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.139+0000 7fd4f27fc640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 0x7fd4dc070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.140+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd4e808cfd0 con 0x7fd4f4102240 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.140+0000 7fd4f9939640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 0x7fd4dc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.140+0000 7fd4f9939640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 0x7fd4dc070300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4004750 tx=0x7fd4e4004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.144+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4e80564c0 con 0x7fd4f4102240 2024-03-18T15:03:36.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:35 smithi012 ceph-mon[111904]: pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:36.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:35 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2069371239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:36.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:35 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2767668750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:36.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:35 smithi100 ceph-mon[84868]: pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:35 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2069371239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:36.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:35 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2767668750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.328+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd4f404bcc0 con 0x7fd4f4102240 2024-03-18T15:03:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.329+0000 7fd4f27fc640 1 -- 172.21.15.12:0/273742387 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd4e8059b80 con 0x7fd4f4102240 2024-03-18T15:03:36.329 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 msgr2=0x7fd4dc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 0x7fd4dc070300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4004750 tx=0x7fd4e4004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 msgr2=0x7fd4f40fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8002a50 tx=0x7fd4e8004600 comp rx=0 tx=0).stop 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 shutdown_connections 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd4dc06de50 0x7fd4dc070300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4f4102240 0x7fd4f40fb720 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 --2- 172.21.15.12:0/273742387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40643a0 0x7fd4f40fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 >> 172.21.15.12:0/273742387 conn(0x7fd4f40710b0 msgr2=0x7fd4f40f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 shutdown_connections 2024-03-18T15:03:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.331+0000 7fd4fbb9d640 1 -- 172.21.15.12:0/273742387 wait complete. 2024-03-18T15:03:36.342 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.447+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2977065176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 msgr2=0x7f8cc0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.447+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2977065176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0101510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0048d0 tx=0x7f8cbc009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2977065176 shutdown_connections 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2977065176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 0x7f8cc00fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2977065176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0101510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2977065176 >> 172.21.15.12:0/2977065176 conn(0x7f8cc0071130 msgr2=0x7f8cc0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2977065176 shutdown_connections 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2977065176 wait complete. 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 Processor -- start 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- start start 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 0x7f8cc0101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0103dd0 con 0x7f8cc0101140 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.448+0000 7f8cc7c2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0102970 con 0x7f8cc00fb110 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37054/0 (socket says 172.21.15.12:37054) 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 -- 172.21.15.12:0/2042343024 learned_addr learned my addr 172.21.15.12:0/2042343024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc59ca640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 0x7f8cc0101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 -- 172.21.15.12:0/2042343024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 msgr2=0x7f8cc0101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 0x7f8cc0101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 -- 172.21.15.12:0/2042343024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc005040 con 0x7f8cc0101140 2024-03-18T15:03:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc51c9640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0002a50 tx=0x7f8cb0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cb0007c30 con 0x7f8cc0101140 2024-03-18T15:03:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cb000f040 con 0x7f8cc0101140 2024-03-18T15:03:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0102c50 con 0x7f8cc0101140 2024-03-18T15:03:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cb000db50 con 0x7f8cc0101140 2024-03-18T15:03:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.449+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc0119400 con 0x7f8cc0101140 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.450+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc00fc800 con 0x7f8cc0101140 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.451+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8cb001c400 con 0x7f8cc0101140 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.451+0000 7f8cb67fc640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 0x7f8cac070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.451+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8cb008d080 con 0x7f8cc0101140 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.451+0000 7f8cc59ca640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 0x7f8cac070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.452+0000 7f8cc59ca640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 0x7f8cac070350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000c00 tx=0x7f8cbc009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.459+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8cb00c1740 con 0x7f8cc0101140 2024-03-18T15:03:36.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.714+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc01195f0 con 0x7f8cc0101140 2024-03-18T15:03:36.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.715+0000 7f8cb67fc640 1 -- 172.21.15.12:0/2042343024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5218 (secure 0 0 0) 0x7f8cb0056520 con 0x7f8cc0101140 2024-03-18T15:03:36.716 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:36.716 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/3606275918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3606275918},{"type":"v1","addr":"172.21.15.100:6827","nonce":3606275918}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:47.530638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-18T15:03:36.716 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-03-18T15:03:36.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 msgr2=0x7f8cac070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 0x7f8cac070350 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000c00 tx=0x7f8cbc009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 msgr2=0x7f8cc0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0002a50 tx=0x7f8cb0004600 comp rx=0 tx=0).stop 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 shutdown_connections 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8cac06dea0 0x7f8cac070350 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cc0101140 0x7f8cc0102430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 --2- 172.21.15.12:0/2042343024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00fb110 0x7f8cc0101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 >> 172.21.15.12:0/2042343024 conn(0x7f8cc0071130 msgr2=0x7f8cc00f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 shutdown_connections 2024-03-18T15:03:36.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.717+0000 7f8cc7c2e640 1 -- 172.21.15.12:0/2042343024 wait complete. 2024-03-18T15:03:36.729 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-03-18T15:03:36.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 -- 172.21.15.12:0/251986346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 msgr2=0x7f8c88102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/251986346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88102620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8c780048d0 tx=0x7f8c78009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 -- 172.21.15.12:0/251986346 shutdown_connections 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/251986346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 0x7f8c880fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/251986346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88102620 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 -- 172.21.15.12:0/251986346 >> 172.21.15.12:0/251986346 conn(0x7f8c880f7e60 msgr2=0x7f8c880fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 -- 172.21.15.12:0/251986346 shutdown_connections 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.829+0000 7f8c8e86e640 1 -- 172.21.15.12:0/251986346 wait complete. 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 Processor -- start 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 -- start start 2024-03-18T15:03:36.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 0x7f8c881131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881137f0 con 0x7f8c88102250 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c8e86e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8810ce40 con 0x7f8c88102bf0 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37066/0 (socket says 172.21.15.12:37066) 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 -- 172.21.15.12:0/1244691426 learned_addr learned my addr 172.21.15.12:0/1244691426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c877fe640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 0x7f8c881131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 -- 172.21.15.12:0/1244691426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 msgr2=0x7f8c881131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 0x7f8c881131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.830+0000 7f8c87fff640 1 -- 172.21.15.12:0/1244691426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78005040 con 0x7f8c88102250 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c78004930 tx=0x7f8c78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c780094c0 con 0x7f8c88102250 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c78032e50 con 0x7f8c88102250 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8810d0c0 con 0x7f8c88102250 2024-03-18T15:03:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c780433b0 con 0x7f8c88102250 2024-03-18T15:03:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.831+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8810d580 con 0x7f8c88102250 2024-03-18T15:03:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.832+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880fd9c0 con 0x7f8c88102250 2024-03-18T15:03:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.833+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8c7802f070 con 0x7f8c88102250 2024-03-18T15:03:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.833+0000 7f8c84ff9640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 0x7f8c70070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.833+0000 7f8c877fe640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 0x7f8c70070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.833+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8c780af5c0 con 0x7f8c88102250 2024-03-18T15:03:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.834+0000 7f8c877fe640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 0x7f8c70070300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007fd0 tx=0x7f8c7c009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:36.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:36.838+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c78078a30 con 0x7f8c88102250 2024-03-18T15:03:37.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.101+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f8c8804bcc0 con 0x7f8c88102250 2024-03-18T15:03:37.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.101+0000 7f8c84ff9640 1 -- 172.21.15.12:0/1244691426 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v39) v1 ==== 107+0+4090 (secure 0 0 0) 0x7f8c7807c0f0 con 0x7f8c88102250 2024-03-18T15:03:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:37.102 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:14.855436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.12:6827/1453814985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1453814985},{"type":"v1","addr":"172.21.15.12:6827","nonce":1453814985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14476,"qdb_cluster":[14476]},"id":1}]} 2024-03-18T15:03:37.102 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-03-18T15:03:37.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.103+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 msgr2=0x7f8c70070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.103+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 0x7f8c70070300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007fd0 tx=0x7f8c7c009e90 comp rx=0 tx=0).stop 2024-03-18T15:03:37.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.103+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 msgr2=0x7f8c88112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.103+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8c78004930 tx=0x7f8c78009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.103+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 shutdown_connections 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c88102bf0 0x7f8c881131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f8c7006de50 0x7f8c70070300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 --2- 172.21.15.12:0/1244691426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c88102250 0x7f8c88112cb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 >> 172.21.15.12:0/1244691426 conn(0x7f8c880f7e60 msgr2=0x7f8c880f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 shutdown_connections 2024-03-18T15:03:37.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.104+0000 7f8c8e86e640 1 -- 172.21.15.12:0/1244691426 wait complete. 2024-03-18T15:03:37.115 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 16 2024-03-18T15:03:37.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-18T15:03:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/873548608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac1043f0 msgr2=0x7f2aac1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/873548608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac1043f0 0x7f2aac1047c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80048d0 tx=0x7f2aa8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/873548608 shutdown_connections 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/873548608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac104d90 0x7f2aac0fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/873548608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac1043f0 0x7f2aac1047c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/873548608 >> 172.21.15.12:0/873548608 conn(0x7f2aac0f9fc0 msgr2=0x7f2aac0fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/873548608 shutdown_connections 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.214+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/873548608 wait complete. 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 Processor -- start 2024-03-18T15:03:37.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 -- start start 2024-03-18T15:03:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac1043f0 0x7f2aac114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac115a20 con 0x7f2aac104d90 2024-03-18T15:03:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.215+0000 7f2ab3a2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac10f070 con 0x7f2aac1043f0 2024-03-18T15:03:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37094/0 (socket says 172.21.15.12:37094) 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 -- 172.21.15.12:0/1766065366 learned_addr learned my addr 172.21.15.12:0/1766065366 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab17ca640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac1043f0 0x7f2aac114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 -- 172.21.15.12:0/1766065366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac1043f0 msgr2=0x7f2aac114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac1043f0 0x7f2aac114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 -- 172.21.15.12:0/1766065366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8005040 con 0x7f2aac104d90 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab0fc9640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c002a50 tx=0x7f2a9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c007c30 con 0x7f2aac104d90 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a9c00f040 con 0x7f2aac104d90 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac10f350 con 0x7f2aac104d90 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c00db50 con 0x7f2aac104d90 2024-03-18T15:03:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.216+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac10f870 con 0x7f2aac104d90 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.218+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac0ffb60 con 0x7f2aac104d90 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.218+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f2a9c01c400 con 0x7f2aac104d90 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.218+0000 7f2aa1ffb640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 0x7f2a98070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.218+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2a9c08cfa0 con 0x7f2aac104d90 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.218+0000 7f2ab17ca640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 0x7f2a98070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.219+0000 7f2ab17ca640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 0x7f2a98070300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003080 tx=0x7f2aa8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:37.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.223+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a9c0c1740 con 0x7f2aac104d90 2024-03-18T15:03:37.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:36 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2591098440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:37.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:36 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/273742387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:37.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:36 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2042343024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:03:37.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:36 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2591098440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:37.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:36 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/273742387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:37.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:36 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2042343024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:03:37.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.516+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f2aac10b550 con 0x7f2aac104d90 2024-03-18T15:03:37.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.516+0000 7f2aa1ffb640 1 -- 172.21.15.12:0/1766065366 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v39) v1 ==== 107+0+4869 (secure 0 0 0) 0x7f2a9c0564c0 con 0x7f2aac104d90 2024-03-18T15:03:37.518 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:37.518 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:14.855436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.12:6827/1453814985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1453814985},{"type":"v1","addr":"172.21.15.12:6827","nonce":1453814985}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14476,"qdb_cluster":[14476]},"id":1}]} 2024-03-18T15:03:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-03-18T15:03:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 msgr2=0x7f2a98070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 0x7f2a98070300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003080 tx=0x7f2aa8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 msgr2=0x7f2aac115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c002a50 tx=0x7f2a9c004600 comp rx=0 tx=0).stop 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 shutdown_connections 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f2a9806de50 0x7f2a98070300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2aac104d90 0x7f2aac115420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 --2- 172.21.15.12:0/1766065366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2aac1043f0 0x7f2aac114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 >> 172.21.15.12:0/1766065366 conn(0x7f2aac0f9fc0 msgr2=0x7f2aac107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 shutdown_connections 2024-03-18T15:03:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.518+0000 7f2ab3a2e640 1 -- 172.21.15.12:0/1766065366 wait complete. 2024-03-18T15:03:37.534 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-03-18T15:03:37.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-18T15:03:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 -- 172.21.15.12:0/531301069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 msgr2=0x7f1cf41027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 --2- 172.21.15.12:0/531301069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf41027b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00048d0 tx=0x7f1cf0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 -- 172.21.15.12:0/531301069 shutdown_connections 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 --2- 172.21.15.12:0/531301069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf4102d80 0x7f1cf40fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 --2- 172.21.15.12:0/531301069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf41027b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 -- 172.21.15.12:0/531301069 >> 172.21.15.12:0/531301069 conn(0x7f1cf40f7fb0 msgr2=0x7f1cf40fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 -- 172.21.15.12:0/531301069 shutdown_connections 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.635+0000 7f1cfc607640 1 -- 172.21.15.12:0/531301069 wait complete. 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 Processor -- start 2024-03-18T15:03:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 -- start start 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf4102d80 0x7f1cf4113440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4113a40 con 0x7f1cf41023e0 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfc607640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf410d090 con 0x7f1cf4102d80 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfa3a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfa3a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37112/0 (socket says 172.21.15.12:37112) 2024-03-18T15:03:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.636+0000 7f1cfa3a3640 1 -- 172.21.15.12:0/1339734913 learned_addr learned my addr 172.21.15.12:0/1339734913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfa3a3640 1 -- 172.21.15.12:0/1339734913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf4102d80 msgr2=0x7f1cf4113440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfa3a3640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf4102d80 0x7f1cf4113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfa3a3640 1 -- 172.21.15.12:0/1339734913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0005040 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfa3a3640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0034040 tx=0x7f1cf0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf00092b0 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cf003b400 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf410d310 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf00435d0 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.637+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf410d7d0 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.638+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf40fdb50 con 0x7f1cf41023e0 2024-03-18T15:03:37.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.639+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f1cf002f070 con 0x7f1cf41023e0 2024-03-18T15:03:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.639+0000 7f1ceaffd640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 0x7f1ce0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.639+0000 7f1cf9ba2640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 0x7f1ce0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.640+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1cf00af570 con 0x7f1cf41023e0 2024-03-18T15:03:37.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.640+0000 7f1cf9ba2640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 0x7f1ce0070300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cf410e7c0 tx=0x7f1ce400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:37.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.644+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1cf00789e0 con 0x7f1cf41023e0 2024-03-18T15:03:37.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.866+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f1cf4109570 con 0x7f1cf41023e0 2024-03-18T15:03:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.866+0000 7f1ceaffd640 1 -- 172.21.15.12:0/1339734913 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v39) v1 ==== 107+0+4072 (secure 0 0 0) 0x7f1cf007c0a0 con 0x7f1cf41023e0 2024-03-18T15:03:37.867 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:37.867 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:19.128691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:37.867 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 18 2024-03-18T15:03:37.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.868+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 msgr2=0x7f1ce0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.868+0000 7f1cfc607640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 0x7f1ce0070300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cf410e7c0 tx=0x7f1ce400b040 comp rx=0 tx=0).stop 2024-03-18T15:03:37.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.868+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 msgr2=0x7f1cf4112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:37.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.868+0000 7f1cfc607640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0034040 tx=0x7f1cf0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 shutdown_connections 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf4102d80 0x7f1cf4113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f1ce006de50 0x7f1ce0070300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 --2- 172.21.15.12:0/1339734913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cf41023e0 0x7f1cf4112f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 >> 172.21.15.12:0/1339734913 conn(0x7f1cf40f7fb0 msgr2=0x7f1cf40f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 shutdown_connections 2024-03-18T15:03:37.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.869+0000 7f1cfc607640 1 -- 172.21.15.12:0/1339734913 wait complete. 2024-03-18T15:03:37.880 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-03-18T15:03:37.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-18T15:03:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 -- 172.21.15.12:0/2645640271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 msgr2=0x7fd4a81014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 --2- 172.21.15.12:0/2645640271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a81014a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0048d0 tx=0x7fd49c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 -- 172.21.15.12:0/2645640271 shutdown_connections 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 --2- 172.21.15.12:0/2645640271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a8101a70 0x7fd4a80fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 --2- 172.21.15.12:0/2645640271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a81014a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 -- 172.21.15.12:0/2645640271 >> 172.21.15.12:0/2645640271 conn(0x7fd4a80f6ca0 msgr2=0x7fd4a80f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 -- 172.21.15.12:0/2645640271 shutdown_connections 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:37.999+0000 7fd4af025640 1 -- 172.21.15.12:0/2645640271 wait complete. 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 Processor -- start 2024-03-18T15:03:38.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 -- start start 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a8101a70 0x7fd4a81120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81126e0 con 0x7fd4a81010d0 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4af025640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a810bd30 con 0x7fd4a8101a70 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4acdc1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4acdc1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37128/0 (socket says 172.21.15.12:37128) 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.000+0000 7fd4acdc1640 1 -- 172.21.15.12:0/2953958592 learned_addr learned my addr 172.21.15.12:0/2953958592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4acdc1640 1 -- 172.21.15.12:0/2953958592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a8101a70 msgr2=0x7fd4a81120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4acdc1640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a8101a70 0x7fd4a81120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4acdc1640 1 -- 172.21.15.12:0/2953958592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c005040 con 0x7fd4a81010d0 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4acdc1640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd49c004930 tx=0x7fd49c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c0094c0 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a810bfb0 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd49c032e50 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c0433b0 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.001+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a810c470 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.002+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a80fc840 con 0x7fd4a81010d0 2024-03-18T15:03:38.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.003+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd49c02f070 con 0x7fd4a81010d0 2024-03-18T15:03:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.004+0000 7fd4a57fa640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 0x7fd498070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.004+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd49c0af650 con 0x7fd4a81010d0 2024-03-18T15:03:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.004+0000 7fd4a7fff640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 0x7fd498070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.004+0000 7fd4a7fff640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 0x7fd498070300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a810d460 tx=0x7fd494009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.009+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd49c078ac0 con 0x7fd4a81010d0 2024-03-18T15:03:38.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.230+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fd4a810c690 con 0x7fd4a81010d0 2024-03-18T15:03:38.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.231+0000 7fd4a57fa640 1 -- 172.21.15.12:0/2953958592 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4083 (secure 0 0 0) 0x7fd49c07c180 con 0x7fd4a81010d0 2024-03-18T15:03:38.231 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:38.231 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:19.133089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:38.232 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2024-03-18T15:03:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 msgr2=0x7fd498070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 0x7fd498070300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4a810d460 tx=0x7fd494009c50 comp rx=0 tx=0).stop 2024-03-18T15:03:38.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 msgr2=0x7fd4a8111ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd49c004930 tx=0x7fd49c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 shutdown_connections 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4a8101a70 0x7fd4a81120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd49806de50 0x7fd498070300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 --2- 172.21.15.12:0/2953958592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4a81010d0 0x7fd4a8111ba0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 >> 172.21.15.12:0/2953958592 conn(0x7fd4a80f6ca0 msgr2=0x7fd4a80f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 shutdown_connections 2024-03-18T15:03:38.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.233+0000 7fd4af025640 1 -- 172.21.15.12:0/2953958592 wait complete. 2024-03-18T15:03:38.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:37 smithi012 ceph-mon[111904]: pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:38.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:37 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1244691426' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-18T15:03:38.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:37 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1766065366' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-18T15:03:38.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:37 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1339734913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-18T15:03:38.244 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-03-18T15:03:38.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-18T15:03:38.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:37 smithi100 ceph-mon[84868]: pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:37 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1244691426' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-18T15:03:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:37 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1766065366' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-18T15:03:38.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:37 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1339734913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-18T15:03:38.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- 172.21.15.12:0/1264139160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 msgr2=0x7f96040fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 --2- 172.21.15.12:0/1264139160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f96040fb570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0048d0 tx=0x7f95ec009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- 172.21.15.12:0/1264139160 shutdown_connections 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 --2- 172.21.15.12:0/1264139160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f96040fb570 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 --2- 172.21.15.12:0/1264139160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 0x7f9604101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- 172.21.15.12:0/1264139160 >> 172.21.15.12:0/1264139160 conn(0x7f9604071120 msgr2=0x7f9604071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- 172.21.15.12:0/1264139160 shutdown_connections 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- 172.21.15.12:0/1264139160 wait complete. 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 Processor -- start 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- start start 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 0x7f96041120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96041126d0 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960899d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960410bd20 con 0x7f9604101090 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.345+0000 7f960259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37146/0 (socket says 172.21.15.12:37146) 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 -- 172.21.15.12:0/1578626913 learned_addr learned my addr 172.21.15.12:0/1578626913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f95fbfff640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 0x7f96041120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 -- 172.21.15.12:0/1578626913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 msgr2=0x7f96041120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 0x7f96041120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 -- 172.21.15.12:0/1578626913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec005040 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960259c640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95f400eb70 tx=0x7f95f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f4007c30 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960410bfa0 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f400f040 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f40095b0 con 0x7f96040fb100 2024-03-18T15:03:38.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.346+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960410c490 con 0x7f96040fb100 2024-03-18T15:03:38.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.347+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96040fc790 con 0x7f96040fb100 2024-03-18T15:03:38.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.348+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f95f4009730 con 0x7f96040fb100 2024-03-18T15:03:38.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.348+0000 7f95faffd640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 0x7f95f0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.348+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f95f408d470 con 0x7f96040fb100 2024-03-18T15:03:38.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.348+0000 7f95fbfff640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 0x7f95f0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.349+0000 7f95fbfff640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 0x7f95f0070300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9604105720 tx=0x7f95ec009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.353+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95f4056990 con 0x7f96040fb100 2024-03-18T15:03:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.575+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f960410d290 con 0x7f96040fb100 2024-03-18T15:03:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.576+0000 7f95faffd640 1 -- 172.21.15.12:0/1578626913 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f95f405a050 con 0x7f96040fb100 2024-03-18T15:03:38.576 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:38.576 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:27.211692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:reconnect","state_seq":177,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:38.577 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2024-03-18T15:03:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.577+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 msgr2=0x7f95f0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.577+0000 7f960899d640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 0x7f95f0070300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9604105720 tx=0x7f95ec009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.577+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 msgr2=0x7f9604111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.577+0000 7f960899d640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95f400eb70 tx=0x7f95f400b040 comp rx=0 tx=0).stop 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 shutdown_connections 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9604101090 0x7f96041120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f95f006de50 0x7f95f0070300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 --2- 172.21.15.12:0/1578626913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96040fb100 0x7f9604111b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 >> 172.21.15.12:0/1578626913 conn(0x7f9604071120 msgr2=0x7f96040fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 shutdown_connections 2024-03-18T15:03:38.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.578+0000 7f960899d640 1 -- 172.21.15.12:0/1578626913 wait complete. 2024-03-18T15:03:38.589 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-03-18T15:03:38.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/998340056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 msgr2=0x7fd0ec1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/998340056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec1061e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00048d0 tx=0x7fd0e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/998340056 shutdown_connections 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/998340056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec1061e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/998340056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 0x7fd0ec0fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/998340056 >> 172.21.15.12:0/998340056 conn(0x7fd0ec0f6cb0 msgr2=0x7fd0ec0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/998340056 shutdown_connections 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.700+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/998340056 wait complete. 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 Processor -- start 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 -- start start 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 0x7fd0ec114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec115a80 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0f0e6e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec10f0d0 con 0x7fd0ec0fd200 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54134/0 (socket says 172.21.15.12:54134) 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 -- 172.21.15.12:0/655751190 learned_addr learned my addr 172.21.15.12:0/655751190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0ea59c640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 0x7fd0ec114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 -- 172.21.15.12:0/655751190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 msgr2=0x7fd0ec114f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 0x7fd0ec114f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 -- 172.21.15.12:0/655751190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0005040 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.701+0000 7fd0e9d9b640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0034040 tx=0x7fd0e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.702+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e00092b0 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.702+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0e003b400 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.702+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec10f350 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.702+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e00435d0 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.702+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec10f810 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.703+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ec065980 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.703+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fd0e002f070 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.704+0000 7fd0daffd640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 0x7fd0d4070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.704+0000 7fd0ea59c640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 0x7fd0d4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.704+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd0e00af330 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.704+0000 7fd0ea59c640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 0x7fd0d4070300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009af0 tx=0x7fd0dc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:38.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.708+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0e00788d0 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.972+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fd0ec109330 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.973+0000 7fd0daffd640 1 -- 172.21.15.12:0/655751190 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4932 (secure 0 0 0) 0x7fd0e007bf90 con 0x7fd0ec0fdba0 2024-03-18T15:03:38.973 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:38.974 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:28.215069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":178,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:38.974 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.974+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 msgr2=0x7fd0d4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.974+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 0x7fd0d4070300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009af0 tx=0x7fd0dc00b040 comp rx=0 tx=0).stop 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.974+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 msgr2=0x7fd0ec115480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.974+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0034040 tx=0x7fd0e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 shutdown_connections 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fd0d406de50 0x7fd0d4070300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0ec0fdba0 0x7fd0ec115480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 --2- 172.21.15.12:0/655751190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0ec0fd200 0x7fd0ec114f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 >> 172.21.15.12:0/655751190 conn(0x7fd0ec0f6cb0 msgr2=0x7fd0ec0f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 shutdown_connections 2024-03-18T15:03:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:38.975+0000 7fd0f0e6e640 1 -- 172.21.15.12:0/655751190 wait complete. 2024-03-18T15:03:38.986 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-18T15:03:38.986 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 -- 172.21.15.12:0/4257988696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3444102c70 msgr2=0x7f34440fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 --2- 172.21.15.12:0/4257988696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3444102c70 0x7f34440fc1f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34380048d0 tx=0x7f3438009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 -- 172.21.15.12:0/4257988696 shutdown_connections 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 --2- 172.21.15.12:0/4257988696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3444102c70 0x7f34440fc1f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 --2- 172.21.15.12:0/4257988696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441022d0 0x7f34441026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 -- 172.21.15.12:0/4257988696 >> 172.21.15.12:0/4257988696 conn(0x7f34440f7d90 msgr2=0x7f34440fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 -- 172.21.15.12:0/4257988696 shutdown_connections 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 -- 172.21.15.12:0/4257988696 wait complete. 2024-03-18T15:03:39.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.085+0000 7f344aa25640 1 Processor -- start 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f344aa25640 1 -- start start 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f344aa25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f344aa25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3444102c70 0x7f3444113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f344aa25640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444113950 con 0x7f34441022d0 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f344aa25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344410cfa0 con 0x7f3444102c70 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54150/0 (socket says 172.21.15.12:54150) 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 -- 172.21.15.12:0/2598046239 learned_addr learned my addr 172.21.15.12:0/2598046239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f34437fe640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3444102c70 0x7f3444113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 -- 172.21.15.12:0/2598046239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3444102c70 msgr2=0x7f3444113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3444102c70 0x7f3444113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.086+0000 7f3443fff640 1 -- 172.21.15.12:0/2598046239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438005040 con 0x7f34441022d0 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f3443fff640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f343400c810 tx=0x7f34340043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f343400a820 con 0x7f34441022d0 2024-03-18T15:03:39.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f343400ae50 con 0x7f34441022d0 2024-03-18T15:03:39.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344410d280 con 0x7f34441022d0 2024-03-18T15:03:39.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34340115c0 con 0x7f34441022d0 2024-03-18T15:03:39.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.087+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344410d770 con 0x7f34441022d0 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.088+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344404bcc0 con 0x7f34441022d0 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.088+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f343400a980 con 0x7f34441022d0 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.089+0000 7f3440ff9640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 0x7f342c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.089+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f343408dd20 con 0x7f34441022d0 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.089+0000 7f34437fe640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 0x7f342c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.089+0000 7f34437fe640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 0x7f342c070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34440fa840 tx=0x7f3438009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.093+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3434057190 con 0x7f34441022d0 2024-03-18T15:03:39.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:38 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2953958592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-18T15:03:39.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:38 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1578626913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-18T15:03:39.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:38 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/655751190' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-18T15:03:39.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:38 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2953958592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-18T15:03:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:38 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1578626913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-18T15:03:39.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:38 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/655751190' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-18T15:03:39.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.454+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f34440fd930 con 0x7f34441022d0 2024-03-18T15:03:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.454+0000 7f3440ff9640 1 -- 172.21.15.12:0/2598046239 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+4941 (secure 0 0 0) 0x7f343405a850 con 0x7f34441022d0 2024-03-18T15:03:39.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:39.456 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34374,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/3243464363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3243464363},{"type":"v1","addr":"172.21.15.12:6829","nonce":3243464363}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:30.219376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:active","state_seq":179,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14484,"qdb_cluster":[14484]},"id":1}]} 2024-03-18T15:03:39.456 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 22 2024-03-18T15:03:39.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 msgr2=0x7f342c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 0x7f342c070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34440fa840 tx=0x7f3438009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 msgr2=0x7f3444112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f343400c810 tx=0x7f34340043c0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 shutdown_connections 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3444102c70 0x7f3444113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f342c06de50 0x7f342c070300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 --2- 172.21.15.12:0/2598046239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34441022d0 0x7f3444112e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 >> 172.21.15.12:0/2598046239 conn(0x7f34440f7d90 msgr2=0x7f34440f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 shutdown_connections 2024-03-18T15:03:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.456+0000 7f344aa25640 1 -- 172.21.15.12:0/2598046239 wait complete. 2024-03-18T15:03:39.468 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-18T15:03:39.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-18T15:03:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 -- 172.21.15.12:0/3727805481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed4102240 msgr2=0x7f0ed4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/3727805481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed4102240 0x7f0ed4102610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009c50 tx=0x7f0ecc004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 -- 172.21.15.12:0/3727805481 shutdown_connections 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/3727805481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed40643a0 0x7f0ed40647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/3727805481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed4102240 0x7f0ed4102610 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 -- 172.21.15.12:0/3727805481 >> 172.21.15.12:0/3727805481 conn(0x7f0ed40710b0 msgr2=0x7f0ed40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 -- 172.21.15.12:0/3727805481 shutdown_connections 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 -- 172.21.15.12:0/3727805481 wait complete. 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.602+0000 7f0edaf9d640 1 Processor -- start 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0edaf9d640 1 -- start start 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0edaf9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0edaf9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed4102240 0x7f0ed40fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0edaf9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40fd0c0 con 0x7f0ed40643a0 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0edaf9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40fbc60 con 0x7f0ed4102240 2024-03-18T15:03:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54180/0 (socket says 172.21.15.12:54180) 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 -- 172.21.15.12:0/2710507828 learned_addr learned my addr 172.21.15.12:0/2710507828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 -- 172.21.15.12:0/2710507828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed4102240 msgr2=0x7f0ed40fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed4102240 0x7f0ed40fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 -- 172.21.15.12:0/2710507828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0098b0 con 0x7f0ed40643a0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.603+0000 7f0ed8d39640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc004f80 tx=0x7f0ecc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.604+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc03a070 con 0x7f0ed40643a0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.604+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed40fbee0 con 0x7f0ed40643a0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.604+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ecc009740 con 0x7f0ed40643a0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.604+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc03ea60 con 0x7f0ed40643a0 2024-03-18T15:03:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.604+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4119420 con 0x7f0ed40643a0 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.605+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed40fd940 con 0x7f0ed40643a0 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.605+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f0ecc032070 con 0x7f0ed40643a0 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.606+0000 7f0ed17fa640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 0x7f0ec0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.606+0000 7f0ed3fff640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 0x7f0ec0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.606+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0ecc0af910 con 0x7f0ed40643a0 2024-03-18T15:03:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.606+0000 7f0ed3fff640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 0x7f0ec0070300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40fc910 tx=0x7f0ec400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.610+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ecc078db0 con 0x7f0ed40643a0 2024-03-18T15:03:39.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.833+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f0ed4119610 con 0x7f0ed40643a0 2024-03-18T15:03:39.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.834+0000 7f0ed17fa640 1 -- 172.21.15.12:0/2710507828 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+4162 (secure 0 0 0) 0x7f0ecc07c470 con 0x7f0ed40643a0 2024-03-18T15:03:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:39.835 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:30.219376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:active","state_seq":179,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14484,"qdb_cluster":[14484]},"id":1}]} 2024-03-18T15:03:39.835 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 msgr2=0x7f0ec0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 0x7f0ec0070300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40fc910 tx=0x7f0ec400b040 comp rx=0 tx=0).stop 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 msgr2=0x7f0ed40fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc004f80 tx=0x7f0ecc004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 shutdown_connections 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed4102240 0x7f0ed40fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0ec006de50 0x7f0ec0070300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 --2- 172.21.15.12:0/2710507828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ed40643a0 0x7f0ed40fb1e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 >> 172.21.15.12:0/2710507828 conn(0x7f0ed40710b0 msgr2=0x7f0ed40f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 shutdown_connections 2024-03-18T15:03:39.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.836+0000 7f0edaf9d640 1 -- 172.21.15.12:0/2710507828 wait complete. 2024-03-18T15:03:39.848 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-18T15:03:39.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.965+0000 7f5885f14640 1 -- 172.21.15.12:0/633908388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58801044d0 msgr2=0x7f58801048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.965+0000 7f5885f14640 1 --2- 172.21.15.12:0/633908388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58801044d0 0x7f58801048a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58700048d0 tx=0x7f5870009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- 172.21.15.12:0/633908388 shutdown_connections 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 --2- 172.21.15.12:0/633908388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58800fe490 0x7f58800fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 --2- 172.21.15.12:0/633908388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58801044d0 0x7f58801048a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- 172.21.15.12:0/633908388 >> 172.21.15.12:0/633908388 conn(0x7f58800fa050 msgr2=0x7f58800fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- 172.21.15.12:0/633908388 shutdown_connections 2024-03-18T15:03:39.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- 172.21.15.12:0/633908388 wait complete. 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 Processor -- start 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- start start 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58801044d0 0x7f5880115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880115a20 con 0x7f58800fe490 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f5885f14640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588010f070 con 0x7f58801044d0 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.966+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54192/0 (socket says 172.21.15.12:54192) 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 -- 172.21.15.12:0/2053909710 learned_addr learned my addr 172.21.15.12:0/2053909710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587effd640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58801044d0 0x7f5880115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 -- 172.21.15.12:0/2053909710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58801044d0 msgr2=0x7f5880115420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58801044d0 0x7f5880115420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:39.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 -- 172.21.15.12:0/2053909710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870005040 con 0x7f58800fe490 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f587f7fe640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5870004930 tx=0x7f5870009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58700094c0 con 0x7f58800fe490 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588010f2f0 con 0x7f58800fe490 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5870032e50 con 0x7f58800fe490 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58700433b0 con 0x7f58800fe490 2024-03-18T15:03:39.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.967+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588010f7b0 con 0x7f58800fe490 2024-03-18T15:03:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.968+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58800ffbd0 con 0x7f58800fe490 2024-03-18T15:03:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.969+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f587002f070 con 0x7f58800fe490 2024-03-18T15:03:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.970+0000 7f5867fff640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 0x7f5868070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.970+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f58700af620 con 0x7f58800fe490 2024-03-18T15:03:39.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.970+0000 7f587effd640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 0x7f5868070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:39.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.970+0000 7f587effd640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 0x7f5868070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58801107a0 tx=0x7f587400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:39.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:39.975+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5870078ac0 con 0x7f58800fe490 2024-03-18T15:03:40.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.197+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f588004bcc0 con 0x7f58800fe490 2024-03-18T15:03:40.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.198+0000 7f5867fff640 1 -- 172.21.15.12:0/2053909710 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+5006 (secure 0 0 0) 0x7f587007c180 con 0x7f58800fe490 2024-03-18T15:03:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:40.199 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:00:30.219376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi100.dvtbrf","rank":0,"incarnation":19,"state":"up:active","state_seq":179,"addr":"172.21.15.100:6825/857186455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":857186455},{"type":"v1","addr":"172.21.15.100:6825","nonce":857186455}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14484,"qdb_cluster":[14484]},"id":1}]} 2024-03-18T15:03:40.199 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 msgr2=0x7f5868070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 0x7f5868070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58801107a0 tx=0x7f587400b040 comp rx=0 tx=0).stop 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 msgr2=0x7f5880114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5870004930 tx=0x7f5870009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 shutdown_connections 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58801044d0 0x7f5880115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f586806de50 0x7f5868070300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 --2- 172.21.15.12:0/2053909710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58800fe490 0x7f5880114ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 >> 172.21.15.12:0/2053909710 conn(0x7f58800fa050 msgr2=0x7f58800fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 shutdown_connections 2024-03-18T15:03:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.200+0000 7f5885f14640 1 -- 172.21.15.12:0/2053909710 wait complete. 2024-03-18T15:03:40.210 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:39 smithi012 ceph-mon[111904]: pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:40.210 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:39 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2598046239' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-18T15:03:40.211 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:39 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2710507828' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-18T15:03:40.211 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-18T15:03:40.212 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-18T15:03:40.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:39 smithi100 ceph-mon[84868]: pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:40.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:39 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2598046239' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-18T15:03:40.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:39 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2710507828' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/1465607739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f0102240 msgr2=0x7fe2f0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/1465607739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f0102240 0x7fe2f0102610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00048d0 tx=0x7fe2e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/1465607739 shutdown_connections 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/1465607739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f00643a0 0x7fe2f00647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/1465607739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f0102240 0x7fe2f0102610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/1465607739 >> 172.21.15.12:0/1465607739 conn(0x7fe2f00710b0 msgr2=0x7fe2f00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/1465607739 shutdown_connections 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.310+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/1465607739 wait complete. 2024-03-18T15:03:40.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 Processor -- start 2024-03-18T15:03:40.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 -- start start 2024-03-18T15:03:40.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f0102240 0x7fe2f0113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f0113810 con 0x7fe2f00643a0 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2f6a2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f010ce60 con 0x7fe2f0102240 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54212/0 (socket says 172.21.15.12:54212) 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2effff640 1 -- 172.21.15.12:0/654976570 learned_addr learned my addr 172.21.15.12:0/654976570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.311+0000 7fe2ef7fe640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f0102240 0x7fe2f0113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2effff640 1 -- 172.21.15.12:0/654976570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f0102240 msgr2=0x7fe2f0113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2effff640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f0102240 0x7fe2f0113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2effff640 1 -- 172.21.15.12:0/654976570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0005040 con 0x7fe2f00643a0 2024-03-18T15:03:40.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2effff640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0004930 tx=0x7fe2e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e00094c0 con 0x7fe2f00643a0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2e0032e50 con 0x7fe2f00643a0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f010d0e0 con 0x7fe2f00643a0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e00433b0 con 0x7fe2f00643a0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.312+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f010d5a0 con 0x7fe2f00643a0 2024-03-18T15:03:40.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.313+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f00fd940 con 0x7fe2f00643a0 2024-03-18T15:03:40.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.314+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fe2e002f070 con 0x7fe2f00643a0 2024-03-18T15:03:40.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.314+0000 7fe2ecff9640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 0x7fe2d8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.314+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe2e00af5a0 con 0x7fe2f00643a0 2024-03-18T15:03:40.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.314+0000 7fe2ef7fe640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 0x7fe2d8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.315+0000 7fe2ef7fe640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 0x7fe2d8070300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2f010e590 tx=0x7fe2e400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:40.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.319+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2e0078ac0 con 0x7fe2f00643a0 2024-03-18T15:03:40.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.542+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fe2f004bcc0 con 0x7fe2f00643a0 2024-03-18T15:03:40.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.543+0000 7fe2ecff9640 1 -- 172.21.15.12:0/654976570 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4205 (secure 0 0 0) 0x7fe2e007c180 con 0x7fe2f00643a0 2024-03-18T15:03:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:40.543 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:01.646053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:40.544 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2024-03-18T15:03:40.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 msgr2=0x7fe2d8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 0x7fe2d8070300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe2f010e590 tx=0x7fe2e400b040 comp rx=0 tx=0).stop 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 msgr2=0x7fe2f0112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0004930 tx=0x7fe2e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 shutdown_connections 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f0102240 0x7fe2f0113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fe2d806de50 0x7fe2d8070300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 --2- 172.21.15.12:0/654976570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f00643a0 0x7fe2f0112cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 >> 172.21.15.12:0/654976570 conn(0x7fe2f00710b0 msgr2=0x7fe2f00f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 shutdown_connections 2024-03-18T15:03:40.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.545+0000 7fe2f6a2e640 1 -- 172.21.15.12:0/654976570 wait complete. 2024-03-18T15:03:40.556 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-18T15:03:40.556 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-18T15:03:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- 172.21.15.12:0/4284512051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c101a80 msgr2=0x7fdb8c0fb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 --2- 172.21.15.12:0/4284512051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c101a80 0x7fdb8c0fb110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb800048d0 tx=0x7fdb80009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- 172.21.15.12:0/4284512051 shutdown_connections 2024-03-18T15:03:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 --2- 172.21.15.12:0/4284512051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c101a80 0x7fdb8c0fb110 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 --2- 172.21.15.12:0/4284512051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- 172.21.15.12:0/4284512051 >> 172.21.15.12:0/4284512051 conn(0x7fdb8c0f6cb0 msgr2=0x7fdb8c0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- 172.21.15.12:0/4284512051 shutdown_connections 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- 172.21.15.12:0/4284512051 wait complete. 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 Processor -- start 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- start start 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c101a80 0x7fdb8c112100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c112700 con 0x7fdb8c1010e0 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.656+0000 7fdb9159d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10bd50 con 0x7fdb8c101a80 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54228/0 (socket says 172.21.15.12:54228) 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 -- 172.21.15.12:0/3649680803 learned_addr learned my addr 172.21.15.12:0/3649680803 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8a7fc640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c101a80 0x7fdb8c112100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 -- 172.21.15.12:0/3649680803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c101a80 msgr2=0x7fdb8c112100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c101a80 0x7fdb8c112100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 -- 172.21.15.12:0/3649680803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb80005040 con 0x7fdb8c1010e0 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb8affd640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c002a50 tx=0x7fdb7c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c007c30 con 0x7fdb8c1010e0 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7c00f040 con 0x7fdb8c1010e0 2024-03-18T15:03:40.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c10bf10 con 0x7fdb8c1010e0 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c00dbe0 con 0x7fdb8c1010e0 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.657+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c10c400 con 0x7fdb8c1010e0 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.658+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c0fc850 con 0x7fdb8c1010e0 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.659+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fdb7c00dd40 con 0x7fdb8c1010e0 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.659+0000 7fdb737fe640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 0x7fdb74070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:40.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.659+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdb7c08d5f0 con 0x7fdb8c1010e0 2024-03-18T15:03:40.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.659+0000 7fdb8a7fc640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 0x7fdb74070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:40.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.660+0000 7fdb8a7fc640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 0x7fdb74070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0f96c0 tx=0x7fdb80009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:40.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.664+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb7c056ae0 con 0x7fdb8c1010e0 2024-03-18T15:03:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.888+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fdb8c10d260 con 0x7fdb8c1010e0 2024-03-18T15:03:40.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.889+0000 7fdb737fe640 1 -- 172.21.15.12:0/3649680803 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4216 (secure 0 0 0) 0x7fdb7c05a1a0 con 0x7fdb8c1010e0 2024-03-18T15:03:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:40.890 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:01.651536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:40.890 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2024-03-18T15:03:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 msgr2=0x7fdb74070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 0x7fdb74070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0f96c0 tx=0x7fdb80009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 msgr2=0x7fdb8c111bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c002a50 tx=0x7fdb7c004600 comp rx=0 tx=0).stop 2024-03-18T15:03:40.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 shutdown_connections 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c101a80 0x7fdb8c112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdb7406de50 0x7fdb74070300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 --2- 172.21.15.12:0/3649680803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb8c1010e0 0x7fdb8c111bc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 >> 172.21.15.12:0/3649680803 conn(0x7fdb8c0f6cb0 msgr2=0x7fdb8c0f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 shutdown_connections 2024-03-18T15:03:40.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:40.891+0000 7fdb9159d640 1 -- 172.21.15.12:0/3649680803 wait complete. 2024-03-18T15:03:40.903 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-18T15:03:40.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 -- 172.21.15.12:0/1130229829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b80fb110 msgr2=0x7f21b80fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1130229829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b80fb110 0x7f21b80fb560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0048d0 tx=0x7f21ac009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 -- 172.21.15.12:0/1130229829 shutdown_connections 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1130229829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b80fb110 0x7f21b80fb560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1130229829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b8101140 0x7f21b8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 -- 172.21.15.12:0/1130229829 >> 172.21.15.12:0/1130229829 conn(0x7f21b80710b0 msgr2=0x7f21b80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 -- 172.21.15.12:0/1130229829 shutdown_connections 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.024+0000 7f21bd46e640 1 -- 172.21.15.12:0/1130229829 wait complete. 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 Processor -- start 2024-03-18T15:03:41.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 -- start start 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 0x7f21b8111c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b8112740 con 0x7f21b8101140 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21bd46e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b810bd90 con 0x7f21b80fb110 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54250/0 (socket says 172.21.15.12:54250) 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 -- 172.21.15.12:0/1111339535 learned_addr learned my addr 172.21.15.12:0/1111339535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b6ffd640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 0x7f21b8111c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 -- 172.21.15.12:0/1111339535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 msgr2=0x7f21b8111c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 0x7f21b8111c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b67fc640 1 -- 172.21.15.12:0/1111339535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac005040 con 0x7f21b8101140 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.025+0000 7f21b6ffd640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 0x7f21b8111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:41.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f21b67fc640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0041d0 tx=0x7f21ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21ac03e070 con 0x7f21b8101140 2024-03-18T15:03:41.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21ac030870 con 0x7f21b8101140 2024-03-18T15:03:41.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b810c010 con 0x7f21b8101140 2024-03-18T15:03:41.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21ac02f5f0 con 0x7f21b8101140 2024-03-18T15:03:41.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.026+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b810c4d0 con 0x7f21b8101140 2024-03-18T15:03:41.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.027+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b80fc800 con 0x7f21b8101140 2024-03-18T15:03:41.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.027+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f21ac038a20 con 0x7f21b8101140 2024-03-18T15:03:41.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.028+0000 7f219f7fe640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 0x7f21a0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.028+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f21ac0b1460 con 0x7f21b8101140 2024-03-18T15:03:41.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.028+0000 7f21b6ffd640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 0x7f21a0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.028+0000 7f21b6ffd640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 0x7f21a0070310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21a80041d0 tx=0x7f21a8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.033+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21ac07aa10 con 0x7f21b8101140 2024-03-18T15:03:41.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:41 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2053909710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-18T15:03:41.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:41 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/654976570' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-18T15:03:41.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:41 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3649680803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-18T15:03:41.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.286+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f21b8108250 con 0x7f21b8101140 2024-03-18T15:03:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.286+0000 7f219f7fe640 1 -- 172.21.15.12:0/1111339535 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4221 (secure 0 0 0) 0x7f21ac07e0d0 con 0x7f21b8101140 2024-03-18T15:03:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:41.287 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:09.725475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":186,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:41.287 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-03-18T15:03:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 msgr2=0x7f21a0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 0x7f21a0070310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21a80041d0 tx=0x7f21a8009040 comp rx=0 tx=0).stop 2024-03-18T15:03:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 msgr2=0x7f21b8112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0041d0 tx=0x7f21ac009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 shutdown_connections 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f21a006de60 0x7f21a0070310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21b8101140 0x7f21b8112140 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 --2- 172.21.15.12:0/1111339535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b80fb110 0x7f21b8111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 >> 172.21.15.12:0/1111339535 conn(0x7f21b80710b0 msgr2=0x7f21b80f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 shutdown_connections 2024-03-18T15:03:41.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.288+0000 7f21bd46e640 1 -- 172.21.15.12:0/1111339535 wait complete. 2024-03-18T15:03:41.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:41 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2053909710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-18T15:03:41.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:41 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/654976570' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-18T15:03:41.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:41 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3649680803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-18T15:03:41.300 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-18T15:03:41.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 -- 172.21.15.12:0/3986252430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9841022d0 msgr2=0x7fb9841026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/3986252430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9841022d0 0x7fb9841026a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb9800048d0 tx=0x7fb980009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 -- 172.21.15.12:0/3986252430 shutdown_connections 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/3986252430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb984102c70 0x7fb9840fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/3986252430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9841022d0 0x7fb9841026a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 -- 172.21.15.12:0/3986252430 >> 172.21.15.12:0/3986252430 conn(0x7fb9840f7d90 msgr2=0x7fb9840fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 -- 172.21.15.12:0/3986252430 shutdown_connections 2024-03-18T15:03:41.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.413+0000 7fb98b2ae640 1 -- 172.21.15.12:0/3986252430 wait complete. 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 Processor -- start 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 -- start start 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9841022d0 0x7fb984112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984113920 con 0x7fb984102c70 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98b2ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98410cf70 con 0x7fb9841022d0 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb988849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb988849640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54274/0 (socket says 172.21.15.12:54274) 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb988849640 1 -- 172.21.15.12:0/926954627 learned_addr learned my addr 172.21.15.12:0/926954627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb98904a640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9841022d0 0x7fb984112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.414+0000 7fb988849640 1 -- 172.21.15.12:0/926954627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9841022d0 msgr2=0x7fb984112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb988849640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9841022d0 0x7fb984112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb988849640 1 -- 172.21.15.12:0/926954627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb980005040 con 0x7fb984102c70 2024-03-18T15:03:41.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb988849640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb974002a50 tx=0x7fb974004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb974007c30 con 0x7fb984102c70 2024-03-18T15:03:41.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb97400f040 con 0x7fb984102c70 2024-03-18T15:03:41.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97400db50 con 0x7fb984102c70 2024-03-18T15:03:41.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98410d250 con 0x7fb984102c70 2024-03-18T15:03:41.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.415+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98410d720 con 0x7fb984102c70 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.416+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98404bcc0 con 0x7fb984102c70 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.417+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fb97401c400 con 0x7fb984102c70 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.417+0000 7fb979ffb640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 0x7fb970070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.417+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb97408cfa0 con 0x7fb984102c70 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.417+0000 7fb98904a640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 0x7fb970070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.418+0000 7fb98904a640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 0x7fb970070300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb980003080 tx=0x7fb980009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.422+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9740c1740 con 0x7fb984102c70 2024-03-18T15:03:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.647+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fb9840fd930 con 0x7fb984102c70 2024-03-18T15:03:41.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.648+0000 7fb979ffb640 1 -- 172.21.15.12:0/926954627 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4218 (secure 0 0 0) 0x7fb9740564c0 con 0x7fb984102c70 2024-03-18T15:03:41.649 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:41.649 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:09.972208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":187,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:41.649 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 msgr2=0x7fb970070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 0x7fb970070300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb980003080 tx=0x7fb980009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 msgr2=0x7fb984113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb974002a50 tx=0x7fb974004600 comp rx=0 tx=0).stop 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 shutdown_connections 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fb97006de50 0x7fb970070300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb984102c70 0x7fb984113320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 --2- 172.21.15.12:0/926954627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9841022d0 0x7fb984112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 >> 172.21.15.12:0/926954627 conn(0x7fb9840f7d90 msgr2=0x7fb9840f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 shutdown_connections 2024-03-18T15:03:41.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.650+0000 7fb98b2ae640 1 -- 172.21.15.12:0/926954627 wait complete. 2024-03-18T15:03:41.661 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-18T15:03:41.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- 172.21.15.12:0/204329016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 msgr2=0x7f9c380fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/204329016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c380fb560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0048d0 tx=0x7f9c2c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- 172.21.15.12:0/204329016 shutdown_connections 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/204329016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c380fb560 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/204329016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 0x7f9c38101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- 172.21.15.12:0/204329016 >> 172.21.15.12:0/204329016 conn(0x7f9c380710b0 msgr2=0x7f9c380714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- 172.21.15.12:0/204329016 shutdown_connections 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- 172.21.15.12:0/204329016 wait complete. 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 Processor -- start 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.766+0000 7f9c3f35d640 1 -- start start 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3f35d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3f35d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 0x7f9c38113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3f35d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c38113830 con 0x7f9c380fb110 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3f35d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381139a0 con 0x7f9c38101140 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54280/0 (socket says 172.21.15.12:54280) 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 -- 172.21.15.12:0/408982881 learned_addr learned my addr 172.21.15.12:0/408982881 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3c8f8640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 0x7f9c38113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 -- 172.21.15.12:0/408982881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 msgr2=0x7f9c38113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 0x7f9c38113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 -- 172.21.15.12:0/408982881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c005040 con 0x7f9c380fb110 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3d0f9640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000db00 tx=0x7f9c30004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c30017070 con 0x7f9c380fb110 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3810d030 con 0x7f9c380fb110 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c3000f040 con 0x7f9c380fb110 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.767+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c30003ae0 con 0x7f9c380fb110 2024-03-18T15:03:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.768+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3810d520 con 0x7f9c380fb110 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.768+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c380fc800 con 0x7f9c380fb110 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.769+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9c30003c40 con 0x7f9c380fb110 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.769+0000 7f9c35ffb640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 0x7f9c20070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.769+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9c3008d860 con 0x7f9c380fb110 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.769+0000 7f9c3c8f8640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 0x7f9c20070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.770+0000 7f9c3c8f8640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 0x7f9c20070300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:41.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:41.774+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9c30056cd0 con 0x7f9c380fb110 2024-03-18T15:03:42.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.020+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f9c3804bcc0 con 0x7f9c380fb110 2024-03-18T15:03:42.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.021+0000 7f9c35ffb640 1 -- 172.21.15.12:0/408982881 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+4227 (secure 0 0 0) 0x7f9c3005a390 con 0x7f9c380fb110 2024-03-18T15:03:42.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:42.022 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:11.976061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":0,"incarnation":26,"state":"up:active","state_seq":188,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-03-18T15:03:42.022 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 msgr2=0x7f9c20070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 0x7f9c20070300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 msgr2=0x7f9c38112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000db00 tx=0x7f9c30004e60 comp rx=0 tx=0).stop 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 shutdown_connections 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c38101140 0x7f9c38113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9c2006de50 0x7f9c20070300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 --2- 172.21.15.12:0/408982881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c380fb110 0x7f9c38112cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 >> 172.21.15.12:0/408982881 conn(0x7f9c380710b0 msgr2=0x7f9c38064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 shutdown_connections 2024-03-18T15:03:42.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.024+0000 7f9c3f35d640 1 -- 172.21.15.12:0/408982881 wait complete. 2024-03-18T15:03:42.037 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-18T15:03:42.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-18T15:03:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 -- 172.21.15.12:0/868924688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 msgr2=0x7f1050102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 --2- 172.21.15.12:0/868924688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050102610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f10440048d0 tx=0x7f1044009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 -- 172.21.15.12:0/868924688 shutdown_connections 2024-03-18T15:03:42.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 --2- 172.21.15.12:0/868924688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f10500647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 --2- 172.21.15.12:0/868924688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050102610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 -- 172.21.15.12:0/868924688 >> 172.21.15.12:0/868924688 conn(0x7f10500710b0 msgr2=0x7f10500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.139+0000 7f105711d640 1 -- 172.21.15.12:0/868924688 shutdown_connections 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 -- 172.21.15.12:0/868924688 wait complete. 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 Processor -- start 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 -- start start 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f1050112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050113810 con 0x7f1050102240 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.140+0000 7f105711d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105010ce60 con 0x7f10500643a0 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54296/0 (socket says 172.21.15.12:54296) 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 -- 172.21.15.12:0/2852391036 learned_addr learned my addr 172.21.15.12:0/2852391036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f1054eb9640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f1050112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 -- 172.21.15.12:0/2852391036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 msgr2=0x7f1050112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f1050112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 -- 172.21.15.12:0/2852391036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044005040 con 0x7f1050102240 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f1054eb9640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f1050112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:03:42.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104ffff640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f103c00eb70 tx=0x7f103c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c007c30 con 0x7f1050102240 2024-03-18T15:03:42.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f103c00f040 con 0x7f1050102240 2024-03-18T15:03:42.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105010d0e0 con 0x7f1050102240 2024-03-18T15:03:42.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c0095b0 con 0x7f1050102240 2024-03-18T15:03:42.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.141+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105010d600 con 0x7f1050102240 2024-03-18T15:03:42.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.142+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10500fd940 con 0x7f1050102240 2024-03-18T15:03:42.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.143+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f103c009730 con 0x7f1050102240 2024-03-18T15:03:42.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.143+0000 7f104d7fa640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 0x7f1040070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.143+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f103c08d520 con 0x7f1050102240 2024-03-18T15:03:42.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.143+0000 7f1054eb9640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 0x7f1040070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.144+0000 7f1054eb9640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 0x7f1040070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1044000c00 tx=0x7f1044009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.148+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f103c056990 con 0x7f1050102240 2024-03-18T15:03:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:42 smithi100 ceph-mon[84868]: pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:42 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1111339535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-18T15:03:42.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:42 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/926954627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-18T15:03:42.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.370+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f1050109320 con 0x7f1050102240 2024-03-18T15:03:42.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.371+0000 7f104d7fa640 1 -- 172.21.15.12:0/2852391036 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+5077 (secure 0 0 0) 0x7f103c05a050 con 0x7f1050102240 2024-03-18T15:03:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:42.372 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:11.976061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi100.rutxmj","rank":0,"incarnation":26,"state":"up:active","state_seq":188,"addr":"172.21.15.100:6827/2597653923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2597653923},{"type":"v1","addr":"172.21.15.100:6827","nonce":2597653923}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-03-18T15:03:42.372 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 msgr2=0x7f1040070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 0x7f1040070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1044000c00 tx=0x7f1044009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 msgr2=0x7f1050113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f103c00eb70 tx=0x7f103c00b040 comp rx=0 tx=0).stop 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 shutdown_connections 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f104006de50 0x7f1040070300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1050102240 0x7f1050113210 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 --2- 172.21.15.12:0/2852391036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10500643a0 0x7f1050112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 >> 172.21.15.12:0/2852391036 conn(0x7f10500710b0 msgr2=0x7f10500f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 shutdown_connections 2024-03-18T15:03:42.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.373+0000 7f105711d640 1 -- 172.21.15.12:0/2852391036 wait complete. 2024-03-18T15:03:42.383 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:42 smithi012 ceph-mon[111904]: pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-18T15:03:42.384 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:42 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1111339535' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-18T15:03:42.384 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:42 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/926954627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-18T15:03:42.385 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-18T15:03:42.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-18T15:03:42.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- 172.21.15.12:0/3581571330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 msgr2=0x7ffbac0647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 --2- 172.21.15.12:0/3581571330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac0647f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0048d0 tx=0x7ffb9c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- 172.21.15.12:0/3581571330 shutdown_connections 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 --2- 172.21.15.12:0/3581571330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac0647f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 --2- 172.21.15.12:0/3581571330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 0x7ffbac1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- 172.21.15.12:0/3581571330 >> 172.21.15.12:0/3581571330 conn(0x7ffbac0710b0 msgr2=0x7ffbac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- 172.21.15.12:0/3581571330 shutdown_connections 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- 172.21.15.12:0/3581571330 wait complete. 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 Processor -- start 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.484+0000 7ffbb391d640 1 -- start start 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb391d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb391d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 0x7ffbac113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb391d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac113820 con 0x7ffbac0643a0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb391d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac10ce70 con 0x7ffbac102210 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54310/0 (socket says 172.21.15.12:54310) 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 -- 172.21.15.12:0/1078740549 learned_addr learned my addr 172.21.15.12:0/1078740549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb0eb8640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 0x7ffbac113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 -- 172.21.15.12:0/1078740549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 msgr2=0x7ffbac113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 0x7ffbac113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 -- 172.21.15.12:0/1078740549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c005040 con 0x7ffbac0643a0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb16b9640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffba8002a50 tx=0x7ffba8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffba8007c30 con 0x7ffbac0643a0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac10d150 con 0x7ffbac0643a0 2024-03-18T15:03:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffba800f040 con 0x7ffbac0643a0 2024-03-18T15:03:42.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.485+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffba800db50 con 0x7ffbac0643a0 2024-03-18T15:03:42.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.486+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac10d640 con 0x7ffbac0643a0 2024-03-18T15:03:42.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.487+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac0fd910 con 0x7ffbac0643a0 2024-03-18T15:03:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.487+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7ffba801c400 con 0x7ffbac0643a0 2024-03-18T15:03:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.487+0000 7ffba1ffb640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 0x7ffb98070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.488+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffba808cfa0 con 0x7ffbac0643a0 2024-03-18T15:03:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.488+0000 7ffbb0eb8640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 0x7ffb98070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.488+0000 7ffbb0eb8640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 0x7ffb98070300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0fb090 tx=0x7ffb9c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.496+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffba80c1740 con 0x7ffbac0643a0 2024-03-18T15:03:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.741+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7ffbac10e350 con 0x7ffbac0643a0 2024-03-18T15:03:42.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.742+0000 7ffba1ffb640 1 -- 172.21.15.12:0/1078740549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+4273 (secure 0 0 0) 0x7ffba80564c0 con 0x7ffbac0643a0 2024-03-18T15:03:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:42.743 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:27.308610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:42.743 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 msgr2=0x7ffb98070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 0x7ffb98070300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0fb090 tx=0x7ffb9c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 msgr2=0x7ffbac112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffba8002a50 tx=0x7ffba8004600 comp rx=0 tx=0).stop 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 shutdown_connections 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac102210 0x7ffbac113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7ffb9806de50 0x7ffb98070300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 --2- 172.21.15.12:0/1078740549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbac0643a0 0x7ffbac112ce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 >> 172.21.15.12:0/1078740549 conn(0x7ffbac0710b0 msgr2=0x7ffbac0f95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 shutdown_connections 2024-03-18T15:03:42.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.745+0000 7ffbb391d640 1 -- 172.21.15.12:0/1078740549 wait complete. 2024-03-18T15:03:42.757 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-18T15:03:42.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-18T15:03:42.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 -- 172.21.15.12:0/3185645388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded806f810 msgr2=0x7fded806fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 --2- 172.21.15.12:0/3185645388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded806f810 0x7fded806fbe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdec80048d0 tx=0x7fdec8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 -- 172.21.15.12:0/3185645388 shutdown_connections 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 --2- 172.21.15.12:0/3185645388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded8070120 0x7fded81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 --2- 172.21.15.12:0/3185645388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded806f810 0x7fded806fbe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 -- 172.21.15.12:0/3185645388 >> 172.21.15.12:0/3185645388 conn(0x7fded80f7e60 msgr2=0x7fded80fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 -- 172.21.15.12:0/3185645388 shutdown_connections 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.855+0000 7fdedd19d640 1 -- 172.21.15.12:0/3185645388 wait complete. 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 Processor -- start 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 -- start start 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded806f810 0x7fded8114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded8115a70 con 0x7fded8070120 2024-03-18T15:03:42.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fdedd19d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded810f110 con 0x7fded806f810 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54326/0 (socket says 172.21.15.12:54326) 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 -- 172.21.15.12:0/4035418826 learned_addr learned my addr 172.21.15.12:0/4035418826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded6d9d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded806f810 0x7fded8114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 -- 172.21.15.12:0/4035418826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded806f810 msgr2=0x7fded8114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded806f810 0x7fded8114f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.856+0000 7fded659c640 1 -- 172.21.15.12:0/4035418826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8005040 con 0x7fded8070120 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fded659c640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdecc002a50 tx=0x7fdecc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdecc007c30 con 0x7fded8070120 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded810f3f0 con 0x7fded8070120 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdecc00f040 con 0x7fded8070120 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdecc00db50 con 0x7fded8070120 2024-03-18T15:03:42.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.857+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded810f910 con 0x7fded8070120 2024-03-18T15:03:42.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.858+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded8070f50 con 0x7fded8070120 2024-03-18T15:03:42.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.858+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fdecc01c400 con 0x7fded8070120 2024-03-18T15:03:42.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.859+0000 7fdebf7fe640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 0x7fdec0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:42.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.859+0000 7fded6d9d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 0x7fdec0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:42.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.859+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdecc08d050 con 0x7fded8070120 2024-03-18T15:03:42.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.860+0000 7fded6d9d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 0x7fdec0070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdec8003080 tx=0x7fdec8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:42.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:42.864+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdecc0564c0 con 0x7fded8070120 2024-03-18T15:03:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.088+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fded810fb30 con 0x7fded8070120 2024-03-18T15:03:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.089+0000 7fdebf7fe640 1 -- 172.21.15.12:0/4035418826 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+4352 (secure 0 0 0) 0x7fdecc059b80 con 0x7fded8070120 2024-03-18T15:03:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:43.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:27.315552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2024-03-18T15:03:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 msgr2=0x7fdec0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 0x7fdec0070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdec8003080 tx=0x7fdec8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 msgr2=0x7fded8115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdecc002a50 tx=0x7fdecc004600 comp rx=0 tx=0).stop 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 shutdown_connections 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdec006de50 0x7fdec0070300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded8070120 0x7fded8115470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 --2- 172.21.15.12:0/4035418826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded806f810 0x7fded8114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 >> 172.21.15.12:0/4035418826 conn(0x7fded80f7e60 msgr2=0x7fded8104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 shutdown_connections 2024-03-18T15:03:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.091+0000 7fdedd19d640 1 -- 172.21.15.12:0/4035418826 wait complete. 2024-03-18T15:03:43.102 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-18T15:03:43.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 -- 172.21.15.12:0/195565074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 msgr2=0x7fea600647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 --2- 172.21.15.12:0/195565074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea600647d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fea480048d0 tx=0x7fea48009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 -- 172.21.15.12:0/195565074 shutdown_connections 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 --2- 172.21.15.12:0/195565074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea600647d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 --2- 172.21.15.12:0/195565074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea60102240 0x7fea60102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 -- 172.21.15.12:0/195565074 >> 172.21.15.12:0/195565074 conn(0x7fea600710b0 msgr2=0x7fea600714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 -- 172.21.15.12:0/195565074 shutdown_connections 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.230+0000 7fea6486e640 1 -- 172.21.15.12:0/195565074 wait complete. 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 Processor -- start 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 -- start start 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea60102240 0x7fea60113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60113810 con 0x7fea600643a0 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6010ce60 con 0x7fea60102240 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54348/0 (socket says 172.21.15.12:54348) 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 -- 172.21.15.12:0/2087008161 learned_addr learned my addr 172.21.15.12:0/2087008161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 -- 172.21.15.12:0/2087008161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea60102240 msgr2=0x7fea60113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea60102240 0x7fea60113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 -- 172.21.15.12:0/2087008161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48005040 con 0x7fea600643a0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea5e59c640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea50002a50 tx=0x7fea50004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea50007c30 con 0x7fea600643a0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6010d140 con 0x7fea600643a0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea5000f040 con 0x7fea600643a0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea5000db50 con 0x7fea600643a0 2024-03-18T15:03:43.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.231+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6010d660 con 0x7fea600643a0 2024-03-18T15:03:43.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.233+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea600fd940 con 0x7fea600643a0 2024-03-18T15:03:43.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.233+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fea5001c400 con 0x7fea600643a0 2024-03-18T15:03:43.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.233+0000 7fea56ffd640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 0x7fea4c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.234+0000 7fea57fff640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 0x7fea4c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.234+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fea5008d050 con 0x7fea600643a0 2024-03-18T15:03:43.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.234+0000 7fea57fff640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 0x7fea4c070300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea480043d0 tx=0x7fea48009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.238+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea500564c0 con 0x7fea600643a0 2024-03-18T15:03:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:43 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/408982881' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-18T15:03:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:43 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2852391036' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-18T15:03:43.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:43 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1078740549' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-18T15:03:43.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:43 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/408982881' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-18T15:03:43.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:43 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2852391036' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-18T15:03:43.491 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:43 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1078740549' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-18T15:03:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.495+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fea6004bcc0 con 0x7fea600643a0 2024-03-18T15:03:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.496+0000 7fea56ffd640 1 -- 172.21.15.12:0/2087008161 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+4356 (secure 0 0 0) 0x7fea50059b80 con 0x7fea600643a0 2024-03-18T15:03:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:43.496 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:35.491607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:43.497 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 msgr2=0x7fea4c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 0x7fea4c070300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea480043d0 tx=0x7fea48009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 msgr2=0x7fea60112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea50002a50 tx=0x7fea50004600 comp rx=0 tx=0).stop 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 shutdown_connections 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea60102240 0x7fea60113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fea4c06de50 0x7fea4c070300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 --2- 172.21.15.12:0/2087008161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea600643a0 0x7fea60112cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 >> 172.21.15.12:0/2087008161 conn(0x7fea600710b0 msgr2=0x7fea600f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 shutdown_connections 2024-03-18T15:03:43.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.498+0000 7fea6486e640 1 -- 172.21.15.12:0/2087008161 wait complete. 2024-03-18T15:03:43.513 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-18T15:03:43.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-18T15:03:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 -- 172.21.15.12:0/4179290362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde81012d0 msgr2=0x7fcde81016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 --2- 172.21.15.12:0/4179290362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde81012d0 0x7fcde81016a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80048d0 tx=0x7fcdd8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 -- 172.21.15.12:0/4179290362 shutdown_connections 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 --2- 172.21.15.12:0/4179290362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde8101c70 0x7fcde80fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 --2- 172.21.15.12:0/4179290362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde81012d0 0x7fcde81016a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 -- 172.21.15.12:0/4179290362 >> 172.21.15.12:0/4179290362 conn(0x7fcde80f6ea0 msgr2=0x7fcde80f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 -- 172.21.15.12:0/4179290362 shutdown_connections 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 -- 172.21.15.12:0/4179290362 wait complete. 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.617+0000 7fcdee79d640 1 Processor -- start 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcdee79d640 1 -- start start 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcdee79d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde81012d0 0x7fcde8115190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcdee79d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcdee79d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8115cd0 con 0x7fcde8101c70 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcdee79d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde810f320 con 0x7fcde81012d0 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54366/0 (socket says 172.21.15.12:54366) 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 -- 172.21.15.12:0/936135148 learned_addr learned my addr 172.21.15.12:0/936135148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde7fff640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde81012d0 0x7fcde8115190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 -- 172.21.15.12:0/936135148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde81012d0 msgr2=0x7fcde8115190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde81012d0 0x7fcde8115190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 -- 172.21.15.12:0/936135148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8005040 con 0x7fcde8101c70 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.618+0000 7fcde77fe640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00c790 tx=0x7fcddc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.619+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcddc00a820 con 0x7fcde8101c70 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.619+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde810f600 con 0x7fcde8101c70 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.619+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcddc00ae50 con 0x7fcde8101c70 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.619+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcddc0115e0 con 0x7fcde8101c70 2024-03-18T15:03:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.619+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde810fb20 con 0x7fcde8101c70 2024-03-18T15:03:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.620+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde80fca40 con 0x7fcde8101c70 2024-03-18T15:03:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.620+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcddc012400 con 0x7fcde8101c70 2024-03-18T15:03:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.621+0000 7fcde4ff9640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 0x7fcdd0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.621+0000 7fcde7fff640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 0x7fcdd0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.621+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcddc08d8a0 con 0x7fcde8101c70 2024-03-18T15:03:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.621+0000 7fcde7fff640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 0x7fcdd0070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8031870 tx=0x7fcdd8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.625+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcddc056d90 con 0x7fcde8101c70 2024-03-18T15:03:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.854+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fcde8108430 con 0x7fcde8101c70 2024-03-18T15:03:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.855+0000 7fcde4ff9640 1 -- 172.21.15.12:0/936135148 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+4353 (secure 0 0 0) 0x7fcddc05a450 con 0x7fcde8101c70 2024-03-18T15:03:43.856 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:43.856 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:35.560843+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-18T15:03:43.856 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 msgr2=0x7fcdd0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 0x7fcdd0070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8031870 tx=0x7fcdd8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 msgr2=0x7fcde81156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00c790 tx=0x7fcddc0043c0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 shutdown_connections 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdd006de50 0x7fcdd0070300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcde8101c70 0x7fcde81156d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 --2- 172.21.15.12:0/936135148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde81012d0 0x7fcde8115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 >> 172.21.15.12:0/936135148 conn(0x7fcde80f6ea0 msgr2=0x7fcde80f84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 shutdown_connections 2024-03-18T15:03:43.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.857+0000 7fcdee79d640 1 -- 172.21.15.12:0/936135148 wait complete. 2024-03-18T15:03:43.869 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-18T15:03:43.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 -- 172.21.15.12:0/624670451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 msgr2=0x7f0d201047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 --2- 172.21.15.12:0/624670451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d201047c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0048d0 tx=0x7f0d1c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 -- 172.21.15.12:0/624670451 shutdown_connections 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 --2- 172.21.15.12:0/624670451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d200fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 --2- 172.21.15.12:0/624670451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d201047c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.967+0000 7f0d2815d640 1 -- 172.21.15.12:0/624670451 >> 172.21.15.12:0/624670451 conn(0x7f0d200f9fc0 msgr2=0x7f0d200fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 -- 172.21.15.12:0/624670451 shutdown_connections 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 -- 172.21.15.12:0/624670451 wait complete. 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 Processor -- start 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 -- start start 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d20115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20115a20 con 0x7f0d201043f0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d2815d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2010f070 con 0x7f0d20104d90 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d25ef9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d25ef9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54372/0 (socket says 172.21.15.12:54372) 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d25ef9640 1 -- 172.21.15.12:0/1321873061 learned_addr learned my addr 172.21.15.12:0/1321873061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.968+0000 7f0d256f8640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d20115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d25ef9640 1 -- 172.21.15.12:0/1321873061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 msgr2=0x7f0d20115420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d25ef9640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d20115420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d25ef9640 1 -- 172.21.15.12:0/1321873061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c005040 con 0x7f0d201043f0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d256f8640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d20115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d25ef9640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c034040 tx=0x7f0d1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c0092b0 con 0x7f0d201043f0 2024-03-18T15:03:43.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d1c03b400 con 0x7f0d201043f0 2024-03-18T15:03:43.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c0435d0 con 0x7f0d201043f0 2024-03-18T15:03:43.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2010f2f0 con 0x7f0d201043f0 2024-03-18T15:03:43.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.969+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2010f7b0 con 0x7f0d201043f0 2024-03-18T15:03:43.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.970+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d200ffb60 con 0x7f0d201043f0 2024-03-18T15:03:43.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.970+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f0d1c02f070 con 0x7f0d201043f0 2024-03-18T15:03:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.971+0000 7f0d167fc640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 0x7f0d0c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.971+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0d1c0af400 con 0x7f0d201043f0 2024-03-18T15:03:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.971+0000 7f0d256f8640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 0x7f0d0c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:43.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.971+0000 7f0d256f8640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 0x7f0d0c070300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d201107a0 tx=0x7f0d1000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:43.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:43.976+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d1c0788f0 con 0x7f0d201043f0 2024-03-18T15:03:44.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.221+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f0d2004bcc0 con 0x7f0d201043f0 2024-03-18T15:03:44.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.221+0000 7f0d167fc640 1 -- 172.21.15.12:0/1321873061 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+4362 (secure 0 0 0) 0x7f0d1c07bfb0 con 0x7f0d201043f0 2024-03-18T15:03:44.222 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:44.222 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:37.566653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-18T15:03:44.222 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 35 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 msgr2=0x7f0d0c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 0x7f0d0c070300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d201107a0 tx=0x7f0d1000b040 comp rx=0 tx=0).stop 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 msgr2=0x7f0d20114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c034040 tx=0x7f0d1c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 shutdown_connections 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d20104d90 0x7f0d20115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f0d0c06de50 0x7f0d0c070300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 --2- 172.21.15.12:0/1321873061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d201043f0 0x7f0d20114ee0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 >> 172.21.15.12:0/1321873061 conn(0x7f0d200f9fc0 msgr2=0x7f0d20107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 shutdown_connections 2024-03-18T15:03:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.223+0000 7f0d2815d640 1 -- 172.21.15.12:0/1321873061 wait complete. 2024-03-18T15:03:44.233 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:44 smithi012 ceph-mon[111904]: pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:44.234 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:44 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/4035418826' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-18T15:03:44.234 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:44 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2087008161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-18T15:03:44.234 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:44 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/936135148' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-18T15:03:44.235 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-03-18T15:03:44.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-18T15:03:44.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:44 smithi100 ceph-mon[84868]: pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:44 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/4035418826' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-18T15:03:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:44 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2087008161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-18T15:03:44.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:44 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/936135148' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 -- 172.21.15.12:0/3490473655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 msgr2=0x7f743c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 --2- 172.21.15.12:0/3490473655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c102610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74300048d0 tx=0x7f7430009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 -- 172.21.15.12:0/3490473655 shutdown_connections 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 --2- 172.21.15.12:0/3490473655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c0647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 --2- 172.21.15.12:0/3490473655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c102610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 -- 172.21.15.12:0/3490473655 >> 172.21.15.12:0/3490473655 conn(0x7f743c0710b0 msgr2=0x7f743c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 -- 172.21.15.12:0/3490473655 shutdown_connections 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.335+0000 7f744271d640 1 -- 172.21.15.12:0/3490473655 wait complete. 2024-03-18T15:03:44.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 Processor -- start 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 -- start start 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c113810 con 0x7f743c102240 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f744271d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c10ce60 con 0x7f743c0643a0 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54378/0 (socket says 172.21.15.12:54378) 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 -- 172.21.15.12:0/2551939884 learned_addr learned my addr 172.21.15.12:0/2551939884 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743bfff640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 -- 172.21.15.12:0/2551939884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 msgr2=0x7f743c112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743b7fe640 1 -- 172.21.15.12:0/2551939884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430005040 con 0x7f743c102240 2024-03-18T15:03:44.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.336+0000 7f743bfff640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f743b7fe640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f742c002a50 tx=0x7f742c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742c007c30 con 0x7f743c102240 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f742c00f040 con 0x7f743c102240 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c10d140 con 0x7f743c102240 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742c00db50 con 0x7f743c102240 2024-03-18T15:03:44.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.337+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c10d660 con 0x7f743c102240 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.338+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0fd940 con 0x7f743c102240 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.338+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f742c01c400 con 0x7f743c102240 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.339+0000 7f7438ff9640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 0x7f7424070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.339+0000 7f743bfff640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 0x7f7424070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.339+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f742c08d050 con 0x7f743c102240 2024-03-18T15:03:44.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.339+0000 7f743bfff640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 0x7f7424070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7430000c00 tx=0x7f7430009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:44.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.343+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f742c0564c0 con 0x7f743c102240 2024-03-18T15:03:44.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.604+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f743c109320 con 0x7f743c102240 2024-03-18T15:03:44.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.604+0000 7f7438ff9640 1 -- 172.21.15.12:0/2551939884 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+5212 (secure 0 0 0) 0x7f742c059b80 con 0x7f743c102240 2024-03-18T15:03:44.605 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:44.605 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/3606275918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3606275918},{"type":"v1","addr":"172.21.15.100:6827","nonce":3606275918}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:37.566653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-18T15:03:44.605 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2024-03-18T15:03:44.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 msgr2=0x7f7424070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 0x7f7424070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7430000c00 tx=0x7f7430009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 msgr2=0x7f743c113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f742c002a50 tx=0x7f742c004600 comp rx=0 tx=0).stop 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 shutdown_connections 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f742406de50 0x7f7424070300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f743c102240 0x7f743c113210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 --2- 172.21.15.12:0/2551939884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743c0643a0 0x7f743c112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 >> 172.21.15.12:0/2551939884 conn(0x7f743c0710b0 msgr2=0x7f743c0f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 shutdown_connections 2024-03-18T15:03:44.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.607+0000 7f744271d640 1 -- 172.21.15.12:0/2551939884 wait complete. 2024-03-18T15:03:44.618 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-03-18T15:03:44.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-18T15:03:44.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- 172.21.15.12:0/1442355687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8858102240 msgr2=0x7f8858102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 --2- 172.21.15.12:0/1442355687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8858102240 0x7f8858102610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8850009c50 tx=0x7f8850004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- 172.21.15.12:0/1442355687 shutdown_connections 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 --2- 172.21.15.12:0/1442355687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88580643a0 0x7f88580647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 --2- 172.21.15.12:0/1442355687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8858102240 0x7f8858102610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- 172.21.15.12:0/1442355687 >> 172.21.15.12:0/1442355687 conn(0x7f88580710b0 msgr2=0x7f88580714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- 172.21.15.12:0/1442355687 shutdown_connections 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- 172.21.15.12:0/1442355687 wait complete. 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 Processor -- start 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.719+0000 7f885f06e640 1 -- start start 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885f06e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885f06e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8858102240 0x7f8858113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885f06e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858113810 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885f06e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885810ce60 con 0x7f8858102240 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54388/0 (socket says 172.21.15.12:54388) 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 -- 172.21.15.12:0/3290198386 learned_addr learned my addr 172.21.15.12:0/3290198386 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f8857fff640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8858102240 0x7f8858113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 -- 172.21.15.12:0/3290198386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8858102240 msgr2=0x7f8858113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8858102240 0x7f8858113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 -- 172.21.15.12:0/3290198386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500098b0 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.720+0000 7f885ce0a640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8850004f80 tx=0x7f8850004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.721+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f885003a070 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.721+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8850009740 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.721+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885810d0e0 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.721+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f885003ea60 con 0x7f88580643a0 2024-03-18T15:03:44.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.721+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885810d680 con 0x7f88580643a0 2024-03-18T15:03:44.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.722+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88580fd940 con 0x7f88580643a0 2024-03-18T15:03:44.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.722+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f8850032070 con 0x7f88580643a0 2024-03-18T15:03:44.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.722+0000 7f88557fa640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 0x7f8844070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:44.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.723+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f88500af8b0 con 0x7f88580643a0 2024-03-18T15:03:44.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.723+0000 7f8857fff640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 0x7f8844070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:44.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.723+0000 7f8857fff640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 0x7f8844070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f885810e590 tx=0x7f884000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:44.727+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8850078dd0 con 0x7f88580643a0 2024-03-18T15:03:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.024+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f885810e3d0 con 0x7f88580643a0 2024-03-18T15:03:45.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.024+0000 7f88557fa640 1 -- 172.21.15.12:0/3290198386 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+5212 (secure 0 0 0) 0x7f885007c490 con 0x7f88580643a0 2024-03-18T15:03:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:45.025 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/3606275918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3606275918},{"type":"v1","addr":"172.21.15.100:6827","nonce":3606275918}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:45.528641+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-18T15:03:45.026 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 msgr2=0x7f8844070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 0x7f8844070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f885810e590 tx=0x7f884000b040 comp rx=0 tx=0).stop 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 msgr2=0x7f8858112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8850004f80 tx=0x7f8850004d10 comp rx=0 tx=0).stop 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 shutdown_connections 2024-03-18T15:03:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8858102240 0x7f8858113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f884406de50 0x7f8844070300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.026+0000 7f885f06e640 1 --2- 172.21.15.12:0/3290198386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88580643a0 0x7f8858112cd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.027+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 >> 172.21.15.12:0/3290198386 conn(0x7f88580710b0 msgr2=0x7f88580f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.027+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 shutdown_connections 2024-03-18T15:03:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.027+0000 7f885f06e640 1 -- 172.21.15.12:0/3290198386 wait complete. 2024-03-18T15:03:45.038 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-03-18T15:03:45.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-18T15:03:45.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- 172.21.15.12:0/878886386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fdba0 msgr2=0x7f70d81061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 --2- 172.21.15.12:0/878886386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fdba0 0x7f70d81061e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70c40048d0 tx=0x7f70c4009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- 172.21.15.12:0/878886386 shutdown_connections 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 --2- 172.21.15.12:0/878886386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fdba0 0x7f70d81061e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 --2- 172.21.15.12:0/878886386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fd200 0x7f70d80fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- 172.21.15.12:0/878886386 >> 172.21.15.12:0/878886386 conn(0x7f70d80f6cb0 msgr2=0x7f70d80f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- 172.21.15.12:0/878886386 shutdown_connections 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- 172.21.15.12:0/878886386 wait complete. 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 Processor -- start 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.137+0000 7f70decae640 1 -- start start 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70decae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70decae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 0x7f70d8113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70decae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8113880 con 0x7f70d80fd200 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70decae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d810ced0 con 0x7f70d80fdba0 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54396/0 (socket says 172.21.15.12:54396) 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 -- 172.21.15.12:0/2385323954 learned_addr learned my addr 172.21.15.12:0/2385323954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 -- 172.21.15.12:0/2385323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 msgr2=0x7f70d8113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70d7fff640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 0x7f70d8113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 0x7f70d8113280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 -- 172.21.15.12:0/2385323954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c4005040 con 0x7f70d80fd200 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70d7fff640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 0x7f70d8113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.138+0000 7f70dca4a640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00eb70 tx=0x7f70cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.139+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70cc007c30 con 0x7f70d80fd200 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.139+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70cc00f040 con 0x7f70d80fd200 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.139+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d810d150 con 0x7f70d80fd200 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.139+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70cc0095b0 con 0x7f70d80fd200 2024-03-18T15:03:45.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.139+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d810d670 con 0x7f70d80fd200 2024-03-18T15:03:45.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.140+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d8065980 con 0x7f70d80fd200 2024-03-18T15:03:45.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.140+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f70cc009730 con 0x7f70d80fd200 2024-03-18T15:03:45.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.140+0000 7f70d57fa640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 0x7f70c8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.141+0000 7f70d7fff640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 0x7f70c8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.141+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f70cc08d520 con 0x7f70d80fd200 2024-03-18T15:03:45.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.141+0000 7f70d7fff640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 0x7f70c8070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70c4000c00 tx=0x7f70c4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:45.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.145+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70cc056990 con 0x7f70d80fd200 2024-03-18T15:03:45.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:45 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1321873061' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-18T15:03:45.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:45 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2551939884' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-18T15:03:45.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.425+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f70d804bcc0 con 0x7f70d80fd200 2024-03-18T15:03:45.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.426+0000 7f70d57fa640 1 -- 172.21.15.12:0/2385323954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+5211 (secure 0 0 0) 0x7f70cc05a050 con 0x7f70d80fd200 2024-03-18T15:03:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi012.oheuui","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6829/846568565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":846568565},{"type":"v1","addr":"172.21.15.12:6829","nonce":846568565}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44321,"name":"cephfs.smithi100.dvtbrf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6825/3815597664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3815597664},{"type":"v1","addr":"172.21.15.100:6825","nonce":3815597664}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44343,"name":"cephfs.smithi100.rutxmj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6827/3606275918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3606275918},{"type":"v1","addr":"172.21.15.100:6827","nonce":3606275918}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T14:48:36.088832+0000","modified":"2024-03-18T15:01:46.531163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi012.xtadxo","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.12:6827/1804438217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1804438217},{"type":"v1","addr":"172.21.15.12:6827","nonce":1804438217}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-03-18T15:03:45.427 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2024-03-18T15:03:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 msgr2=0x7f70c8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 0x7f70c8070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70c4000c00 tx=0x7f70c4009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 msgr2=0x7f70d8112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00eb70 tx=0x7f70cc00b040 comp rx=0 tx=0).stop 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 shutdown_connections 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d80fdba0 0x7f70d8113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f70c806de50 0x7f70c8070300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 --2- 172.21.15.12:0/2385323954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70d80fd200 0x7f70d8112d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 >> 172.21.15.12:0/2385323954 conn(0x7f70d80f6cb0 msgr2=0x7f70d80f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.428+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 shutdown_connections 2024-03-18T15:03:45.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.429+0000 7f70decae640 1 -- 172.21.15.12:0/2385323954 wait complete. 2024-03-18T15:03:45.439 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:45 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1321873061' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-18T15:03:45.439 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:45 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2551939884' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-18T15:03:45.440 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-18T15:03:45.451 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T15:03:45.452 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T15:03:45.452 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:03:45.468 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-03-18T15:03:45.468 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:03:45.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/1445069918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d8101140 msgr2=0x7fa7d8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/1445069918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d8101140 0x7fa7d8101510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40048d0 tx=0x7fa7d4009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/1445069918 shutdown_connections 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/1445069918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80fb110 0x7fa7d80fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/1445069918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d8101140 0x7fa7d8101510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.661+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/1445069918 >> 172.21.15.12:0/1445069918 conn(0x7fa7d8071130 msgr2=0x7fa7d8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/1445069918 shutdown_connections 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/1445069918 wait complete. 2024-03-18T15:03:45.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 Processor -- start 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 -- start start 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8101140 0x7fa7d81153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81159e0 con 0x7fa7d80fb110 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7dfe2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d810f030 con 0x7fa7d8101140 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7ddbca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.662+0000 7fa7ddbca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54418/0 (socket says 172.21.15.12:54418) 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ddbca640 1 -- 172.21.15.12:0/3641064952 learned_addr learned my addr 172.21.15.12:0/3641064952 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7dd3c9640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8101140 0x7fa7d81153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ddbca640 1 -- 172.21.15.12:0/3641064952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8101140 msgr2=0x7fa7d81153e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ddbca640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8101140 0x7fa7d81153e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ddbca640 1 -- 172.21.15.12:0/3641064952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4005040 con 0x7fa7d80fb110 2024-03-18T15:03:45.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ddbca640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40041d0 tx=0x7fa7d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d4030870 con 0x7fa7d80fb110 2024-03-18T15:03:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7d40092e0 con 0x7fa7d80fb110 2024-03-18T15:03:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d4043400 con 0x7fa7d80fb110 2024-03-18T15:03:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d810f2b0 con 0x7fa7d80fb110 2024-03-18T15:03:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.663+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d810f7d0 con 0x7fa7d80fb110 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.664+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d80fc800 con 0x7fa7d80fb110 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.665+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa7d403b420 con 0x7fa7d80fb110 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.665+0000 7fa7ce7fc640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 0x7fa7c4070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.665+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa7d40b0560 con 0x7fa7d80fb110 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.665+0000 7fa7dd3c9640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 0x7fa7c4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:45.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.665+0000 7fa7dd3c9640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 0x7fa7c4070300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8110760 tx=0x7fa7c800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:45.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.670+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7d4079a90 con 0x7fa7d80fb110 2024-03-18T15:03:45.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.898+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7d804bcc0 con 0x7fa7d80fb110 2024-03-18T15:03:45.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.899+0000 7fa7ce7fc640 1 -- 172.21.15.12:0/3641064952 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fa7d407d150 con 0x7fa7d80fb110 2024-03-18T15:03:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:45.900 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T15:01:27.308391+0000","last_up_change":"2024-03-18T15:00:13.844612+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-18T14:48:34.682425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-18T14:48:35.325444+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6803","nonce":1970485564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6805","nonce":1970485564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6809","nonce":1970485564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6807","nonce":1970485564}]},"public_addr":"172.21.15.12:6803/1970485564","cluster_addr":"172.21.15.12:6805/1970485564","heartbeat_back_addr":"172.21.15.12:6809/1970485564","heartbeat_front_addr":"172.21.15.12:6807/1970485564","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6811","nonce":4045714107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6813","nonce":4045714107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6817","nonce":4045714107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6815","nonce":4045714107}]},"public_addr":"172.21.15.12:6811/4045714107","cluster_addr":"172.21.15.12:6813/4045714107","heartbeat_back_addr":"172.21.15.12:6817/4045714107","heartbeat_front_addr":"172.21.15.12:6815/4045714107","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6819","nonce":3733153396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6821","nonce":3733153396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6825","nonce":3733153396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6823","nonce":3733153396}]},"public_addr":"172.21.15.12:6819/3733153396","cluster_addr":"172.21.15.12:6821/3733153396","heartbeat_back_addr":"172.21.15.12:6825/3733153396","heartbeat_front_addr":"172.21.15.12:6823/3733153396","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6801","nonce":1440116834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6803","nonce":1440116834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6807","nonce":1440116834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6805","nonce":1440116834}]},"public_addr":"172.21.15.100:6801/1440116834","cluster_addr":"172.21.15.100:6803/1440116834","heartbeat_back_addr":"172.21.15.100:6807/1440116834","heartbeat_front_addr":"172.21.15.100:6805/1440116834","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6809","nonce":2146066735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6811","nonce":2146066735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6815","nonce":2146066735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6813","nonce":2146066735}]},"public_addr":"172.21.15.100:6809/2146066735","cluster_addr":"172.21.15.100:6811/2146066735","heartbeat_back_addr":"172.21.15.100:6815/2146066735","heartbeat_front_addr":"172.21.15.100:6813/2146066735","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6817","nonce":1888487408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6819","nonce":1888487408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6823","nonce":1888487408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6821","nonce":1888487408}]},"public_addr":"172.21.15.100:6817/1888487408","cluster_addr":"172.21.15.100:6819/1888487408","heartbeat_back_addr":"172.21.15.100:6823/1888487408","heartbeat_front_addr":"172.21.15.100:6821/1888487408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6827/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6826/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6825/857186455":"2024-03-19T15:01:01.645715+0000","172.21.15.100:0/640996176":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6826/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2958888698":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2624983338":"2024-03-19T14:50:57.290812+0000","172.21.15.100:0/2236779162":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6801/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6828/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/1012959355":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/814075968":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.100:6829/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/3811799837":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/377430941":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/3753470626":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1103543303":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/1749134857":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6827/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6801/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/444337834":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6824/857186455":"2024-03-19T15:01:01.645715+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:03:45.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.901+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 msgr2=0x7fa7c4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.901+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 0x7fa7c4070300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8110760 tx=0x7fa7c800b040 comp rx=0 tx=0).stop 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.901+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 msgr2=0x7fa7d8114ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.901+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40041d0 tx=0x7fa7d4009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.901+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 shutdown_connections 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8101140 0x7fa7d81153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa7c406de50 0x7fa7c4070300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 --2- 172.21.15.12:0/3641064952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7d80fb110 0x7fa7d8114ea0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 >> 172.21.15.12:0/3641064952 conn(0x7fa7d8071130 msgr2=0x7fa7d80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 shutdown_connections 2024-03-18T15:03:45.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:45.902+0000 7fa7dfe2e640 1 -- 172.21.15.12:0/3641064952 wait complete. 2024-03-18T15:03:45.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T14:46:03.095708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-18T14:48:34.682425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-18T14:48:35.325444+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:03:45.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:03:46.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 -- 172.21.15.12:0/747377399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 msgr2=0x7fa3981014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 --2- 172.21.15.12:0/747377399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa3981014b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3940048d0 tx=0x7fa394009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 -- 172.21.15.12:0/747377399 shutdown_connections 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 --2- 172.21.15.12:0/747377399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 0x7fa3980fb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 --2- 172.21.15.12:0/747377399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa3981014b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 -- 172.21.15.12:0/747377399 >> 172.21.15.12:0/747377399 conn(0x7fa3980f6cb0 msgr2=0x7fa3980f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.015+0000 7fa39f51d640 1 -- 172.21.15.12:0/747377399 shutdown_connections 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 -- 172.21.15.12:0/747377399 wait complete. 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 Processor -- start 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 -- start start 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 0x7fa39810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810ea10 con 0x7fa3981010e0 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39f51d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810eb80 con 0x7fa398101a80 2024-03-18T15:03:46.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.016+0000 7fa39d2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54438/0 (socket says 172.21.15.12:54438) 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39d2b9640 1 -- 172.21.15.12:0/2230378831 learned_addr learned my addr 172.21.15.12:0/2230378831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39cab8640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 0x7fa39810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39d2b9640 1 -- 172.21.15.12:0/2230378831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 msgr2=0x7fa39810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39d2b9640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 0x7fa39810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39d2b9640 1 -- 172.21.15.12:0/2230378831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394005040 con 0x7fa3981010e0 2024-03-18T15:03:46.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.017+0000 7fa39d2b9640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa394004930 tx=0x7fa394009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.018+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3940094c0 con 0x7fa3981010e0 2024-03-18T15:03:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.018+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa394032e50 con 0x7fa3981010e0 2024-03-18T15:03:46.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.018+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39810ee00 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.018+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3940433b0 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.018+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa398117250 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.020+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fa39402f070 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.020+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980fc850 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.020+0000 7fa38dffb640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 0x7fa384070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.020+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa3940af3c0 con 0x7fa3981010e0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.020+0000 7fa39cab8640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 0x7fa384070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.021+0000 7fa39cab8640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 0x7fa384070300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa398065890 tx=0x7fa388009120 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.025+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa3940788e0 con 0x7fa3981010e0 2024-03-18T15:03:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.216+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa39810ee00 con 0x7fa3981010e0 2024-03-18T15:03:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.217+0000 7fa38dffb640 1 -- 172.21.15.12:0/2230378831 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa39810ee00 con 0x7fa3981010e0 2024-03-18T15:03:46.217 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-03-18T15:03:46.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 msgr2=0x7fa384070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 0x7fa384070300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa398065890 tx=0x7fa388009120 comp rx=0 tx=0).stop 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 msgr2=0x7fa39810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa394004930 tx=0x7fa394009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 shutdown_connections 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa398101a80 0x7fa39810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fa38406de50 0x7fa384070300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 --2- 172.21.15.12:0/2230378831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa3981010e0 0x7fa39810df90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 >> 172.21.15.12:0/2230378831 conn(0x7fa3980f6cb0 msgr2=0x7fa398104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 shutdown_connections 2024-03-18T15:03:46.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.219+0000 7fa39f51d640 1 -- 172.21.15.12:0/2230378831 wait complete. 2024-03-18T15:03:46.229 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:46 smithi012 ceph-mon[111904]: pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:46.229 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:46 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3290198386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-18T15:03:46.229 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:46 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2385323954' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-18T15:03:46.229 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:46 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3641064952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:46.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-18T15:03:46.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:46 smithi100 ceph-mon[84868]: pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-18T15:03:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:46 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3290198386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-18T15:03:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:46 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2385323954' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-18T15:03:46.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:46 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3641064952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:46.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 -- 172.21.15.12:0/1615715439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec102240 msgr2=0x7fdbec102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 --2- 172.21.15.12:0/1615715439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec102240 0x7fdbec102610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80048d0 tx=0x7fdbe8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 -- 172.21.15.12:0/1615715439 shutdown_connections 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 --2- 172.21.15.12:0/1615715439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec0643a0 0x7fdbec0647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 --2- 172.21.15.12:0/1615715439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec102240 0x7fdbec102610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 -- 172.21.15.12:0/1615715439 >> 172.21.15.12:0/1615715439 conn(0x7fdbec0710b0 msgr2=0x7fdbec0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 -- 172.21.15.12:0/1615715439 shutdown_connections 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.330+0000 7fdbf415d640 1 -- 172.21.15.12:0/1615715439 wait complete. 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 Processor -- start 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 -- start start 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec102240 0x7fdbec0fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0fd0c0 con 0x7fdbec0643a0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf415d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0fbc60 con 0x7fdbec102240 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54466/0 (socket says 172.21.15.12:54466) 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 -- 172.21.15.12:0/3423665728 learned_addr learned my addr 172.21.15.12:0/3423665728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf16f8640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec102240 0x7fdbec0fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 -- 172.21.15.12:0/3423665728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec102240 msgr2=0x7fdbec0fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec102240 0x7fdbec0fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 -- 172.21.15.12:0/3423665728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe8005040 con 0x7fdbec0643a0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.331+0000 7fdbf1ef9640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8004930 tx=0x7fdbe8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.332+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe80094c0 con 0x7fdbec0643a0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.332+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbe8032e50 con 0x7fdbec0643a0 2024-03-18T15:03:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.332+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe80433b0 con 0x7fdbec0643a0 2024-03-18T15:03:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.332+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0fbee0 con 0x7fdbec0643a0 2024-03-18T15:03:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.332+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec1193e0 con 0x7fdbec0643a0 2024-03-18T15:03:46.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.333+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec0fd940 con 0x7fdbec0643a0 2024-03-18T15:03:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.333+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fdbe802f070 con 0x7fdbec0643a0 2024-03-18T15:03:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.334+0000 7fdbe27fc640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 0x7fdbd8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.334+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdbe80af680 con 0x7fdbec0643a0 2024-03-18T15:03:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.334+0000 7fdbf16f8640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 0x7fdbd8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.334+0000 7fdbf16f8640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 0x7fdbd8070300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0fc910 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.338+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdbe8078ba0 con 0x7fdbec0643a0 2024-03-18T15:03:46.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.526+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fdbec04bcc0 con 0x7fdbec0643a0 2024-03-18T15:03:46.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.527+0000 7fdbe27fc640 1 -- 172.21.15.12:0/3423665728 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdbe807c260 con 0x7fdbec0643a0 2024-03-18T15:03:46.527 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:46.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.528+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 msgr2=0x7fdbd8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.528+0000 7fdbf415d640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 0x7fdbd8070300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0fc910 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2024-03-18T15:03:46.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.528+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 msgr2=0x7fdbec0fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.528+0000 7fdbf415d640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8004930 tx=0x7fdbe8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 shutdown_connections 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbec102240 0x7fdbec0fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fdbd806de50 0x7fdbd8070300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 --2- 172.21.15.12:0/3423665728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbec0643a0 0x7fdbec0fb1e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 >> 172.21.15.12:0/3423665728 conn(0x7fdbec0710b0 msgr2=0x7fdbec0f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 shutdown_connections 2024-03-18T15:03:46.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.529+0000 7fdbf415d640 1 -- 172.21.15.12:0/3423665728 wait complete. 2024-03-18T15:03:46.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-18T15:03:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 -- 172.21.15.12:0/431681582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c0643a0 msgr2=0x7fab0c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 --2- 172.21.15.12:0/431681582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c0643a0 0x7fab0c0647d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faafc0048d0 tx=0x7faafc009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 -- 172.21.15.12:0/431681582 shutdown_connections 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 --2- 172.21.15.12:0/431681582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c0643a0 0x7fab0c0647d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 --2- 172.21.15.12:0/431681582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c102240 0x7fab0c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 -- 172.21.15.12:0/431681582 >> 172.21.15.12:0/431681582 conn(0x7fab0c0710b0 msgr2=0x7fab0c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 -- 172.21.15.12:0/431681582 shutdown_connections 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.642+0000 7fab1446e640 1 -- 172.21.15.12:0/431681582 wait complete. 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 Processor -- start 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 -- start start 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 0x7fab0c112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c113800 con 0x7fab0c102240 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1446e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c10ce50 con 0x7fab0c0643a0 2024-03-18T15:03:46.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54484/0 (socket says 172.21.15.12:54484) 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 -- 172.21.15.12:0/64215070 learned_addr learned my addr 172.21.15.12:0/64215070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1220a640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 0x7fab0c112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 -- 172.21.15.12:0/64215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 msgr2=0x7fab0c112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 0x7fab0c112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab11a09640 1 -- 172.21.15.12:0/64215070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc005040 con 0x7fab0c102240 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.643+0000 7fab1220a640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 0x7fab0c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab11a09640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faafc004930 tx=0x7faafc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faafc0094c0 con 0x7fab0c102240 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faafc032e50 con 0x7fab0c102240 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faafc0433b0 con 0x7fab0c102240 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c10d0d0 con 0x7fab0c102240 2024-03-18T15:03:46.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.644+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c10d590 con 0x7fab0c102240 2024-03-18T15:03:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.645+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c0fd940 con 0x7fab0c102240 2024-03-18T15:03:46.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.645+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7faafc02f070 con 0x7fab0c102240 2024-03-18T15:03:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.646+0000 7fab02ffd640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 0x7faaf8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.646+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faafc0af5c0 con 0x7fab0c102240 2024-03-18T15:03:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.646+0000 7fab1220a640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 0x7faaf8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.646+0000 7fab1220a640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 0x7faaf8070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab08004d60 tx=0x7fab0800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.651+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faafc078a30 con 0x7fab0c102240 2024-03-18T15:03:46.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.834+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fab0c109390 con 0x7fab0c102240 2024-03-18T15:03:46.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.835+0000 7fab02ffd640 1 -- 172.21.15.12:0/64215070 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7faafc07c0f0 con 0x7fab0c102240 2024-03-18T15:03:46.836 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 msgr2=0x7faaf8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 0x7faaf8070300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab08004d60 tx=0x7fab0800b040 comp rx=0 tx=0).stop 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 msgr2=0x7fab0c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faafc004930 tx=0x7faafc009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 shutdown_connections 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7faaf806de50 0x7faaf8070300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab0c102240 0x7fab0c113200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 --2- 172.21.15.12:0/64215070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab0c0643a0 0x7fab0c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 >> 172.21.15.12:0/64215070 conn(0x7fab0c0710b0 msgr2=0x7fab0c0f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 shutdown_connections 2024-03-18T15:03:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.837+0000 7fab1446e640 1 -- 172.21.15.12:0/64215070 wait complete. 2024-03-18T15:03:46.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- 172.21.15.12:0/1135704724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102ba0 msgr2=0x7f55940fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 --2- 172.21.15.12:0/1135704724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102ba0 0x7f55940fa8d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55880048d0 tx=0x7f5588009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- 172.21.15.12:0/1135704724 shutdown_connections 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 --2- 172.21.15.12:0/1135704724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102ba0 0x7f55940fa8d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 --2- 172.21.15.12:0/1135704724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594102200 0x7f55941025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- 172.21.15.12:0/1135704724 >> 172.21.15.12:0/1135704724 conn(0x7f55940708f0 msgr2=0x7f5594070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- 172.21.15.12:0/1135704724 shutdown_connections 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- 172.21.15.12:0/1135704724 wait complete. 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 Processor -- start 2024-03-18T15:03:46.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.950+0000 7f559a62e640 1 -- start start 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f559a62e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f559a62e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594102ba0 0x7f5594113300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f559a62e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594113900 con 0x7f5594102200 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f559a62e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559410cf50 con 0x7f5594102ba0 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54500/0 (socket says 172.21.15.12:54500) 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 -- 172.21.15.12:0/1352110085 learned_addr learned my addr 172.21.15.12:0/1352110085 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 -- 172.21.15.12:0/1352110085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594102ba0 msgr2=0x7f5594113300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594102ba0 0x7f5594113300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 -- 172.21.15.12:0/1352110085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588005040 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.951+0000 7f5593fff640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5584002a80 tx=0x7f5584004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.952+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5584007c30 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.952+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f558400f040 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.952+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559410d230 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.952+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55840096b0 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.952+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559410d750 con 0x7f5594102200 2024-03-18T15:03:46.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.953+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55940652a0 con 0x7f5594102200 2024-03-18T15:03:46.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.954+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f558400d6b0 con 0x7f5594102200 2024-03-18T15:03:46.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.954+0000 7f5590ff9640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 0x7f557c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:46.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.954+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f558408d0b0 con 0x7f5594102200 2024-03-18T15:03:46.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.954+0000 7f55937fe640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 0x7f557c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:46.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.954+0000 7f55937fe640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 0x7f557c070300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:46.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:46.963+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55840565a0 con 0x7f5594102200 2024-03-18T15:03:47.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.164+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f559404bcc0 con 0x7f5594102200 2024-03-18T15:03:47.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.165+0000 7f5590ff9640 1 -- 172.21.15.12:0/1352110085 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v82) v1 ==== 81+0+2313 (secure 0 0 0) 0x7f5584059c60 con 0x7f5594102200 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6827/2597653923 2024-03-19T15:01:27.308363+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6826/2597653923 2024-03-19T15:01:27.308363+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6825/857186455 2024-03-19T15:01:01.645715+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/640996176 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6826/1453814985 2024-03-19T15:00:19.128431+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4273277683 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3609733177 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:47.166 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2818744289 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3609733177 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2958888698 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/28013377 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2624983338 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/2236779162 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1147707934 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1147707934 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/965904248 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6828/3399432197 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4273277683 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/1012959355 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/814075968 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2283812099 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:47.167 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6829/3399432197 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/656067026 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3811799837 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1240903034 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/377430941 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2991511080 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:47.168 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3753470626 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3145891342 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/838750899 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1103543303 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/825848523 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1749134857 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3117137762 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6827/1453814985 2024-03-19T15:00:19.128431+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3117137762 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/444337834 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/965904248 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:47.169 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6824/857186455 2024-03-19T15:01:01.645715+0000 2024-03-18T15:03:47.170 INFO:teuthology.orchestra.run.smithi012.stderr:listed 39 entries 2024-03-18T15:03:47.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 msgr2=0x7f557c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 0x7f557c070300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 msgr2=0x7f5594112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5584002a80 tx=0x7f5584004600 comp rx=0 tx=0).stop 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 shutdown_connections 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5594102ba0 0x7f5594113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f557c06de50 0x7f557c070300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 --2- 172.21.15.12:0/1352110085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5594102200 0x7f5594112dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 >> 172.21.15.12:0/1352110085 conn(0x7f55940708f0 msgr2=0x7f5594104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 shutdown_connections 2024-03-18T15:03:47.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.167+0000 7f559a62e640 1 -- 172.21.15.12:0/1352110085 wait complete. 2024-03-18T15:03:47.177 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:47 smithi012 ceph-mon[111904]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:47.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:47 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2230378831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:47 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3423665728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.178 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:47 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/64215070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.179 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-18T15:03:47.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:47.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-18T15:03:47.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T15:03:47.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:47.207 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T15:03:47.268 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T15:03:47.268 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:03:47.287 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-18T15:03:47.288 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:03:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:47 smithi100 ceph-mon[84868]: from='mgr.34102 172.21.15.12:0/2231429725' entity='mgr.smithi012.tmscgt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-18T15:03:47.294 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:47 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2230378831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:47 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3423665728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:47 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/64215070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:47.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:03:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 -- 172.21.15.12:0/3213741148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 msgr2=0x7fec380fb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 --2- 172.21.15.12:0/3213741148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec380fb0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0048d0 tx=0x7fec2c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 -- 172.21.15.12:0/3213741148 shutdown_connections 2024-03-18T15:03:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 --2- 172.21.15.12:0/3213741148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec380fb0e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 --2- 172.21.15.12:0/3213741148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 0x7fec38101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.410+0000 7fec3d407640 1 -- 172.21.15.12:0/3213741148 >> 172.21.15.12:0/3213741148 conn(0x7fec380f6ca0 msgr2=0x7fec380f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 -- 172.21.15.12:0/3213741148 shutdown_connections 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 -- 172.21.15.12:0/3213741148 wait complete. 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 Processor -- start 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 -- start start 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 0x7fec38112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38113810 con 0x7fec38101a50 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.411+0000 7fec3d407640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3810ce60 con 0x7fec381010b0 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54514/0 (socket says 172.21.15.12:54514) 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 -- 172.21.15.12:0/589092178 learned_addr learned my addr 172.21.15.12:0/589092178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec36ffd640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 0x7fec38112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 -- 172.21.15.12:0/589092178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 msgr2=0x7fec38112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 0x7fec38112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 -- 172.21.15.12:0/589092178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c005040 con 0x7fec38101a50 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec367fc640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec2c004930 tx=0x7fec2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c0094c0 con 0x7fec38101a50 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3810d0e0 con 0x7fec38101a50 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec2c032e50 con 0x7fec38101a50 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c0433b0 con 0x7fec38101a50 2024-03-18T15:03:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.412+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3810d5a0 con 0x7fec38101a50 2024-03-18T15:03:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.413+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380fc820 con 0x7fec38101a50 2024-03-18T15:03:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.414+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fec2c02f070 con 0x7fec38101a50 2024-03-18T15:03:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.414+0000 7fec1f7fe640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 0x7fec20070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.414+0000 7fec36ffd640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 0x7fec20070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:47.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.414+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fec2c0af650 con 0x7fec38101a50 2024-03-18T15:03:47.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.415+0000 7fec36ffd640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 0x7fec20070300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec28007fd0 tx=0x7fec28009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:47.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.419+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec2c078ac0 con 0x7fec38101a50 2024-03-18T15:03:47.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.615+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec38108210 con 0x7fec38101a50 2024-03-18T15:03:47.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.616+0000 7fec1f7fe640 1 -- 172.21.15.12:0/589092178 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fec2c07c180 con 0x7fec38101a50 2024-03-18T15:03:47.617 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-03-18T15:03:47.617 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"74f0a062-e535-11ee-95c9-87774f69a715","created":"2024-03-18T14:41:25.323113+0000","modified":"2024-03-18T15:01:27.308391+0000","last_up_change":"2024-03-18T15:00:13.844612+0000","last_in_change":"2024-03-18T14:46:57.471531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T14:46:03.095708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-18T14:48:34.682425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-18T14:48:35.325444+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83bef24b-6305-4fac-971a-7fe5250379b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6803","nonce":1970485564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6805","nonce":1970485564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6809","nonce":1970485564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1970485564},{"type":"v1","addr":"172.21.15.12:6807","nonce":1970485564}]},"public_addr":"172.21.15.12:6803/1970485564","cluster_addr":"172.21.15.12:6805/1970485564","heartbeat_back_addr":"172.21.15.12:6809/1970485564","heartbeat_front_addr":"172.21.15.12:6807/1970485564","state":["exists","up"]},{"osd":1,"uuid":"a52709aa-6ce3-4e50-8dc8-c4ea6a5913b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6811","nonce":4045714107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6813","nonce":4045714107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6817","nonce":4045714107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4045714107},{"type":"v1","addr":"172.21.15.12:6815","nonce":4045714107}]},"public_addr":"172.21.15.12:6811/4045714107","cluster_addr":"172.21.15.12:6813/4045714107","heartbeat_back_addr":"172.21.15.12:6817/4045714107","heartbeat_front_addr":"172.21.15.12:6815/4045714107","state":["exists","up"]},{"osd":2,"uuid":"879ccbfa-0108-49c6-a6d9-c3badfec53f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6819","nonce":3733153396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6821","nonce":3733153396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6825","nonce":3733153396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3733153396},{"type":"v1","addr":"172.21.15.12:6823","nonce":3733153396}]},"public_addr":"172.21.15.12:6819/3733153396","cluster_addr":"172.21.15.12:6821/3733153396","heartbeat_back_addr":"172.21.15.12:6825/3733153396","heartbeat_front_addr":"172.21.15.12:6823/3733153396","state":["exists","up"]},{"osd":3,"uuid":"b1efeb93-b505-43ad-854b-c58259129d99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6801","nonce":1440116834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6803","nonce":1440116834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6807","nonce":1440116834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1440116834},{"type":"v1","addr":"172.21.15.100:6805","nonce":1440116834}]},"public_addr":"172.21.15.100:6801/1440116834","cluster_addr":"172.21.15.100:6803/1440116834","heartbeat_back_addr":"172.21.15.100:6807/1440116834","heartbeat_front_addr":"172.21.15.100:6805/1440116834","state":["exists","up"]},{"osd":4,"uuid":"3d2105a2-ae9a-492a-b5a6-9df98f19131c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6809","nonce":2146066735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6811","nonce":2146066735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6815","nonce":2146066735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2146066735},{"type":"v1","addr":"172.21.15.100:6813","nonce":2146066735}]},"public_addr":"172.21.15.100:6809/2146066735","cluster_addr":"172.21.15.100:6811/2146066735","heartbeat_back_addr":"172.21.15.100:6815/2146066735","heartbeat_front_addr":"172.21.15.100:6813/2146066735","state":["exists","up"]},{"osd":5,"uuid":"822b9e7b-edd3-49b1-b668-d6c34e799d33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6817","nonce":1888487408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6819","nonce":1888487408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6823","nonce":1888487408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1888487408},{"type":"v1","addr":"172.21.15.100:6821","nonce":1888487408}]},"public_addr":"172.21.15.100:6817/1888487408","cluster_addr":"172.21.15.100:6819/1888487408","heartbeat_back_addr":"172.21.15.100:6823/1888487408","heartbeat_front_addr":"172.21.15.100:6821/1888487408","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:04.139647+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:45:31.859221+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:00.278726+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:24.080856+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:46:49.685537+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T14:47:14.521764+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6827/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6826/2597653923":"2024-03-19T15:01:27.308363+0000","172.21.15.100:6825/857186455":"2024-03-19T15:01:01.645715+0000","172.21.15.100:0/640996176":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6826/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6800/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6801/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2818744289":"2024-03-19T14:41:58.905986+0000","172.21.15.12:6800/3609733177":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2958888698":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/28013377":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/2624983338":"2024-03-19T14:50:57.290812+0000","172.21.15.100:0/2236779162":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6800/1147707934":"2024-03-19T14:43:50.762971+0000","172.21.15.12:6801/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6828/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:6801/4273277683":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/1012959355":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/814075968":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/2283812099":"2024-03-19T14:43:50.762971+0000","172.21.15.100:6829/3399432197":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/656067026":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/3811799837":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/1240903034":"2024-03-19T14:41:58.905986+0000","172.21.15.100:0/377430941":"2024-03-19T14:52:46.432360+0000","172.21.15.12:0/2991511080":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/3753470626":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/3145891342":"2024-03-19T14:42:30.567027+0000","172.21.15.12:0/838750899":"2024-03-19T14:43:50.762971+0000","172.21.15.12:0/1103543303":"2024-03-19T14:50:57.290812+0000","172.21.15.12:0/825848523":"2024-03-19T14:41:58.905986+0000","172.21.15.12:0/1749134857":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6827/1453814985":"2024-03-19T15:00:19.128431+0000","172.21.15.12:6801/3117137762":"2024-03-19T14:55:46.050858+0000","172.21.15.12:0/444337834":"2024-03-19T14:55:46.050858+0000","172.21.15.12:6800/965904248":"2024-03-19T14:50:57.290812+0000","172.21.15.100:6824/857186455":"2024-03-19T15:01:01.645715+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:03:47.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.618+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 msgr2=0x7fec20070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.618+0000 7fec3d407640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 0x7fec20070300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec28007fd0 tx=0x7fec28009e90 comp rx=0 tx=0).stop 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.618+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 msgr2=0x7fec38113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.618+0000 7fec3d407640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec2c004930 tx=0x7fec2c009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 shutdown_connections 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fec2006de50 0x7fec20070300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec38101a50 0x7fec38113210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 --2- 172.21.15.12:0/589092178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec381010b0 0x7fec38112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 >> 172.21.15.12:0/589092178 conn(0x7fec380f6ca0 msgr2=0x7fec38065520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 shutdown_connections 2024-03-18T15:03:47.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.619+0000 7fec3d407640 1 -- 172.21.15.12:0/589092178 wait complete. 2024-03-18T15:03:47.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T14:46:03.095708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-18T14:48:34.682425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-18T14:48:35.325444+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:03:47.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:03:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 -- 172.21.15.12:0/16454606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 msgr2=0x7f15ec0fb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/16454606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec0fb390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f15e80048d0 tx=0x7f15e8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 -- 172.21.15.12:0/16454606 shutdown_connections 2024-03-18T15:03:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/16454606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0fb8d0 0x7f15ec108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/16454606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec0fb390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 -- 172.21.15.12:0/16454606 >> 172.21.15.12:0/16454606 conn(0x7f15ec0f6cb0 msgr2=0x7f15ec0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 -- 172.21.15.12:0/16454606 shutdown_connections 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.743+0000 7f15f3e6e640 1 -- 172.21.15.12:0/16454606 wait complete. 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 Processor -- start 2024-03-18T15:03:47.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 -- start start 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0fb8d0 0x7f15ec115340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec115940 con 0x7f15ec0fafc0 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f3e6e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec10ef90 con 0x7f15ec0fb8d0 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54534/0 (socket says 172.21.15.12:54534) 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 -- 172.21.15.12:0/3110755626 learned_addr learned my addr 172.21.15.12:0/3110755626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 -- 172.21.15.12:0/3110755626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0fb8d0 msgr2=0x7f15ec115340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0fb8d0 0x7f15ec115340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.744+0000 7f15f1c0a640 1 -- 172.21.15.12:0/3110755626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8005040 con 0x7f15ec0fafc0 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15f1c0a640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15e8003080 tx=0x7f15e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e803b7f0 con 0x7f15ec0fafc0 2024-03-18T15:03:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec10f270 con 0x7f15ec0fafc0 2024-03-18T15:03:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e8031780 con 0x7f15ec0fafc0 2024-03-18T15:03:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e80439a0 con 0x7f15ec0fafc0 2024-03-18T15:03:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.745+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec10f810 con 0x7f15ec0fafc0 2024-03-18T15:03:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.746+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec065920 con 0x7f15ec0fafc0 2024-03-18T15:03:47.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.747+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f15e8043b00 con 0x7f15ec0fafc0 2024-03-18T15:03:47.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.747+0000 7f15e27fc640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 0x7f15d8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:47.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.747+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f15e80afff0 con 0x7f15ec0fafc0 2024-03-18T15:03:47.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.747+0000 7f15f1409640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 0x7f15d8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:47.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.748+0000 7f15f1409640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 0x7f15d8070300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1106c0 tx=0x7f15dc007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:47.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.752+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15e8079420 con 0x7f15ec0fafc0 2024-03-18T15:03:47.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.937+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15ec10b470 con 0x7f15ec0fafc0 2024-03-18T15:03:47.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.938+0000 7f15e27fc640 1 -- 172.21.15.12:0/3110755626 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15e807cae0 con 0x7f15ec0fafc0 2024-03-18T15:03:47.939 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 msgr2=0x7f15d8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 0x7f15d8070300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1106c0 tx=0x7f15dc007c30 comp rx=0 tx=0).stop 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 msgr2=0x7f15ec114e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15e8003080 tx=0x7f15e8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 shutdown_connections 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0fb8d0 0x7f15ec115340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f15d806de50 0x7f15d8070300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 --2- 172.21.15.12:0/3110755626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15ec0fafc0 0x7f15ec114e00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 >> 172.21.15.12:0/3110755626 conn(0x7f15ec0f6cb0 msgr2=0x7f15ec0f7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 shutdown_connections 2024-03-18T15:03:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:47.940+0000 7f15f3e6e640 1 -- 172.21.15.12:0/3110755626 wait complete. 2024-03-18T15:03:47.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-18T15:03:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.053+0000 7f66fdaae640 1 -- 172.21.15.12:0/1061412249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8102c70 msgr2=0x7f66f80fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.053+0000 7f66fdaae640 1 --2- 172.21.15.12:0/1061412249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8102c70 0x7f66f80fc1f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f66e00048d0 tx=0x7f66e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- 172.21.15.12:0/1061412249 shutdown_connections 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 --2- 172.21.15.12:0/1061412249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8102c70 0x7f66f80fc1f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 --2- 172.21.15.12:0/1061412249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f81022d0 0x7f66f81026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- 172.21.15.12:0/1061412249 >> 172.21.15.12:0/1061412249 conn(0x7f66f80f7d90 msgr2=0x7f66f80fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- 172.21.15.12:0/1061412249 shutdown_connections 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- 172.21.15.12:0/1061412249 wait complete. 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 Processor -- start 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- start start 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f8102c70 0x7f66f806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80716b0 con 0x7f66f81022d0 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66fdaae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8070250 con 0x7f66f8102c70 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54556/0 (socket says 172.21.15.12:54556) 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.054+0000 7f66f77fe640 1 -- 172.21.15.12:0/2406435545 learned_addr learned my addr 172.21.15.12:0/2406435545 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66effff640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f8102c70 0x7f66f806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f77fe640 1 -- 172.21.15.12:0/2406435545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f8102c70 msgr2=0x7f66f806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f77fe640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f8102c70 0x7f66f806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f77fe640 1 -- 172.21.15.12:0/2406435545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0005040 con 0x7f66f81022d0 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f77fe640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66e8002a50 tx=0x7f66e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e8007c30 con 0x7f66f81022d0 2024-03-18T15:03:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66e800f040 con 0x7f66f81022d0 2024-03-18T15:03:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f8070530 con 0x7f66f81022d0 2024-03-18T15:03:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e800db50 con 0x7f66f81022d0 2024-03-18T15:03:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.055+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f8119530 con 0x7f66f81022d0 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.056+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f804bcc0 con 0x7f66f81022d0 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.056+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f66e800dcf0 con 0x7f66f81022d0 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.057+0000 7f66f4ff9640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 0x7f66e4070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.057+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f66e808cfa0 con 0x7f66f81022d0 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.057+0000 7f66effff640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 0x7f66e4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.057+0000 7f66effff640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 0x7f66e4070300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66f8070f00 tx=0x7f66e0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.062+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f66e80564c0 con 0x7f66f81022d0 2024-03-18T15:03:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:48 smithi012 ceph-mon[111904]: pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:48 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1352110085' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:03:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:48 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/589092178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:48.241 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:48 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3110755626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:48.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.243+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f66f8109430 con 0x7f66f81022d0 2024-03-18T15:03:48.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.243+0000 7f66f4ff9640 1 -- 172.21.15.12:0/2406435545 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f66e8017020 con 0x7f66f81022d0 2024-03-18T15:03:48.244 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 msgr2=0x7f66e4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 0x7f66e4070300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66f8070f00 tx=0x7f66e0009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 msgr2=0x7f66f806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f66e8002a50 tx=0x7f66e8004600 comp rx=0 tx=0).stop 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 shutdown_connections 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66f8102c70 0x7f66f806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f66e406de50 0x7f66e4070300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 --2- 172.21.15.12:0/2406435545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f81022d0 0x7f66f806f7d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 >> 172.21.15.12:0/2406435545 conn(0x7f66f80f7d90 msgr2=0x7f66f80f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 shutdown_connections 2024-03-18T15:03:48.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.245+0000 7f66fdaae640 1 -- 172.21.15.12:0/2406435545 wait complete. 2024-03-18T15:03:48.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-18T15:03:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:48 smithi100 ceph-mon[84868]: pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-18T15:03:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:48 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1352110085' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:03:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:48 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/589092178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:03:48.295 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:48 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3110755626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 -- 172.21.15.12:0/4135302612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 msgr2=0x7fcdd40647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 --2- 172.21.15.12:0/4135302612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd40647d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80048d0 tx=0x7fcdc8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 -- 172.21.15.12:0/4135302612 shutdown_connections 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 --2- 172.21.15.12:0/4135302612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd40647d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 --2- 172.21.15.12:0/4135302612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 0x7fcdd4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 -- 172.21.15.12:0/4135302612 >> 172.21.15.12:0/4135302612 conn(0x7fcdd40710b0 msgr2=0x7fcdd40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 -- 172.21.15.12:0/4135302612 shutdown_connections 2024-03-18T15:03:48.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 -- 172.21.15.12:0/4135302612 wait complete. 2024-03-18T15:03:48.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.358+0000 7fcddb02e640 1 Processor -- start 2024-03-18T15:03:48.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcddb02e640 1 -- start start 2024-03-18T15:03:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcddb02e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcddb02e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 0x7fcdd4113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcddb02e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4113810 con 0x7fcdd40643a0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcddb02e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd410ce60 con 0x7fcdd4102240 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54574/0 (socket says 172.21.15.12:54574) 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 -- 172.21.15.12:0/3107319962 learned_addr learned my addr 172.21.15.12:0/3107319962 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd3fff640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 0x7fcdd4113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 -- 172.21.15.12:0/3107319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 msgr2=0x7fcdd4113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 0x7fcdd4113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.359+0000 7fcdd8dca640 1 -- 172.21.15.12:0/3107319962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8005040 con 0x7fcdd40643a0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcdd8dca640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00db00 tx=0x7fcdcc004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc017070 con 0x7fcdd40643a0 2024-03-18T15:03:48.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd410d140 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdcc00f040 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc003ae0 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.360+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd410d630 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.361+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd40fd940 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.361+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7fcdcc003c40 con 0x7fcdd40643a0 2024-03-18T15:03:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.362+0000 7fcdd17fa640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 0x7fcdbc070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.362+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcdcc08d910 con 0x7fcdd40643a0 2024-03-18T15:03:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.362+0000 7fcdd3fff640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 0x7fcdbc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.362+0000 7fcdd3fff640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 0x7fcdbc070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40fafe0 tx=0x7fcdc8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.366+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdcc056d80 con 0x7fcdd40643a0 2024-03-18T15:03:48.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.551+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fcdd410e340 con 0x7fcdd40643a0 2024-03-18T15:03:48.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.551+0000 7fcdd17fa640 1 -- 172.21.15.12:0/3107319962 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcdcc05a440 con 0x7fcdd40643a0 2024-03-18T15:03:48.552 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 msgr2=0x7fcdbc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 0x7fcdbc070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40fafe0 tx=0x7fcdc8009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 msgr2=0x7fcdd4112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00db00 tx=0x7fcdcc004e60 comp rx=0 tx=0).stop 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 shutdown_connections 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdd4102240 0x7fcdd4113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7fcdbc06de50 0x7fcdbc070300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 --2- 172.21.15.12:0/3107319962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd40643a0 0x7fcdd4112cd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 >> 172.21.15.12:0/3107319962 conn(0x7fcdd40710b0 msgr2=0x7fcdd40f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 shutdown_connections 2024-03-18T15:03:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.553+0000 7fcddb02e640 1 -- 172.21.15.12:0/3107319962 wait complete. 2024-03-18T15:03:48.565 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1201601550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 msgr2=0x7f9e740fe420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1201601550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e740fe420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9e600048d0 tx=0x7f9e60009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1201601550 shutdown_connections 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1201601550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e740fe420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1201601550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e741047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1201601550 >> 172.21.15.12:0/1201601550 conn(0x7f9e740f9fc0 msgr2=0x7f9e740fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1201601550 shutdown_connections 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.664+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1201601550 wait complete. 2024-03-18T15:03:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 Processor -- start 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 -- start start 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e74114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74115a10 con 0x7f9e74104d90 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e7b19d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7410f060 con 0x7f9e741043f0 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e73fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e73fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43062/0 (socket says 172.21.15.12:43062) 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e73fff640 1 -- 172.21.15.12:0/1630641938 learned_addr learned my addr 172.21.15.12:0/1630641938 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e78f39640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e74114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e73fff640 1 -- 172.21.15.12:0/1630641938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 msgr2=0x7f9e74114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.665+0000 7f9e73fff640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e74114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e73fff640 1 -- 172.21.15.12:0/1630641938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60005040 con 0x7f9e74104d90 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e78f39640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e74114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e73fff640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e60034040 tx=0x7f9e60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e600092b0 con 0x7f9e74104d90 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7410f2e0 con 0x7f9e74104d90 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e6003b400 con 0x7f9e74104d90 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e600435d0 con 0x7f9e74104d90 2024-03-18T15:03:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.666+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7410f7a0 con 0x7f9e74104d90 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.667+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e740ffb60 con 0x7f9e74104d90 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.667+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 40) v1 ==== 91841+0+0 (secure 0 0 0) 0x7f9e6002f070 con 0x7f9e74104d90 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.668+0000 7f9e717fa640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 0x7f9e64070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.668+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9e600af570 con 0x7f9e74104d90 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.668+0000 7f9e78f39640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 0x7f9e64070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:03:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.668+0000 7f9e78f39640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 0x7f9e64070300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e680049d0 tx=0x7f9e6800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:03:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.672+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e600789e0 con 0x7f9e74104d90 2024-03-18T15:03:48.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.877+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e7410b550 con 0x7f9e74104d90 2024-03-18T15:03:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.878+0000 7f9e717fa640 1 -- 172.21.15.12:0/1630641938 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v82) v1 ==== 81+0+2313 (secure 0 0 0) 0x7f9e6007c0a0 con 0x7f9e74104d90 2024-03-18T15:03:48.879 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6827/2597653923 2024-03-19T15:01:27.308363+0000 2024-03-18T15:03:48.879 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6826/2597653923 2024-03-19T15:01:27.308363+0000 2024-03-18T15:03:48.879 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6825/857186455 2024-03-19T15:01:01.645715+0000 2024-03-18T15:03:48.879 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/640996176 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6826/1453814985 2024-03-19T15:00:19.128431+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4273277683 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3609733177 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2818744289 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3609733177 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:48.880 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2958888698 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/28013377 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2624983338 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/2236779162 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1147707934 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1147707934 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/965904248 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6828/3399432197 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4273277683 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/1012959355 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/814075968 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2283812099 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6829/3399432197 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/656067026 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:48.881 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3811799837 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1240903034 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:0/377430941 2024-03-19T14:52:46.432360+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2991511080 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3753470626 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3145891342 2024-03-19T14:42:30.567027+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/838750899 2024-03-19T14:43:50.762971+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1103543303 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/825848523 2024-03-19T14:41:58.905986+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1749134857 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3117137762 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6827/1453814985 2024-03-19T15:00:19.128431+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3117137762 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/444337834 2024-03-19T14:55:46.050858+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/965904248 2024-03-19T14:50:57.290812+0000 2024-03-18T15:03:48.882 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.100:6824/857186455 2024-03-19T15:01:01.645715+0000 2024-03-18T15:03:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:listed 39 entries 2024-03-18T15:03:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 msgr2=0x7f9e64070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 0x7f9e64070300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e680049d0 tx=0x7f9e6800b040 comp rx=0 tx=0).stop 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 msgr2=0x7f9e74115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e60034040 tx=0x7f9e60009da0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 shutdown_connections 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:6800/4292057823,v1:172.21.15.12:6801/4292057823] conn(0x7f9e6406de50 0x7f9e64070300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e74104d90 0x7f9e74115410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 --2- 172.21.15.12:0/1630641938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e741043f0 0x7f9e74114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 >> 172.21.15.12:0/1630641938 conn(0x7f9e740f9fc0 msgr2=0x7f9e740faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 shutdown_connections 2024-03-18T15:03:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-03-18T15:03:48.880+0000 7f9e7b19d640 1 -- 172.21.15.12:0/1630641938 wait complete. 2024-03-18T15:03:48.893 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-18T15:03:48.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:48.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-18T15:03:48.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-03-18T15:03:48.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:48.930 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-18T15:03:48.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:48.999 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-03-18T15:03:49.014 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T15:03:49.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:49.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:03:49.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:49.081 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-03-18T15:03:49.146 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/2406435545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:49.146 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/3107319962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:49.146 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 ceph-mon[111904]: from='client.? 172.21.15.12:0/1630641938' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:03:49.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:49.171 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-03-18T15:03:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-18T15:03:49.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:49.190 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-18T15:03:49.257 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:49 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/2406435545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-18T15:03:49.257 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:49 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/3107319962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-18T15:03:49.257 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:49 smithi100 ceph-mon[84868]: from='client.? 172.21.15.12:0/1630641938' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:03:49.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:03:49.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-03-18T15:03:49.376 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-18T15:03:49.389 INFO:tasks.cephadm:Teardown begin 2024-03-18T15:03:49.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:03:49.421 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:03:49.451 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-18T15:03:49.451 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:03:49.478 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:03:49.506 INFO:tasks.cephadm:Stopping all daemons... 2024-03-18T15:03:49.506 INFO:tasks.cephadm.mon.smithi012:Stopping mon.smithi012... 2024-03-18T15:03:49.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012 2024-03-18T15:03:49.741 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 systemd[1]: Stopping Ceph mon.smithi012 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:03:50.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[111900]: 2024-03-18T15:03:49.851+0000 7fe148dff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi012 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:03:50.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012[111900]: 2024-03-18T15:03:49.851+0000 7fe148dff700 -1 mon.smithi012@0(leader) e3 *** Got Signal Terminated *** 2024-03-18T15:03:50.242 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:49 smithi012 podman[153381]: 2024-03-18 15:03:49.98400982 +0000 UTC m=+0.258193636 container died 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:03:50.677 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:50 smithi012 podman[153381]: 2024-03-18 15:03:50.350814202 +0000 UTC m=+0.624998013 container cleanup 3b6df403a483504312ff33cde5a755fd8174f768fd4e9052aed103253ade57b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-18T15:03:50.677 INFO:journalctl@ceph.mon.smithi012.smithi012.stdout:Mar 18 15:03:50 smithi012 bash[153381]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi012 2024-03-18T15:03:50.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi012.service' 2024-03-18T15:03:50.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:03:50.887 INFO:tasks.cephadm.mon.smithi012:Stopped mon.smithi012 2024-03-18T15:03:50.887 INFO:tasks.cephadm.mon.smithi100:Stopping mon.smithi100... 2024-03-18T15:03:50.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi100 2024-03-18T15:03:51.187 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:50 smithi100 systemd[1]: Stopping Ceph mon.smithi100 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:03:51.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:51 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100[84864]: 2024-03-18T15:03:51.185+0000 7fcbfd10a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi100 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:03:51.544 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:51 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100[84864]: 2024-03-18T15:03:51.185+0000 7fcbfd10a700 -1 mon.smithi100@1(peon) e3 *** Got Signal Terminated *** 2024-03-18T15:03:51.545 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:51 smithi100 podman[117364]: 2024-03-18 15:03:51.319195905 +0000 UTC m=+0.238950669 container died 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T15:03:52.044 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:51 smithi100 podman[117364]: 2024-03-18 15:03:51.811985753 +0000 UTC m=+0.731740516 container cleanup 9af15206c43d7f1dcfbb990d7773d26bdd3c961b40bd05f4d2615e7548ef6e59 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:03:52.045 INFO:journalctl@ceph.mon.smithi100.smithi100.stdout:Mar 18 15:03:51 smithi100 bash[117364]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-mon-smithi100 2024-03-18T15:03:52.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@mon.smithi100.service' 2024-03-18T15:03:52.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:03:52.274 INFO:tasks.cephadm.mon.smithi100:Stopped mon.smithi100 2024-03-18T15:03:52.274 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-18T15:03:52.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0 2024-03-18T15:03:52.581 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:52 smithi012 systemd[1]: Stopping Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:03:52.991 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:52 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T15:03:52.580+0000 7fe22d997700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:03:52.991 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:52 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T15:03:52.580+0000 7fe22d997700 -1 osd.0 82 *** Got signal Terminated *** 2024-03-18T15:03:52.992 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:52 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0[123391]: 2024-03-18T15:03:52.580+0000 7fe22d997700 -1 osd.0 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:03:58.365 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:58 smithi012 podman[153544]: 2024-03-18 15:03:58.080780787 +0000 UTC m=+5.607720760 container died f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True) 2024-03-18T15:03:58.732 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:58 smithi012 podman[153544]: 2024-03-18 15:03:58.463694535 +0000 UTC m=+5.990634507 container cleanup f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T15:03:58.732 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:58 smithi012 bash[153544]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0 2024-03-18T15:03:58.991 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:58 smithi012 podman[153572]: 2024-03-18 15:03:58.732180497 +0000 UTC m=+0.643174807 container remove f0724e03f5299fc0468891a0f5a93938f1997df8df99c26aa7dc04c37db672bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T15:03:59.369 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:58.997637501 +0000 UTC m=+0.018270367 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:03:59.369 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:59.124936976 +0000 UTC m=+0.145569841 container create ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:03:59.369 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:59.369211743 +0000 UTC m=+0.389844609 container init ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T15:03:59.673 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:59.373598055 +0000 UTC m=+0.394230931 container start ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T15:03:59.673 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:59.477883658 +0000 UTC m=+0.498516541 container attach ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-18T15:03:59.982 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:03:59 smithi012 podman[153665]: 2024-03-18 15:03:59.67387741 +0000 UTC m=+0.694510287 container died ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-18T15:04:00.232 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:04:00 smithi012 podman[153692]: 2024-03-18 15:04:00.215262568 +0000 UTC m=+0.536088261 container remove ef432ecbe9db24b4be5d3809278da9674ad66a5baf85c9978f627c397b6bdbe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-18T15:04:00.232 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:04:00 smithi012 systemd[1]: ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-18T15:04:00.232 INFO:journalctl@ceph.osd.0.smithi012.stdout:Mar 18 15:04:00 smithi012 systemd[1]: Stopped Ceph osd.0 for 74f0a062-e535-11ee-95c9-87774f69a715. 2024-03-18T15:04:00.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.0.service' 2024-03-18T15:04:00.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:00.277 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-18T15:04:00.277 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-18T15:04:00.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.1 2024-03-18T15:04:00.491 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:00 smithi012 systemd[1]: Stopping Ceph osd.1 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:04:00.991 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:00 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T15:04:00.617+0000 7fc7109fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:04:00.992 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:00 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T15:04:00.617+0000 7fc7109fd700 -1 osd.1 82 *** Got signal Terminated *** 2024-03-18T15:04:00.992 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:00 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1[129075]: 2024-03-18T15:04:00.617+0000 7fc7109fd700 -1 osd.1 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:04:06.327 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:06 smithi012 podman[153772]: 2024-03-18 15:04:06.027231818 +0000 UTC m=+5.518217268 container died a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-18T15:04:06.685 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:06 smithi012 podman[153772]: 2024-03-18 15:04:06.426061675 +0000 UTC m=+5.917047134 container cleanup a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T15:04:06.685 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:06 smithi012 bash[153772]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1 2024-03-18T15:04:06.991 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:06 smithi012 podman[153792]: 2024-03-18 15:04:06.68598872 +0000 UTC m=+0.653004592 container remove a33f797f55dc9f71dec9efd6abbb3d45e91fbf843678253a88ef531db2cdd68b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:04:07.456 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.013616449 +0000 UTC m=+0.018957054 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:04:07.456 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.145379807 +0000 UTC m=+0.150720406 container create d651c80f0583a26a56588ac9b0b5e4eb9d7c11434c428edf9e7770c34815ee51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2) 2024-03-18T15:04:07.456 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.356454316 +0000 UTC m=+0.361794931 container init d651c80f0583a26a56588ac9b0b5e4eb9d7c11434c428edf9e7770c34815ee51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-18T15:04:07.456 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.359484971 +0000 UTC m=+0.364825577 container start d651c80f0583a26a56588ac9b0b5e4eb9d7c11434c428edf9e7770c34815ee51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:04:07.457 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.456528137 +0000 UTC m=+0.461868744 container attach d651c80f0583a26a56588ac9b0b5e4eb9d7c11434c428edf9e7770c34815ee51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:04:07.741 INFO:journalctl@ceph.osd.1.smithi012.stdout:Mar 18 15:04:07 smithi012 podman[153880]: 2024-03-18 15:04:07.650950608 +0000 UTC m=+0.656291211 container died d651c80f0583a26a56588ac9b0b5e4eb9d7c11434c428edf9e7770c34815ee51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T15:04:08.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.1.service' 2024-03-18T15:04:08.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:08.418 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-18T15:04:08.418 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-18T15:04:08.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.2 2024-03-18T15:04:08.741 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:08 smithi012 systemd[1]: Stopping Ceph osd.2 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:04:09.241 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T15:04:08.787+0000 7f9b553f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:04:09.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T15:04:08.787+0000 7f9b553f6700 -1 osd.2 82 *** Got signal Terminated *** 2024-03-18T15:04:09.242 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:08 smithi012 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2[134739]: 2024-03-18T15:04:08.787+0000 7f9b553f6700 -1 osd.2 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:04:14.581 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:14 smithi012 podman[153986]: 2024-03-18 15:04:14.254581547 +0000 UTC m=+5.602980478 container died f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:04:14.880 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:14 smithi012 podman[153986]: 2024-03-18 15:04:14.688090246 +0000 UTC m=+6.036489194 container cleanup f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:04:14.880 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:14 smithi012 bash[153986]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2 2024-03-18T15:04:15.241 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:14 smithi012 podman[154003]: 2024-03-18 15:04:14.88106955 +0000 UTC m=+0.622819468 container remove f97e19b7fb9e14b69913b53cac9a22a22fb25b5c21768cb231a94a57cee6a68a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-18T15:04:15.591 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.219504506 +0000 UTC m=+0.019266805 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:04:15.592 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.482024602 +0000 UTC m=+0.281786882 container create 4e665208dcd4d22917296f35c37f4ed6fbdcba02904bb320cabf66625ec61d4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-18T15:04:15.991 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.692864599 +0000 UTC m=+0.492626882 container init 4e665208dcd4d22917296f35c37f4ed6fbdcba02904bb320cabf66625ec61d4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-18T15:04:15.992 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.69607068 +0000 UTC m=+0.495832964 container start 4e665208dcd4d22917296f35c37f4ed6fbdcba02904bb320cabf66625ec61d4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-18T15:04:15.992 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.759589263 +0000 UTC m=+0.559351549 container attach 4e665208dcd4d22917296f35c37f4ed6fbdcba02904bb320cabf66625ec61d4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T15:04:16.375 INFO:journalctl@ceph.osd.2.smithi012.stdout:Mar 18 15:04:15 smithi012 podman[154098]: 2024-03-18 15:04:15.993770979 +0000 UTC m=+0.793533268 container died 4e665208dcd4d22917296f35c37f4ed6fbdcba02904bb320cabf66625ec61d4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:04:16.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.2.service' 2024-03-18T15:04:16.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:16.631 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-18T15:04:16.631 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-18T15:04:16.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.3 2024-03-18T15:04:16.912 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:16 smithi100 systemd[1]: Stopping Ceph osd.3 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:04:17.295 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:16 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:16.930+0000 7fb5e31d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:04:17.295 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:16 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:16.930+0000 7fb5e31d1700 -1 osd.3 82 *** Got signal Terminated *** 2024-03-18T15:04:17.295 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:16 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:16.930+0000 7fb5e31d1700 -1 osd.3 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:04:18.888 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:18 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:18.634+0000 7fb5db7bd700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:56.608022+0000 front 2024-03-18T15:03:56.607841+0000 (oldest deadline 2024-03-18T15:04:18.307836+0000) 2024-03-18T15:04:19.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:18 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:18.887+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:19.878 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:19 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:19.621+0000 7fb5db7bd700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:56.608022+0000 front 2024-03-18T15:03:56.607841+0000 (oldest deadline 2024-03-18T15:04:18.307836+0000) 2024-03-18T15:04:20.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:19 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:19.878+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:20.882 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:20 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:20.589+0000 7fb5db7bd700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:56.608022+0000 front 2024-03-18T15:03:56.607841+0000 (oldest deadline 2024-03-18T15:04:18.307836+0000) 2024-03-18T15:04:21.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:20 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:20.882+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:21.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:21 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3[94069]: 2024-03-18T15:04:21.631+0000 7fb5db7bd700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:56.608022+0000 front 2024-03-18T15:03:56.607841+0000 (oldest deadline 2024-03-18T15:04:18.307836+0000) 2024-03-18T15:04:21.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:21 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:21.542+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:22.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:21 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:21.837+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:22.748 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:22 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:22.539+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:22.748 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:22 smithi100 podman[117484]: 2024-03-18 15:04:22.440874169 +0000 UTC m=+5.614090243 container died 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:04:23.044 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:22 smithi100 podman[117484]: 2024-03-18 15:04:22.930857544 +0000 UTC m=+6.104073603 container cleanup 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-18T15:04:23.045 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:22 smithi100 bash[117484]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3 2024-03-18T15:04:23.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:22 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:22.804+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:23.541 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117503]: 2024-03-18 15:04:23.232708847 +0000 UTC m=+0.787097710 container remove 22484c48a3ea96c9773cb3c07255b58e54a4bff3dba5ffe51b248caf489822bb (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-18T15:04:23.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117590]: 2024-03-18 15:04:23.541915999 +0000 UTC m=+0.112321324 container create 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T15:04:23.795 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117590]: 2024-03-18 15:04:23.447717703 +0000 UTC m=+0.018122990 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:04:23.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:23 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:23.572+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:24.114 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:23 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:23.797+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:24.114 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117590]: 2024-03-18 15:04:23.819839719 +0000 UTC m=+0.390245000 container init 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:04:24.114 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117590]: 2024-03-18 15:04:23.823049687 +0000 UTC m=+0.393454973 container start 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:04:24.114 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:23 smithi100 podman[117590]: 2024-03-18 15:04:23.911609343 +0000 UTC m=+0.482014638 container attach 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T15:04:24.431 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:24 smithi100 podman[117590]: 2024-03-18 15:04:24.11457276 +0000 UTC m=+0.684978052 container died 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, CEPH_POINT_RELEASE=) 2024-03-18T15:04:24.682 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:24 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:24.537+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:24.682 INFO:journalctl@ceph.osd.3.smithi100.stdout:Mar 18 15:04:24 smithi100 podman[117616]: 2024-03-18 15:04:24.665763969 +0000 UTC m=+0.545034977 container remove 9558a585dbaebb144646ca2f6047b20831ee0cf23bb16ee2b636eecc6fadda6e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-18T15:04:24.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.3.service' 2024-03-18T15:04:24.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:24.726 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-18T15:04:24.726 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-18T15:04:24.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.4 2024-03-18T15:04:25.035 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:24 smithi100 systemd[1]: Stopping Ceph osd.4 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:04:25.035 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:24 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:24.766+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:25.294 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:25 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:25.034+0000 7f9437750700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:04:25.294 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:25 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:25.034+0000 7f9437750700 -1 osd.4 82 *** Got signal Terminated *** 2024-03-18T15:04:25.295 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:25 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:25.034+0000 7f9437750700 -1 osd.4 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:04:25.794 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:25 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:25.518+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:25.795 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:25 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:25.748+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:26.783 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:26 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:26.516+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:27.044 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:26 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:26.783+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:27.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:27.506+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:27.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:27.506+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:02.207399+0000 front 2024-03-18T15:04:02.207457+0000 (oldest deadline 2024-03-18T15:04:26.907421+0000) 2024-03-18T15:04:28.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:27 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:27.826+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:28.794 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:28.529+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:28.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:28.529+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:02.207399+0000 front 2024-03-18T15:04:02.207457+0000 (oldest deadline 2024-03-18T15:04:26.907421+0000) 2024-03-18T15:04:29.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:28.813+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:29.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:28 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:28.813+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:29.794 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:29 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:29.504+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:54.706393+0000 front 2024-03-18T15:03:54.706471+0000 (oldest deadline 2024-03-18T15:04:21.106786+0000) 2024-03-18T15:04:29.795 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:29 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4[99256]: 2024-03-18T15:04:29.504+0000 7f9433d44700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:02.207399+0000 front 2024-03-18T15:04:02.207457+0000 (oldest deadline 2024-03-18T15:04:26.907421+0000) 2024-03-18T15:04:30.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:29 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:29.842+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:30.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:29 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:29.842+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:30.819 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:30 smithi100 podman[117695]: 2024-03-18 15:04:30.549657443 +0000 UTC m=+5.606169871 container died 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:04:31.169 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:30 smithi100 podman[117695]: 2024-03-18 15:04:30.985164819 +0000 UTC m=+6.041677256 container cleanup 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True) 2024-03-18T15:04:31.170 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:30 smithi100 bash[117695]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4 2024-03-18T15:04:31.170 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:30 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:30.820+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:31.170 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:30 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:30.820+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:31.544 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117713]: 2024-03-18 15:04:31.170035485 +0000 UTC m=+0.616458042 container remove 66948a22ce8ea16aec8c6b62d95d6bb0986ef923a455a1f266c9f4348dbc1143 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-18T15:04:31.840 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117803]: 2024-03-18 15:04:31.58766688 +0000 UTC m=+0.110982790 container create 12e4e8759af8e24c1f742d9aa259e3b0a21bc575392c7b48207d901878e83703 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117803]: 2024-03-18 15:04:31.494708103 +0000 UTC m=+0.018024017 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117803]: 2024-03-18 15:04:31.756804737 +0000 UTC m=+0.280120654 container init 12e4e8759af8e24c1f742d9aa259e3b0a21bc575392c7b48207d901878e83703 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117803]: 2024-03-18 15:04:31.75981179 +0000 UTC m=+0.283127706 container start 12e4e8759af8e24c1f742d9aa259e3b0a21bc575392c7b48207d901878e83703 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:31 smithi100 podman[117803]: 2024-03-18 15:04:31.840031617 +0000 UTC m=+0.363347542 container attach 12e4e8759af8e24c1f742d9aa259e3b0a21bc575392c7b48207d901878e83703 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:31 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:31.791+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:31.841 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:31 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:31.791+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:32.294 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 18 15:04:32 smithi100 podman[117803]: 2024-03-18 15:04:32.047054872 +0000 UTC m=+0.570370776 container died 12e4e8759af8e24c1f742d9aa259e3b0a21bc575392c7b48207d901878e83703 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T15:04:32.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.4.service' 2024-03-18T15:04:32.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:32.721 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-18T15:04:32.721 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-18T15:04:32.722 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.5 2024-03-18T15:04:33.042 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:32.756+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:33.042 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:32 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:32.756+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:33.042 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:32 smithi100 systemd[1]: Stopping Ceph osd.5 for 74f0a062-e535-11ee-95c9-87774f69a715... 2024-03-18T15:04:33.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:33.038+0000 7f8082aa2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:04:33.294 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:33.038+0000 7f8082aa2700 -1 osd.5 82 *** Got signal Terminated *** 2024-03-18T15:04:33.295 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:33.038+0000 7f8082aa2700 -1 osd.5 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:04:34.044 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:33.723+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:34.044 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:33 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:33.723+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:35.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:34 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:34.708+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:35.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:34 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:34.708+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:36.044 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:35 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:35.686+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:36.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:35 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:35.686+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:37.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:36 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:36.712+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:37.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:36 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:36.712+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:37.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:36 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:36.712+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-03-18T15:04:08.001401+0000 front 2024-03-18T15:04:08.001418+0000 (oldest deadline 2024-03-18T15:04:36.201759+0000) 2024-03-18T15:04:38.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:37 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:37.688+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-03-18T15:03:57.099133+0000 front 2024-03-18T15:03:57.099159+0000 (oldest deadline 2024-03-18T15:04:18.199329+0000) 2024-03-18T15:04:38.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:37 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:37.688+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-03-18T15:04:03.400439+0000 front 2024-03-18T15:04:03.400534+0000 (oldest deadline 2024-03-18T15:04:28.001023+0000) 2024-03-18T15:04:38.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:37 smithi100 ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5[104517]: 2024-03-18T15:04:37.688+0000 7f807b08e700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-03-18T15:04:08.001401+0000 front 2024-03-18T15:04:08.001418+0000 (oldest deadline 2024-03-18T15:04:36.201759+0000) 2024-03-18T15:04:38.647 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:38 smithi100 podman[117909]: 2024-03-18 15:04:38.355032571 +0000 UTC m=+5.413391034 container died 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-03-18T15:04:39.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:38 smithi100 podman[117909]: 2024-03-18 15:04:38.821734881 +0000 UTC m=+5.880093387 container cleanup 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T15:04:39.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:38 smithi100 bash[117909]: ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5 2024-03-18T15:04:39.435 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[117925]: 2024-03-18 15:04:39.08136174 +0000 UTC m=+0.723228109 container remove 9978289a0827d128eccc7acb486e3b087dd93b10813e1374523aa5511b7b9eb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-18T15:04:39.735 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.339770494 +0000 UTC m=+0.018041803 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:04:39.735 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.457758598 +0000 UTC m=+0.136029902 container create f023b5cb2689e4f8749b615701e199408913457f00babebbbf1dd9a1045bb237 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-18T15:04:39.735 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.668316866 +0000 UTC m=+0.346588175 container init f023b5cb2689e4f8749b615701e199408913457f00babebbbf1dd9a1045bb237 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-18T15:04:39.735 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.671531011 +0000 UTC m=+0.349802318 container start f023b5cb2689e4f8749b615701e199408913457f00babebbbf1dd9a1045bb237 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:04:39.735 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.735244363 +0000 UTC m=+0.413515667 container attach f023b5cb2689e4f8749b615701e199408913457f00babebbbf1dd9a1045bb237 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-18T15:04:40.045 INFO:journalctl@ceph.osd.5.smithi100.stdout:Mar 18 15:04:39 smithi100 podman[118018]: 2024-03-18 15:04:39.955486655 +0000 UTC m=+0.633757966 container died f023b5cb2689e4f8749b615701e199408913457f00babebbbf1dd9a1045bb237 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-74f0a062-e535-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T15:04:40.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74f0a062-e535-11ee-95c9-87774f69a715@osd.5.service' 2024-03-18T15:04:40.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:04:40.614 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-18T15:04:40.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74f0a062-e535-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T15:05:01.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74f0a062-e535-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T15:05:11.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:05:11.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:05:11.988 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-18T15:05:11.990 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi012/crash 2024-03-18T15:05:11.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash -- . 2024-03-18T15:05:12.026 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T15:05:12.026 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-03-18T15:05:12.027 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi100/crash 2024-03-18T15:05:12.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash -- . 2024-03-18T15:05:12.054 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/74f0a062-e535-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T15:05:12.054 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-03-18T15:05:12.055 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-18T15:05:12.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-18T15:05:12.101 INFO:teuthology.orchestra.run.smithi012.stdout:2024-03-18T14:57:08.759165+0000 mon.smithi012 (mon.0) 246 : cluster [WRN] Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T15:05:12.102 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-18T15:05:12.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[SEC\]' /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-18T15:05:12.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]' /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-18T15:05:12.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[WRN\]' /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-18T15:05:12.318 INFO:teuthology.orchestra.run.smithi012.stdout:2024-03-18T14:57:08.759165+0000 mon.smithi012 (mon.0) 246 : cluster [WRN] Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED) 2024-03-18T15:05:12.318 INFO:tasks.cephadm:Compressing logs... 2024-03-18T15:05:12.319 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:05:12.361 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:05:12.386 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:: No such file or directory 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.oheuui.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.oheuui.log: /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.xtadxo.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi012.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.1.log: /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.xtadxo.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi012.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T15:05:12.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log 2024-03-18T15:05:12.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T15:05:12.393 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-18T15:05:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log: 91.0% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T15:05:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T15:05:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.2.log: 87.7% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T15:05:12.394 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi012.tmscgt.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:: No such file or directory 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi100.rwnrcn.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi100.rwnrcn.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.rutxmj.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.rutxmj.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi100.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6%/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi100.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T15:05:12.398 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.dvtbrf.log 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.dvtbrf.log: 91.2% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T15:05:12.399 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log: 87.7% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T15:05:12.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log: 90.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T15:05:12.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi012.tmscgt.log: 91.1% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T15:05:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T15:05:12.444 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi100.rwnrcn.log.gz 2024-03-18T15:05:12.771 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mgr.smithi012.tmscgt.log.gz 2024-03-18T15:05:12.791 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi100.log.gz 2024-03-18T15:05:13.514 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mon.smithi012.log.gz 2024-03-18T15:05:17.466 INFO:teuthology.orchestra.run.smithi100.stderr: 94.9% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.rutxmj.log.gz 2024-03-18T15:05:17.615 INFO:teuthology.orchestra.run.smithi100.stderr: 94.9% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi100.dvtbrf.log.gz 2024-03-18T15:05:17.781 INFO:teuthology.orchestra.run.smithi012.stderr: 93.4% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-18T15:05:18.542 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-18T15:05:18.854 INFO:teuthology.orchestra.run.smithi012.stderr: 93.5% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-18T15:05:18.896 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-18T15:05:19.849 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-18T15:05:19.851 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-18T15:05:19.851 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m7.476s 2024-03-18T15:05:19.851 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m29.836s 2024-03-18T15:05:19.851 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m1.046s 2024-03-18T15:05:20.074 INFO:teuthology.orchestra.run.smithi012.stderr: 93.5% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-18T15:05:25.615 INFO:teuthology.orchestra.run.smithi012.stderr: 94.9% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.oheuui.log.gz 2024-03-18T15:06:49.479 INFO:teuthology.orchestra.run.smithi012.stderr: 92.7% -- replaced with /var/log/ceph/74f0a062-e535-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi012.xtadxo.log.gz 2024-03-18T15:06:49.479 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-03-18T15:06:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:real 1m37.106s 2024-03-18T15:06:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:user 2m7.580s 2024-03-18T15:06:49.479 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m3.631s 2024-03-18T15:06:49.480 INFO:tasks.cephadm:Archiving logs... 2024-03-18T15:06:49.480 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi012/log 2024-03-18T15:06:49.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:07:07.858 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi100/log 2024-03-18T15:07:07.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:07:11.637 INFO:tasks.cephadm:Removing cluster... 2024-03-18T15:07:11.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74f0a062-e535-11ee-95c9-87774f69a715 --force 2024-03-18T15:07:12.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74f0a062-e535-11ee-95c9-87774f69a715 --force 2024-03-18T15:07:12.840 INFO:tasks.cephadm:Removing cephadm ... 2024-03-18T15:07:12.840 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T15:07:12.856 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T15:07:12.871 INFO:tasks.cephadm:Teardown complete 2024-03-18T15:07:12.871 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T15:07:12.885 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T15:07:12.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:07:12.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:07:12.982 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T15:07:12.993 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T15:07:12.993 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:07:12.996 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:07:13.008 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:07:13.009 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +1774us[+1762us] +/- 111ms 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -787us[ -799us] +/- 50ms 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -74us[ -86us] +/- 81ms 2024-03-18T15:07:13.012 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +1821us[+1821us] +/- 111ms 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -771us[ -762us] +/- 50ms 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -44us[ -44us] +/- 81ms 2024-03-18T15:07:13.014 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:07:13.014 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T15:07:13.025 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T15:07:13.026 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T15:07:13.036 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T15:07:13.145 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T15:07:13.223 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:07:13.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:07:13.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:07:13.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:07:13.284 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T15:07:13.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T15:07:13.307 INFO:teuthology.task.internal:Duration was 2055.637254 seconds 2024-03-18T15:07:13.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T15:07:13.318 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T15:07:13.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:07:13.321 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:07:13.362 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:07:13.368 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:07:13.776 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T15:07:13.777 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-03-18T15:07:13.777 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:07:13.838 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-03-18T15:07:13.839 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:07:13.869 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T15:07:13.870 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:07:13.881 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:07:13.938 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T15:07:13.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:07:13.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:07:14.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T15:07:14.590 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T15:07:14.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:07:14.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:07:14.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T15:07:14.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:07:14.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:07:14.682 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-03-18T15:07:14.711 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-03-18T15:07:14.727 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:07:14.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:07:14.752 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:07:14.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:07:14.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T15:07:14.794 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T15:07:14.795 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi012 2024-03-18T15:07:14.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:07:14.846 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837/remote/smithi100 2024-03-18T15:07:14.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:07:14.885 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T15:07:14.885 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:07:14.888 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:07:14.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T15:07:14.947 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T15:07:14.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T15:07:14.959 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T15:07:14.959 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:07:14.961 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:07:14.973 INFO:teuthology.orchestra.run.smithi012.stdout: 265570 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:07 /home/ubuntu/cephtest 2024-03-18T15:07:14.988 INFO:teuthology.orchestra.run.smithi100.stdout: 265300 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:07 /home/ubuntu/cephtest 2024-03-18T15:07:14.989 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T15:07:15.001 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T15:07:15.026 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T15:07:15.052 DEBUG:teuthology.nuke:shortname: smithi012 2024-03-18T15:07:15.052 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:07:15.077 DEBUG:teuthology.nuke:shortname: smithi100 2024-03-18T15:07:15.078 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:07:15.081 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 14:21:21.435928', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8b+VFjfnKLwEQ6uY3JcaOTZH5yxx6WL+C+zBe1SIW8UO/HnHcyx2nfIhl4/QqVIeuGlyTWAdcypvujXZVRiKs='} 2024-03-18T15:07:15.102 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608837', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 14:21:21.433729', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTRjhLugTLlle1QVsYXcxdanH8C8KI6mwzrbyxYF4QNxlQSXCZlguETbBttISi1gL0qJ06WbHJ7J/XmH25BsSo='} 2024-03-18T15:07:15.142 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-18T15:07:15.143 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:07:15.158 INFO:teuthology.orchestra.console.smithi012:Power off 2024-03-18T15:07:15.158 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:07:15.168 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-18T15:07:15.168 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:15.182 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2024-03-18T15:07:15.182 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:15.192 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-18T15:07:15.205 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-03-18T15:07:19.193 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:19.207 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:19.303 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-18T15:07:19.316 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-03-18T15:07:23.308 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:23.317 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:23.418 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-18T15:07:23.426 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-03-18T15:07:23.519 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-18T15:07:27.430 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:07:27.539 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2024-03-18T15:07:27.640 INFO:teuthology.orchestra.console.smithi012:Power off completed 2024-03-18T15:07:27.745 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2055.6372537612915 failure_reason: '"2024-03-18T14:57:08.759165+0000 mon.smithi012 (mon.0) 246 : cluster [WRN] Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-18T15:07:27.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T15:07:27.833 INFO:teuthology.run:FAIL